September 22, 2015

USATT Rating Inflation

I blogged about this back on June 19, 2014. The subject came up last night during the USATT Teleconference, where we were discussing creating USATT leagues that would be processed with regular USATT ratings, rather than the separate USATT league ratings that are currently used in many leagues. The question was whether we needed to use adjustments. I explained why the rating system would actually deflate without adjustments, but that in the past we've had the adjustments too high, which led to inflation. I've experienced this in the leagues I've run or been involved with, where over and over we've seen the ratings deflate, leading to us putting in an adjustment factor. (At MDTTC, we give bonus points for winning your group.)

Until now, I've always assumed my analysis of this was my own from about 15 years or so ago, and didn't realize anyone else had studied this. (I have a bachelor's in math, and if not for table tennis might have become a math professor…) Here's how I explained it back then.

"If there were no adjustment factor, the system would be deflationary, and the average rating would be dropping. Why? Because the average player improves after his initial rating. Assuming no adjustment factor, let's say that the average first rating is 1200, and that the average player then improves to 1500. That means the player takes 300 rating points from others in the system. Result? Assuming the same number of players in the system, there are now 300 less points distributed among them, and so the average rating goes down - even though the average level of those players has stayed the same. This should be true of any rating system where there's a direct or indirect exchange of rating points."

"Let's assume that the average player instead got worse on average. Then they'd be giving the system points, and so the system would be inflationary."

But last night CEO Gordon mentioned the Elo rating system, which ours is based on. And lo and behold, it said exactly what I had figured out years ago! It includes a segment on Rating Inflation and Deflation. It includes this statement, which matches what I've argued for years, though many have been skeptical of this.

“In a pure Elo system, each game ends in an equal transaction of rating points. If the winner gains N rating points, the loser will drop by N rating points. This prevents points from entering or leaving the system when games are played and rated. However, players tend to enter the system as novices with a low rating and retire from the system as experienced players with a high rating. Therefore, in the long run a system with strictly equal transactions tends to result in rating deflation.”

Below that is a segment on Combatting Deflation, which includes this:

“Because of the significant difference in timing of when inflation and deflation occur, and in order to combat deflation, most implementations of Elo ratings have a mechanism for injecting points into the system in order to maintain relative ratings over time.”

Regarding USATT ratings, my impression is that there was inflation from the time it started in the mid-1970 through the 1990s, but it seems to have slowed in the last 15 years or so, with the upwards adjustments roughly matching the built-in deflation. I don't know if there was a change to the adjustment factor, a natural convergence to a stable norm, or what. I've seen the same deflationary problem in proposed USATT rating systems that did not have an adjustment factor.

In the Zone: Training Emotional Skill in Table Tennis, Parts 1-3

I previously linked to part 1. Here's all three.

  • Part 1: Introduction and the Nature of Emotional Skill
  • Part 2: Ten Attributes of Poised Players, What About Us?, and Diagnose
  • Part 3: Intervention and Changing Goals

Li Xiaodong on Serve and Attack

Here's the video (25:56). I linked on Sept. 8 to his earlier lecture, "Li Xiaodong on Serving" (23:28). Li Xiaodong is 22 years coach of Chinese National Team and 12 years coach of Beijing Team. He was also Head Coach of Chinese Women Team. Now he is Deputy Head of Technical Studies Committee.

Akron, Latest Home as Demand Grows for Development Courses in United States

Here's the ITTF article about the recent ITTF Coaching Course in Akron, Ohio. It was organized by Samson Dubina and run by Richard McAfee.

Liang Xu Wins 2nd Annual Knoxville Secret City gold Dollar Upset Open

Here's the article (by Jude Lam) and pictures.

Zhang Jike Tribute

Here's the new video (5:50). It's in Chinese, but you get to see a lot of footage.

Court Table Tennis?

Here's the video (1:27) with Adam Bobrow and the Singapore Team at a bus station in Czech Repubic.

Water Pong

Here's the video (13 sec)!

***
Send us your own coaching news!

You should read RatingCentral's method for rating. It is much more accurate and takes the time since last played into account which IMO makes it much more accurate-- especially for Jr players.

In reply to by despreston

Does it have rating adjustments to avoid deflation? (I was familiar with the system many years ago, but it's been a while, plus there might have been changes.) There were pluses and minuses to that system. If I start getting into that, there'll like be a huge debate. We've been through that many times, and I don't have time or interest in going through that again. 

In reply to by Larry Hodges

There is no need for adjustments. There is a second value assigned to the player that tracks how long it has been since they played. The higher this value, the less certain we are of the accuracy of the rating assigned to that player. The second value is what helps determine how many points you win/lose from a match.

In reply to by despreston

The number you refer to has nothing to do with rating deflation or inflation. Having a number that tells you how accurate the rating is relative to others in the system is not the same thing. 

In reply to by Larry Hodges

Sure it does. It allows us to go about rating players without having to rely on pumping points into the system. The reason (at least from what I see) for adjustments is because players are so underrated that it would be unfair to other players to NOT adjust their rating. Reasons players appear in tournaments highly unrated: kids that learn quickly; coaches that hold players out of tournaments until they are significantly underrated; players that go a long time without playing a US tournament (busy training in China, etc). Most of this is based on the time between recorded tournaments. Its the standard deviation that helps determine how many points you should lose and how many points should be given. There are further solutions that could be taken with this system as well-- giving juniors a higher standard deviation is one of them.

In reply to by despreston

Adjustments are done specifically for the reasons given in my blog and the linked article - to stop rating deflation. They tend to go to under-rated players, but that's actually arbitrary to what's needed to stop deflation, which is to pump points into the system to keep it stable. If you could calculate how much a system deflates in a year, you could get the same result by simply adding that many points to everyone's rating each year. (My guess is it's never been more than 5 points a year, and it's less if anything now, but that adds up over time. See the link to my earlier blog on this above.) 

Keeping track of the time between tournaments only gives an indication that the rating may be less accurate, and so the SD goes up. While this could affect the rate of deflation, it works both ways. You assume players who don't play for a time are under-rated, when it's probably just as likely that players stop playing for a time, and when they come back, might be out of practice. But either way, at most it would change the rate of deflation. (Coaches holding players out of tournaments so they can be under-rated is generally rare, though I've heard of the accusations against a specific club in California. Any good coach knows you need regular tournaments to maximize improvement. It's an isolated case out of a large country.) 

The system can stay internally accurate while deflating as a whole. Players will still tend to improve, and so be under-rated. They'll then, on average, beat players and get a more accurate rating, while other players he play will go down more than they should, due to his being under-rated. This leads inevitably to deflation, unless there is a mechanism to stop this. I'm tempted to cite data and findings from long ago, but just don't have time and interest to get involved in another long ratings debate - too many other priorities. 

By adjustments, do you mean the points that are injected into the system when a player stands to gain more than 50 rating points in a tournament? It would seem hard to analyze the overall effect of this because players who gradually improve would not inject points, whereas sudden improvements would add points.

Or are there some other kinds of adjustments?

In reply to by dhill

correct, the rating adjustments in the USATT rating system are for players who gain more than 50 points. It does make analysis tricky. 

I second the Ratings Central system, it appears to be much more accurate than USATT ratings from what we see in the leagues out here in Portland, OR.

It is not a zero-sum system like elo, so it is not 'naturally' deflationary in the same way that that system is.  

The benefits of it are that:

Players who have not played in a while have high deviations and therefore don't affect the ratings of players with lower deviations as much if there is an upset in either direction.

Ratings += are not fixed values, they move based on relative ratings and deviations of both players.

The ratings get more accurate the more that a person plays (if they play a variety of other players).

The downside is that if players have not played in a while, their ratings will vary wildly for a few matches, and if only take wins or losses over those few matches, their rating can move by hundreds of points.  

Here is their page on how the system works:

http://www.ratingscentral.com/HowItWorks.php

 

I highly recommend that you try it out for your Capital Area League.  We input the data for our matches in leagues and tournaments out here every week and it makes it super fun.  Players get emails when the events are posted, and those ratings are used for placement in the weekly tournaments and leagues that we have.  I've never heard anyone complain about the accuracy of the system in the many years it's been used.