Item # 17 - Modifications to the Can. Rating System / Report of the Rating Auditor

[Bob Armstrong]

Item # 17 - Modifications to the Can. Rating System / Report of the Rating Auditor

The introductory post is by Bill Doubleday, Rating Auditor:

Rating System Issues
The CFC Rating system allows players to have a reasonable expectation of the outcome when they play. Underlying the system is the assumption that players have a level of competence which persists. Statistical analyses of individual ratings over decades usually show a rapid rise over the first few years and a slower rise to a peak which persists for decades followed by a slow decline.
Ratings are self adjusting so that higher competence than reflected in the current rating is rewarded with a rating increase and ratings that are too high decrease as a result of tournament results.
In recent years, chess has been promoted in schools, resulting in large numbers of young players learning the game. Typically they begin as weak players with ratings of 1000 or less. With coaching and extensive practice, their competence rises. Much of this practice does not involve CFC rated games. Thus, competence as shown in CFC tournaments sometimes rises rapidly and discontinuously.
If a player begins playing in CFC rated tournaments, a provisional rating is created, based on performance. After a rating is established, changes occur due to transfers of points to or from other players. For a player rated 1000 to rise to 2000, other players have to give up 1000 points. The opponents who lose these points may be playing as well as before, but they lose points to the improving newcomer. This process deflates ratings. Players losing points often complain. There have been examples of sudden losses of 50-100 points.
In recent years the CFC has addressed this deflation by adding points to ratings as a one time boon or bonus reflecting activity or as participation points for each game played. This stopped and reversed the deflation, but did not address the root cause. Two equal players would see both their ratings increase after a drawn game even if there was no increase in skill. The effect was substantial, often 50 rating points per year or more.
A number of solutions to this problem could be considered.
a. Start juniors with higher provisional ratings (say 1500). That way an increase to 2000 would only take 500 points from established players, reducing the deflation.
b. Instead of the redistribution of points as now occurs. Opponents of rapidly rising players could have ratings revised based on their performance rating rather than the outdated official rating. This would require a standard of what constitutes a rapid rise to protect against random variations in performance.
I suggested approach b at the AGM, but would like to do some analyses and simulations to refine it and verify how it would work. Roger Patterson has agreed to work with me on this.

There are other issues as well. The current software gives incorrect ratings for players above 2200 who lose enough points to fall below 2200 – a factor 0f 32 is used instead of 16 as described in the handbook. Also, the program is compiled an undocumented which makes auditing more difficult.

Active ratings are much lower than regular ratings. I suggested adjusting them to bring average ratings to the same level but there was a lack of interest at the AGM. Some people think they should be abolished. I see some value in having them , but is it worth the effort to have two ratings.

Some food for thought.

[Michael von Keitz]

Approach b sounds great, except for the fact that it may represent a significant time investment. What sort of time to completion would Bill be proposing?

I am among those who would be willing to see active ratings disappear. That said, I think a survey of our current members who possess active ratings is in order, as I cannot claim to represent that group.

[Fred McKim]

Totally against touching Active ratings. Here in PEI we have 50% regular, 50% active. Leave participation points on active rating system. Once a year increase players (who have played) ratings who have lower active rating by half the difference, or something like that.

Have recommended applying participation/feedback points for playing juniors. Alternatively, instead of using the pre-event rating of a junior use an average of pre-event rating and performance rating for those whose Rp is 100 points > Ro.

all sorts of ideas.

[Stuart Brammall]

What is the current problem with the system we have?

If it is fast improving juniors that in your opinion are under-rated that you are trying to correct, it seems to me that any mechanism which is used to artificially increase their rating will cause rating inflation---

This is my math intuition talking and I have not done any statistics to back it up, but it seems if you increase a low rated player you also increase all of his future oponents-- insofar as those he loses to will gain more, but perhaps more important, those he beats will lose less. Is this not inflating the pool as a whole?

If someone is really under-rated it does not take long for there rating to correct using the standard system-- and also, those he brings down on his way up should correct themselves quickly as well. Unless they were overrated.

[Fred McKim]

Sorry, but the rating system does not have a built-in way of re-introducing rating points taken from the pool by a player as they improve through the ranks.

[Stuart Brammall]

They don't get taken away, they get re-dristributed. If an old master say 2250 has a bad event and loses to a bunch of kids, the next event, when he manages to dogde the kids, or when the kids have got their ratings up, he should still perform at 2250 and go back up, unless he was overrated to begin with.

It is important to note that rating never indicates a persons chess strength, only their strength relative to the pool. If the pool is getting stronger and you are not you go down, even if your strength is not.

[Christopher Mallon]

I proposed this once a few years ago when I was on the ratings committee, but what about simply saying that anyone under 18, their rating acts as a provisional rating based on their most recent 24 games?

Yes I can think of a couple problems with that, but it's still worth discussing I think.

[Fred McKim]

They don't get taken away, they get re-dristributed. If an old master say 2250 has a bad event and loses to a bunch of kids, the next event, when he manages to dogde the kids, or when the kids have got their ratings up, he should still perform at 2250 and go back up, unless he was overrated to begin with.

It is important to note that rating never indicates a persons chess strength, only their strength relative to the pool. If the pool is getting stronger and you are not you go down, even if your strength is not.

Ratings are meant to be stable, not fluctuating due to other people joining the pool. At least that what's how the CFC and FIDE systems work.

[Rob Clark]

I agree that something needs to be done about the rating system. In regards to Stuart's post, I disagree. If the pool that the master is in gets stronger, than points will be taken from him and passed down to the up and coming juniors with established ratings. The only way to rectify this is for him to take points from another pool. However, coming from way up in Thunder Bay where the nearest large city is 10 hours away, it’s not always a possibility. Also this assumes that their ratings have not been deflated as well, which invariably over time they will. It is a problem, especially with how much emphasis is put on ratings, since this system will inevitably lead to deflation (especially in the small rating pools with many juniors). As for the active ratings, I think they serve a purpose. My club in Thunder Bay holds an active tournament every weekend with great results. Active chess is fun and it’s great that it’s far less of a time commitment than a regular tournament.

[Paul Leblanc]

I have raised the issue of under-rated juniors several times, most recently after the Langley Open last month when five of the juniors had performance ratings 300 to 500 points above their current CFC ratings. There is a great deal of dissatisfaction among the experienced players who get crushed by 1300 rated grade 3 kids whose real playing strength is several hundred points higher.
Now that participation points have been eliminated (a good move), there will most certainly be deflation in the system since players tend to come into competitive chess with an initial low rating and leave with a higher rating. That leaves room to boost up the ratings of under-rated juniors.
Chris Mallon's idea has a great deal of merit but I'd like to see a smaller number of games used to calculate the current rating. Some of the juniors out here are playing only 6-12 CFC rated games per year but are improving very rapidly through their participation in unrated junior events, online chess, coaching and casual play.