Improving the CFC Rating System

[Bob Armstrong]

CFC Secretary, Lyle Craver, has posted a cursory summary of the Agenda Item 7 thread under " Rating Auditor Report " in the confidential 2011 Fall Governors' On-line Meeting. The Minutes/Summaries will be posted in due course.

But since members' have a particular interest in the rating system, one of the prime benefits of CFC membership, I am posting a " beefier " summary I did personally, in case members may have any comments now:

CFC 2011 Fall Meeting – Agenda Item 7 – Rating Auditor ( Paul Leblanc ) Report – Unofficial Thread Summary


1.The Average CFC’er. The average rating of all CFC members active in the past 12 months is 1190.

2.Bonus Point Formula. The existing bonus point formula needs to be replaced with a model that targets truly exceptional performance. Paul hopes to bring a motion at the 2012 Winter Meeting.

3.Rating Software Issues. There are a few errors in the rating software that will be corrected.

4.Rated All-Junior Events -Time Control. Even all-junior events must be at least Game/60 to be “ Regular “ rated. “ Active “ junior tournaments are still being sent in and not caught and rated “ regular “. However some thought “ regular “ rating junior “ active “ tournaments encouraged participation, in a short time, had little effect on over-all ratings, produced future adult members, and that all-junior events will now cease being CFC-rated at all. Some said Juniors were uninterested in “ active ratings “, whereas others maintained they were valuable, and CFC is poorly marketing them. Does CFC prefer juniors show up at their first adult event with no regular rating, so their provisional rating can be set against the skill of their adult opponents? And in any event, making the junior time control 60 minutes will affect nothing in regard to ratings since the junior games will still be done in 20 minutes no matter what the time control and their interactive rating changes will still be identical. A solution to the problem might be that the active rating (or rename it junior rating) be for junior-only tournaments and the regular rating be for adult tournaments where juniors played – though then this removes active ratings from the adult purview, and some see “ active or rapid “ tournaments as a good marketing tool to encourage later regular tournament play. This system may negatively affect the YCC qualifiers, since they are usually 30 minute games.

5. Under-rated Juniors. Juniors start off low usually, and increase strength fast, but the rating system doesn’t keep up. So juniors win games against higher-rated players ( usually adults ), against whom they should not be winning as many rating points as the system awards them. This is a serious concern – felt other changes will eliminate or at least minimize this problem. The “ regular “ rating of all-junior “ active “ tournaments, greatly exacerbates this problem in Paul’s opinion. One suggestion was for all U 12 players ( or age based on statistical analysis ), they have a continuous “ provisional “ rating, based on their most recent 24 games, to allow faster rise in their ratings. Also, putting a floor of 800 or 1000 on the CFC system might help ( European model ). Another proposal dealt with using under-rated junior ( however they may be identified ) “ performance “ ratings: rating changes are calculated based on junior’s performance rating ) not their current rating ):

Example

Middle aged guy (1800) loses to superstar kid (1300).
But for the tournament, superstar kid has performance rating of 1800.

Current system - Middle aged guy loses 30 rating points.
Proposed system - Middle aged guy loses only 16 points.
The superstar kid still gains 30 points under either systems.


6. General Rating System Issues. Foreign players ratings become stale and inaccurate, but there is no easy answer to this.

Any discussion on any of these issues would be welcome.

Bob A

[Vladimir Drkulec]

This is only a quibble but in your scenario of the 1300 player versus the 1800 player the loss is closer to 32 points than 30.

[Bob Armstrong]

Hi Vlad:

I thought the system capped lost points in a game at " 30 pts. "?

Bob A

P.S. The example was from one of Bob Gillanders' posts.

[Fred McKim]

Hi Vlad:

I thought the system capped lost points in a game at " 30 pts. "?

Bob A

P.S. The example was from one of Bob Gillanders' posts.

That was under the "old" formula that had one rating point for every 25 point rating difference (this can still of course be used as an estimate).

With the new 'FIDE' table we use, yes you could lose 32 points. Check out the rating system from the Ratings section on the web site or in the handbook (they are identical).

[Bob Gillanders]

Hi Vlad:

I thought the system capped lost points in a game at " 30 pts. "?

Bob A

P.S. The example was from one of Bob Gillanders' posts.

Hi Bob,

It used to be capped at 30 points until they modified it (many years ago) to use the stupid expectation chart. I just looked it up, and the exact amount would be 30.72 points, so Vlad is correct.

I like to use the older system, just nostalgic I guess. It does give a very good approximation without having to look up the table.

I see Fred types faster than I. :)

[Christopher Mallon]

The chart is actually based on a formula. I looked it up in 2005 when investigating a revamp of the rating software - a formula being much easier to use than a chart for a computer program.

[Fred McKim]

The chart is actually based on a formula. I looked it up in 2005 when investigating a revamp of the rating software - a formula being much easier to use than a chart for a computer program.

Roger Patterson came to that same conclusion. The Rating program currently uses the chart, but could use the formula when it's rewritten some day.

[roger patterson]

The chart is actually based on a formula. I looked it up in 2005 when investigating a revamp of the rating software - a formula being much easier to use than a chart for a computer program.

Actually it's not. The formula often used is 1/(1 + 10^(delta R/400)). This agrees with the table pretty closely but differs by as much as 0.015 for certain rating intervals. The table is the result of definition that cannot be written in closed form (as explained to me by Jeff Sonas)l

[Christopher Mallon]

Actually it's not. The formula often used is 1/(1 + 10^(delta R/400)). This agrees with the table pretty closely but differs by as much as 0.015 for certain rating intervals. The table is the result of definition that cannot be written in closed form (as explained to me by Jeff Sonas)l

I was of the understanding in 2005 that it was more due to rounding errors. I'd have to look through old emails to figure out more.

[roger patterson]

I was of the understanding in 2005 that it was more due to rounding errors. I'd have to look through old emails to figure out more.

that was my first impression too but a difference of 0.015 is outside rounding error which led to my further inquiries.

[Christopher Mallon]

Well the argument remains that it is in fact the table which is inaccurate, not the formula.

[roger patterson]

Well the argument remains that it is in fact the table which is inaccurate, not the formula.

no... the formula I gave is a closed form expression which is an approximation of the [non closed form] result (and a pretty good approximation but still an approximation). The table is the one that is [more] accurate.

[Kevin Pacey]

Here's a belated anecdote about CFC rating reform around this time, fwiw. Around 2007-2010 I studied hard, especially in late 2007 (after not renewing my CFC membership [I did so less than a year later]), with mixed results for a couple of years to show for it, but, at the age of almost 50, in 2010 I obtained a regular CFC rating of just over 2400. CFC ratings were (IMO) still generally deflating (in Ottawa everyone closest to me who was still playing were sub-2200, including the many who were 2300+ not many years before). I soon noted in passing on chesstalk a couple times, with some pride, that I had managed this achievement at my advanced age (forgetting I had been 2400+ regular USCF in my youth, after 9 games). Some may have taken it as sheer unadulterated (even irksome) boasting.

So, just perhaps not by chance, Roger soon did a study of about 10 players (including my name in it) and posted a graph to show that compared to where he was (B.C.), ratings were deflated compared to where I was (Ottawa, and some other places were thrown in too), as even he was rated 'less than some of his former rating peers' (masters) in Ottawa (I did point out on chesstalk that all in Ottawa were now under 2200, too, except for me, after hard study, but got no acknowledgement on that point).

Well, it did not take long for the CFC rating system to be changed once Roger got things moving - one step the CFC took was to axe participation points, but in return, modified the way bonus points are awarded (sometimes now generously). Whatever the reason (I suspect discontinued study habits, and some new and/or young opponents), my rating nose-dived quickly in the later stages of 2010, well before the new rating system clearly had made any real effect on most people's ratings by itself, IMO.

But, my irksome boasting (it had seemed) was apparently not in vain, as Roger did a very good (and IMO badly overdo) job of correcting the CFC rating system, suffering due to overall deflation as I think it still was. One regret I have is that participation points, if continued in some way, might have been good for the CFC's business, by allowing them to slightly inflate many players' ratings (and egos) and also by arguably encouraging people to play in tournament games more often (so as to get more participation points and potentially higher ratings). I also think a USCF-style rating floor would be good for business, and help fight any potential sandbagging in tournaments.

[Aris Marghetis]

... I also think a USCF-style rating floor would be good for business, and help fight any potential sandbagging in tournaments.

So at first thought, I liked the idea of rating floors. I thought that more people would too. I figured it was a good way for older players to keep playing in rating categories that had a higher proportion of adults than lower rating categories. For example, one is more likely to play more adults in U1900 than in an U1600.

However, when I floated a similar idea once (I recollect it was highest rating in the previous two years), I received quite a bit of pushback. The reasoning, which I completely understand, is that if a player is declining, but we keep him/her above a rating floor, then they'll often lose every single game, which is no fun at all.

I'm interested in what others think. Would you put more value in the "prestige comfort" of playing in the section you used to play in for years, or would you more appreciate having mostly truly competitive games? If there's other angles I haven't thought of, feel free to post them too. This would be a big change in our system.

[Kevin Pacey]

Players do decline in terms of their invisible so-called rating ceiling (not a term used in any rating system, afaik), dependent on such things as talent, age and study, as they get older. In Secrets of Practical Chess, GM Nunn claims most players [never] get anywhere near their rating ceiling [whatever their age at the time] - his point is that they don't study enough or, more according to his point of view, don't do so in the right ways. Perhaps many who object to the idea of a rating floor due to the reasoning you gave are unaware of Nunn's point of view - in which case maybe a rating floor is bad for business, if most people feel that way. If they were to ask me, I'd say regardless of age, unless their health is poor, improved study should help them win games even if there is a rating floor introduced. I'm also not sure how they can lose every game if they don't study, assuming they play in open tournaments at least now and then. [edit: what might be less fun to some such objecting players is if all of the cash-prize weekend tournaments in their area are organized into rating sections, such as in Ottawa, where we are - the players I'd think would tend to care most about winning the odd cash prize, and would care less about open tournaments if these have no such prizes, as is generally the case with open tournaments (or round-robins) at the RA club in Ottawa; again my suggestion to them would be to study better (though local Ottawa organizers, for example, also could provide more cash prize open tournaments in case a rating floor is adopted by the CFC)]

The pushback I got (well before this post) against a rating floor (I guess it would be similar for participation points) is that at least some chess players (and with them, CFC officers) are such sticklers for perfection and truth that they cannot bear the thought of a rating system even temporarily slightly inflated (subject to correction at some later date, if getting out of hand), however good a little inflation of their rating may make them feel when someone unaware of the inflation asks or looks up what their rating is.

[Kevin Pacey]

I've edited my previous post, for any who may have missed that.