CFC Rating System Issues

[Fred McKim]

I have been asked by Tony Li to post the following, as he is waiting for his account to come through

"Hi Paul,

Here are the 2 big ticket items I have brought to your attention in the past on behalf of the Kitchener Waterloo Chess Club. If elected Ratings Auditor, I will prioritize discussing these issues with active organizers. I would like to hear anyone's thoughts on this forum or through email at kwchessclub@gmail.com.

1. Increase the maximum number of regular rated games per day to 6 for U1200. This is a necessary condition for double round U1200 in the common 3 games Saturday / 2 games Sunday format.

Most potential CFC members are U1200 today. U1200 players don't tend to use their time, justifiable so - it is debatable whether they could make better decisions in 2 hours than in 1. Furthermore, the playing hall empties with an hour. Double rounds of 1 hour sudden death will be a huge increase in the value proposition for this group.

I believe a valid concern is fatigue. But professionalism at the board is everyone's own responsibility. Adults show up at weekday tournaments after long working days, and the CFC rates those tournaments as Regular.

Recently, the 2nd COVID adjustment acknowledged that U1200 is a different group of customers. Hopefully, we can provide this group more playing time and more scoresheets to analyze by rating up to 6 regular rated games per day.

2. Increase the junior U2200 k factor to 48 while maintaining the master k level at 16 for everyone.

The CFC recognizes that players under 2200 could improve quicker than players over 2200, by having double the k factor (32 vs 16). However, this is a zero-sum game and will lead to rating deflation in an improving player pool. Given that OTB may be down for a year while many children have lower schoolwork and some "turning pro", it will be a huge issue in 2021.

The performance bonus system you introduced will help a lot of players in an open tournament. But in sectioned tournaments, only a few players will get the bonus. A higher k-factor such as 48 for juniors may be an answer.

FIDE has had a higher k factor for juniors for a number of years, and the only loud criticism came from a misunderstanding. A junior had two outstanding tournaments during a rating period, and the rating gain from both tournaments were calculated based on his initial rating. Had the second tournament been rated reflecting his participation in the first event, the final result would have been reasonable.

Regardless of age, improvement beyond 2200 rarely happens away from OTB tournaments and a common k factor seems reasonable."

Tony Li

[Kevin Pacey]

Here's a CFC rating system issue that may be worth revisiting:

Possibly have a USCF-style rating floor (e.g. if you're ever 15xx you can't go below 1300 ever) - fights sandbagging & may slow rating stagnation/decline of individual players (good for the CFC's business, in spite of any possible slight inflation to the rating system!?).

[Pierre Dénommée]

The Glicko2 rating fight sandbagging by adding a standard deviation to the rating. Ratings with higher uncertainty also change faster. https://www.nufflytics.com/post/ccl-glicko-ratings/

[Kevin Pacey]

Interesting. Fwiw, the wiki for that rating system mentions that the Australian Chess Federation (comparable to ours?) now uses a slightly modified form of Glicko2:

https://en.wikipedia.org/wiki/Glicko_rating_system

[Paul Leblanc]

I have been asked about rating floors before.
First, are you sure there is a problem that needs to be solved? How prevalent is "sand-bagging" and is it worth distorting the rating system in an attempt to prevent it?
Rating floors produce artificial ratings. I guess my rating floor would be around 1900 but I actually play chess at the 1750 level. I don't see any benefit to rating my games as if I was a 1900 player.

[Paul Leblanc]

Tony, these are interesting ideas.
I don't have any objection to 6 games per day for U1200 players. The only downside would be if there was at least one game in each round where both players used up their entire hour. That would lead to a 12 hour day, not counting breaks between rounds. But that's an organizational issue, not a rating issue.

The K factor issue is a little more complicated. I'm not sure if the CFC database has a reliable record of all player's ages. Doubling the K factor for juniors whether their rating is 200 or 2199 somehow seems a bit bold. Maybe a cut-off at 1000 or 1200 would make more sense if we adopted this idea. I'm also not sure it would be a good idea to have both a doubling of the K factor and our existing bonus point system. We're already getting some young players gaining 200+ rating points in a single tournament. Do we want to see 400+ points?

[Vladimir Drkulec]

I would not have a problem with 400 point gains if they resulted in more accurate ratings but again I am not sure that there is a big problem at the moment with face to face otb play on hold.

[Aris Marghetis]

I have been asked about rating floors before.
First, are you sure there is a problem that needs to be solved? How prevalent is "sand-bagging" and is it worth distorting the rating system in an attempt to prevent it?
Rating floors produce artificial ratings. I guess my rating floor would be around 1900 but I actually play chess at the 1750 level. I don't see any benefit to rating my games as if I was a 1900 player.

Hello Paul, I relatively agree with you. If I may suggest, another reason for a rating floor, is that it very well might keep some declining players more active.

I understand that, by definition, that'll inject a math corruption, but from a respectfulness viewpoint, I wouldn't be against re-consideration of rating floors.

Of course, I want to know your expert opinion.

[Kevin Pacey]

I don't know how frequent sandbagging is these days - although that's possibly the main reason why the USCF adopted a rating floor in the case of the US (either their knowing the frequency there, or for trying to keep their members happier, if many said members could have suspected heavy sandbagging).

Personally I'm more for Aris' offered reason - it could be good for the CFC's business to keep declining (or stagnant) players happier, as I also alluded to earlier. A good rating can mean a lot to players, even it's just for [self-]respect/bragging rights reasons, never mind e.g. for qualifying for seniors events. I know of one Ottawa player who retired from chess decades ago after reaching his lifelong goal of a 2200+ rating, presumably for fear of losing that minimum 2200 rating after (only a couple of years ago I gave some half-serious thought to doing the same thing myself, i.e. for that reason, but instead came back to chess and promptly lost some games, to an underrated coached junior, and an underrated young adult - later my city fouled up the transit system to get me to my club and back in the late evenings, but that's another story).

For those who are sticklers for near-mathematical precision/perfection for the CFC's rating system, the alternative measure mentioned above by Pierre (that the CFC switch to Glicko2 rating system) might also satisfy Aris' and my stated desires to some extent as well - albeit with heavy changes to the current CFC rating formulae being necessary.

[Tony Li]

1. 6 regular games per day for U1200

Can we move this to a motion?

2. Varying k-factor k=48 for 800 graded to k=32 for 1600

Having a varying k-factor by rating makes sense. However, it would ideally not be a cut-off but graded over a wide rating range e.g. k=32 at 1600 linearly graded to k=48 at 800. Having a section cut off at the same point as the k-factor cutoff would lead to opponents having the same k-factor, which defeats the purpose.

3. Players gaining 200+ rating points in a single tournament

My personal observation is that some players earn too many bonus points in 7+ round tournaments, due to the fact that the threshold vs number of rounds is too flat. 29-point threshold for 5 rounds seems reasonable, but 39-point threshold for 9 rounds has to be too little. I would be interested in looking at the data to figure out if we can decrease the "13" but make the order on number of rounds higher than square root.

4. 2-Year 200-point floor?

A lifetime floor is unscientific and appears to be a last resort solution USCF adapted to combat sandbagging. Given that Canadian tournament don't tend to have large prizes, it doesn't seem necessary. However, as Aris mentioned, floors could encourage participation by seniors. A 2-year 200-point floor, for example, would still be scientific in the vast majority of cases.

Furthermore, a floor/insurance could be very valuable when OTB returns in 2021. An example would be no one can lose more than 100 points over the first 6 months of OTB.

5. Glicko

There are 2 drivers of higher k-factor ("rd") in Glicko - streaks and inactivity. The current bonus system already addresses streaks. The inactivity component is the tricky part because currently everyone in the rating pool is inactive. This is where identifying juniors as the inactive ones is beneficial because they are the ones likely to be improving away from OTB.


I would not have a problem with 400 point gains if they resulted in more accurate ratings but again I am not sure that there is a big problem at the moment with face to face otb play on hold.

6. 2021 OTB Shock

Hi Vlad,

The return of OTB in 2021 will be the largest-ever one-time increase in the strength of the playing pool.

The quick incentive was a great idea but so far it is mostly juniors playing each other. Hoping to see more adults participate at those events - maybe the solution is to have a floor there? There also seem to be a dearth of 2000+ players there. Maybe the CFC can waive the rating fee on players over 2000, allowing organizers to offer them free entry?

[Paul Leblanc]

I remembered another detail about the limitation to 4 games per day for regular rated tournaments. When we updated the conditions for regular rating a few years ago, we were faced with a few junior organizers who had been submitting fast time control events for regular rating, contrary to the rules. When I started auditing them, there was considerable push-back and there were cases where they simply stopped advertising the time controls so that I could not enforce the rules.
One way to spot fast time controls was to look for one-day events with lots of rounds.
This is less of a problem today but I think it is important to be vigilant.
There is always pressure to water down the requirements for regular rating. We are seeing this now with online chess. I agreed to the COVID-19 bonus and I think it is a sensible thing to do but the original concept that was presented to me was significantly more generous.
There is also always pressure to pump up the ratings of "under-rated juniors". The bonus system was introduced primarily to address this issue and it has been working quite well.
It took Bobby Fischer 5 years to get good. Today, many think that a single tournament is enough to identify the next Bobby Fischer.

[Aris Marghetis]

I remembered another detail about the limitation to 4 games per day for regular rated tournaments. When we updated the conditions for regular rating a few years ago, we were faced with a few junior organizers who had been submitting fast time control events for regular rating, contrary to the rules. When I started auditing them, there was considerable push-back and there were cases where they simply stopped advertising the time controls so that I could not enforce the rules.
One way to spot fast time controls was to look for one-day events with lots of rounds.
This is less of a problem today but I think it is important to be vigilant.
There is always pressure to water down the requirements for regular rating. We are seeing this now with online chess. I agreed to the COVID-19 bonus and I think it is a sensible thing to do but the original concept that was presented to me was significantly more generous.
There is also always pressure to pump up the ratings of "under-rated juniors". The bonus system was introduced primarily to address this issue and it has been working quite well.
It took Bobby Fischer 5 years to get good. Today, many think that a single tournament is enough to identify the next Bobby Fischer.

I agree with the sentiment that we seem to often consider over-rotating to "fix the problem" of "under-rated juniors". Whereas I of course support not slowing down the progress of rapidly improving players, I feel there is much to be said for the developmental value of battling upward through "plateau after plateau".

[Tony Li]

I would like to thank Paul for his many years of service to the CFC. I am not running against him. The CFC needs every current and future volunteer to work together in order to reach the 10,000 member goal.

Here are my ideals - The CFC rating system should do the following and no more:

Track improvement or deterioration in playing strength.
Be mathematically sound.
Only as last resort, take inorganic action in order to avoid significant inflation/deflation.

The CFC rating system should not:

Need to be protected except against unsound rating mechanics such as participation bonuses.
Lag the strength of an improving player because that player should "earn it".
Discourage participation with mathematically unsound clauses such as the maximum number of games per day.
Need performance bonuses, when the organic mechanisms function well. In particular, the bonuses are excessive for tournaments with many rounds because the bonus threshold barely increases with number of rounds.

Issues With the Current System

1. Lack of a supporting anchor of stable adult players. We have fewer than 1000 active players, and most of them are improving juniors or declining seniors. Ratings will drift randomly until we get enough players with stable playing strength.

2. The bonus system is deflationary at the lower end and inflationary at the upper end. This is because it pumps the same number of points to players of all strengths to reflect their improvement (with a lower bonuses for over 2200 that is consistent with the lower k factor). However, improvement occurs much more at lower ratings. This has resulted in inflation at higher ratings (obvious by comparing CFC spread vs FIDE spread), and deflation at lower ratings.

3. The current rating system has no mechanism to deal with the improvement in the player pool's playing strength that happened and will happen while OTB chess is shut down for 1-2 years, and is not ready for the shock when OTB chess returns.

Proposed Solutions

1. Embrace the real core of the future CFC- adult players and stabilized juniors rated between 1200 and 1600. We can do this by the following:


Provide them value for their weekend. 5 games is not enough. Segregating them into quick rated games is not enough. We should provide them double rounds - 10 games in a weekend, at 1 hour time time control. The current system does not allow this by arbitrarily restricting the maximum number of games per day to 4. We should discuss removing the restriction, particularly at lower ratings (e.g. 1200 or 1400 cutoff), as the 1 hour minimum time is enough for predicting players' strength.
Allow them to see their improvement by starting them off at their real strength. Increase the maximum reduction in the provisional rating formula from 400 to 700 (a la FIDE). Right now a player with strength of 800 can walk into a club of 1400-1600s, lose all their games, and start with an 1100 rating. When they work hard and get to 1100 strength in a few years, their rating will barely change. Hard work and lack of visible improvement is hardly a recipe for membership renewal.
2. Remove the bonus system, and increase the k-factor for juniors. This means that points are only pumped into the system when juniors outperform adults relative to their ratings. Simple and organic. No need to calibrate the threshold for bonus or multiplier for number of rounds in a tournament.

3. Introduce a 12-month, 100-point rating floor when OTB reopens. Since a 100 point decline in playing strength is unlikely over a 1 year period, this should provide stable players protection against extremely underrated players when OTB returns. This will be a temporary measure to fight rating deflation.

[Aris Marghetis]

Tony, at first read, your proposals seems geared more towards juniors, vs. the entire spectrum of Canadian players. And I just don't know whether to believe statements like "and most of them are improving juniors or declining seniors". I don't know whether that's true or not, but intuitively, there seems to be many legitimate chess players between the ages of 20 and 55, some improving, some declining, some not moving around much. Anyway, as important as it seems to yourself to recognize rapidly improving juniors, issues like class rating floors (it seems to work in the States) could make a huge difference in player retention?!


Regardless, thank you for your passion and interest, which is great for the CFC!

[Tony Li]

Tony, at first read, your proposals seems geared more towards juniors, vs. the entire spectrum of Canadian players. And I just don't know whether to believe statements like "and most of them are improving juniors or declining seniors". I don't know whether that's true or not, but intuitively, there seems to be many legitimate chess players between the ages of 20 and 55, some improving, some declining, some not moving around much. Anyway, as important as it seems to yourself to recognize rapidly improving juniors, issues like class rating floors (it seems to work in the States) could make a huge difference in player retention?!

Regardless, thank you for your passion and interest, which is great for the CFC!

Hi Aris,

This could vary by geography but you might be thinking of Ottawa which has a great chess culture and has managed to maintain more adult chess players. In Kitchener, we tend to attract over 80% juniors/seniors, and this appears to be the mix in GTA as well.

I would not be in favor of a lifetime rating floor as sandbagging is not yet a problem in Canada. I would be interested to examine the data for whether a 2-year, 200-point rating floor would be a good practical solution. I would also be in favor of a lower k-factor for seniors.

My main idea for player retention is more rounds for lower rated players (more so for juniors and new adult members). I myself appreciate longer time controls - at my last weekender, I played over 13 hours on Saturday over 3 rounds - on Sunday morning the TD jokingly apologized for all the delayed round start times and said he will charge me a higher entry fee next time because I was so getting so much out of it. But 5 rounds over a weekend is a terrible value proposition for 1200s who only get about 5-7 hours of game time. I discussed this issue with Hal a few years ago and he brought up the idea of double rounds. Last year, Egis in Aurora began running a one-day tournament for the lowest section as part of a weekender. We need more ideas to keep this group engaged.

Specifically focused on seniors - they could benefit from more rest in between rounds. A lot of seniors are complaining about blundering. It would make sense to set the time control so that for players have at least 1 hour in between rounds. It may involve shortening the time control a little, but seniors need the rest between rounds.

[Egidijus Zeromskis]

I support Tony on the item to allow to rate standar junior tournaments with more than 5 rounds per day and shorter time controls. I don't believe that it would ruin the standard rating system. However it would attract more young players who would reach their actual rating much faster.


On other items like changing the math -- sorry too hot here to think if they have merit and would make anything better. I don't see that the rating system is broken at all.

[Tony Li]

I support Tony on the item to allow to rate standar junior tournaments with more than 5 rounds per day and shorter time controls. I don't believe that it would ruin the standard rating system. However it would attract more young players who would reach their actual rating much faster.


On other items like changing the math -- sorry too hot here to think if they have merit and would make anything better. I don't see that the rating system is broken at all.

On average the rating system is fine. But looking at Roger Patterson's chart suggests to me that rating spreads are current 600 points instead of 400 - this is a natural effect of the bonus systems which reward higher rated players more vs their slower improvement.

For 2021 the system will have a shock because games haven't been for so long while playing strength would have increased dramatically in many pockets.

http://www.victoriachess.com/cfc/stats/expected.JPG