7. Rating Auditor Report (Paul Leblanc)

[Lyle Craver]

This section is reserved for our ratings auditor Paul Leblanc

[Paul Leblanc]

It has been an interesting 3 months since I accepted the position of Rating Auditor at the AGM. Before I begin my report I'd like to acknowledge the tremendous support I have received from Fred McKim and Roger Patterson. Roger has spent countless hours developing analysis tools to monitor the rating system and model potential changes. Fred has been a valuable link to the CFC office and a great resource to help me understand the background of the issues I am wrestling with.

Overall Comments on the Rating System. The average rating of all CFC members active in the past 12 months is 1190. This figure has been fairly consistant for the past 7 years (1170 plus or minus 35). A slow downtrend from 2004 to early 2009 has been replaced by a slow uptrend but we have not regained the 2004 level. Factors at play include the elimination of the old bonus point system, the introduction of the existing bonus point system and the introduction and subsequent elimination of participation points.

Bonus Point Formula. I am convinced that the existing bonus point formula needs to be replaced with a better model. The biggest flaw is that it awards bonus points on the basis of a percentage score without regard to the strength of the opponents. I am modelling alternatives that target truly exceptional performance and mostly I am finding it in new players who have completed the 25 games necessary to obtain an established rating but who continue to outperform their rating by large margins. I hope to finish this work in time to propose a new bonus point formula at the next meeting.

Handbook Amendments. Many articles in the handbook that deal with ratings are out of date. Article 702 (National Rating Committee) actually refers to individuals by name and is defunct. Article 739 refers to a rating statistician and outlines a cumbersome rating appeal process that pre-dates the internet. Article 712 deems players under 1400 to be inactive after 5 years but is not being followed. Article 717 has errors in the methodology for rating events with unrated and provisionally rated players. Article 719 allows events to be set aside for future ratings if there in certain cases but is not being followed in practice. Article 734 states that players with 3 games or less will not have published ratings but is not followed in practice. I intend to remove/amend all outdated information.

Rating Software Issues. Roger and Fred have identified several errors in the rating software that will be corrected. Two examples: the formula for assigning ratings to unrated players in a tournament where more than half the players are unrated (mostly junior events) is flawed; the software is inadvertently assigning bonus points to players with provisional ratings. These errors will be fixed shortly.

Incorrectly Rated Junior Events. Ten years ago, the CFC decided to allow Junior events with Active time controls to be rated as Regular events. Recognizing the damage that this was doing to the rating system (a 200 point drop in the average rating and the production of many under-rated juniors), the policy was reversed about 5 years ago. However, the practice continued. I have put a stop to it, however it has been a bit of a battle to get the word out to all TDs and organizers. The CFC office is not able to proof read all rating submissions or reverse incorrect ratings due to manpower constraints. I have been doing my best to contact all Junior event organizers and TDs. Some events are still slipping through the cracks, however. If after a reasonable time, this practice continues I may propose that we stop rating Junior events with Regular time controls for the loss-leader charge of 50 cents per player and limit this subsidy to Active rated events only.

Under-rated Juniors. I believe that by enforcing the rating regulations for Junior events and introducing a better bonus point formula we will help reduce this problem.

Comments are welcome.

[Egidijus Zeromskis]

Rating ...Issues

I noticed that quite often the foreign players', who played several years ago in Canada, ratings are not updated to their current FIDE rating to use in the event calculation. Do you plan to take any steps to correct this?

[Fred McKim]

I noticed that quite often the foreign players', who played several years ago in Canada, ratings are not updated to their current FIDE rating to use in the event calculation. Do you plan to take any steps to correct this?

There is no automatic way to do this. It would be up to the event submitter to make a note to Gerry to revise the foreign rating used.

[Ken Jensen]

Do you realize these Junior related moves are effectively designed to eliminate Junior players from participation in the CFC?

I do not understand how regular rating junior events produces under rated juniors who negatively impact adult ratings. Can you please explain that to me? It would seem the more rated games kids play the more closely their rating matches their skill. So if you want ratings to accurately reflect skill I would think you would want them to play lots of rated games. Would you prefer chess players avoid CFC events until they are 1800 rated adults? Or do you want them to show up unrated to adult events and only start their CFC rating then? What am I missing?

I can understand the 200 point reduction in average rating. To me that means you had a lot of junior members and their ratings tended to reduce the overall average. Getting rid of those low rated members increased the CFC average. So is the goal to have an active healthy CFC or is it to have a high average rating? If it is the later we could evict all members with ratings below 2400 and the CFC average would go way up.

Seriously though these statistical solutions are impacting participation and the future potential of the CFC. A fundamental question needs to be answered. Are Juniors an important part of the CFC or a nuisance barely tolerated? These moves indicate the latter. I feel juniors are the future and their participation should be encouraged.

It appears to a junior organizer like myself that the CFC collective wishes Juniors would stop showing up at adult events where they play strong and ruin the day for the regulars. The suggestion that the CFC should stop rating junior events can be read as " the CFC should stop having Junior members." Understand that the only thing the CFC has to offer a 12 year old chess player is a rating. Take that away and 99% of kids playing chess have no use, or interest in the CFC. They can get a rating from CMA, USCF or in our area NWSRS. They are here for a CFC regualr rating. They have zero interest in a CFC active rating.

How many of the current adult CFC members got there without being a junior first? If you want members and revenues to go with them you can grow your own with Juniors. Chess participation and skill is like a pyramid. The elite players are just the tip top. A small group that marks the peak of canadian chess skill. Beneath them are many more strong talented players. Each level grows from a layer beneath it. Each lower layer has much more participants than the one above. What is the foundation of this pyramid? Juniors.

Attract and nurture juniors and you will have steady growth in membership and revenues. Unfortunately the CFC chooses to chip away at juniors and try to get them out of the rating pool. Becareful what you wish for. You just might get it. Every action against juniors weakens the CFC . Maybe it is meant to be just a gentleman's club and no more. Chase away the juniors and you will soon find out.

How successful could the CFC be if it actually nurtured junior chess? Ask Larry at CMA what his membership numbers are, or his annual budget. Which provinces have CMa activity compared to those with CFC?

So be aware that when you talk about improving the average CFC rating by getting rid of juniors you are talking about a CFC without Juniors. You may argue that the CFC doesn't need juniors, and you may be right. I won't argue. I do know that every move you make in this direction drives away some participation. How much can he CFC afford to drive away? The CFC continues to score failing grades in Junior chess.

I apologize for the rambling.
I felt it necessary to make sure people are aware of the consequences of these actions.
I'll get off my soap box now.

Ken Jensen

[Paul Leblanc]

Hi Ken. I guess I could start by saying that I am only enforcing the rules as they stand. 30 minute events are not currently eligible for Regular rating. I happen to agree with the rule but this is a democracy and you could make a motion to change the rule.
What I have observed by looking at many all-junior 30 minute events is that the ratings of the players are very low and once they play 25 games without serious opposition, juniors receive an artificially low established rating. They then "graduate" to events with experienced players and instead of earning a Provisional rating that reflects their true playing strength, they greatly outperform their established rating and cause unnecessary angst to their opponents.
What would be wrong with rating Junior 30 minute events as Active events? Why would that not be inclusive of youth players?

[Ken Jensen]

Hi Paul,

I have been told by BC's juniors and their parents that they have no interest in active ratings. If a junior event is not CFC regular rated it will not be a CFC event. This sentiment is echoed by other Junior organizers. I have heard from more than one that they will simply drop the CFC now that the rating rule change is being enforced. Others have said they will get around the issue by simply making the time control 60 minutes, knowing that the juniors games will be done in 20 minutes no matter what the time control.

If the situation you describe exists I suggest it represents a flaw in the player development program, not the rating system. The BC Junior program consists of monthly Junior Open events where kids compete by rating. The Beginners play in the U800 section and stay there until they win enough to progress. The Intermediate section is where kids develop their skills to the 1200 level. Once they reach 1200 they move to the U1500 Advanced section. The kids above 1500 play in the Adult Open events. In this program I have yet to see a new player get his 25 games in and end up with a rating substantially below his skill level.

What I have seen is bright young kids who study chess several days a week, meet with a personal coach at least once a week and play in a dozen tournaments a year. With this kind of study they can progress very quickly. They show up at an adult tournament and face an opponent who has played one event a year for the last 30 years. These kids are sharp, and they will make you pay if you slip up. Who's "fault" is the result? Is it the keen youngster who's pushing the envelope of his game? Is it the old regular who's a little rusty? Is it the rating system?

Changing the rating system to fix this problem is like getting a new camera because the pictures from the old camera had too many flowers in them.

From what I'm hearing you would prefer juniors show up at their first adult event with no rating, so their provisional rating can be set against the skill of their adult opponents. It seems there is a difference in the intensity level between the Junior program and the adults events. It should be noted that these same juniors compete internationally and their ratings are generally comparable to their counterparts.

My conclusion is that these kids are strong. They study hard and play hard. They are also future adults. To punish them for outworking their older opponents sends the wrong message. To chase them away is removing the foundation from the chess pyramid. Without the juniors the CFC eventually resolves to the old regular who plays the same 1 event a year.

Sorry for another long winded post...

Ken Jensen

[Bob Gillanders]

I have been told by BC's juniors and their parents that they have no interest in active ratings. If a junior event is not CFC regular rated it will not be a CFC event. This sentiment is echoed by other Junior organizers. I have heard from more than one....

Really? If that really is the case, then perhaps another solution can be found, but what if it was explained to the kids and parents that the active rating (or rename it junior rating) was for junior only tournament and regular rating was for when they played in adult (mixed) tournaments. That should work, no?

Kids love their ratings. At the Mississauga junior club we have our own rating system. I would like to switch it over to the CFC rating system, and if I do, it will be sent in as active. Most of the kids don't use clocks, and those that do, it's 25 minute games.

Except for the stronger players, I don't think they care which rating it is, CFC CMA, Mississauga, whatever.

[Ken Jensen]

...what if it was explained to the kids and parents that the active rating (or rename it junior rating) was for junior only tournament and regular rating was for when they played in adult (mixed) tournaments. That should work, no?
...
Except for the stronger players, I don't think they care which rating it is, CFC CMA, Mississauga, whatever.

No, kids will not buy CFC active ratings. In BC I know there is constant pressure, or competition for event ratings. Our players have a choice between CFC, CMA NWSRS and various house ratings.
The tug of war between CFC and CMA is ongoing. So far CFC has won a large market share on the argument that it is The National Rating. As soon as you go away from regular to active the balance shifts away from CFC.

In BC most active events go unrated or house rated. I can guarantee that any current CFC rated event that you strip of it's regular rating will be lost to the CFC. Is that the plan? It may help the under rated juniors problem.

There are a lot of active events in BC. Some are CMA rated. Some are house rated, many are not rated. NONE are CFC Active rated. That's not entirely true, since 2006 there have been 5 CFC Active rated tournaments in BC. That is why I say that if your solution to the high performing juniors "problem" is to deprive them of a rating until they are adults the CFC will not have any juniors.

That said I believe you are on the right track by looking at the ratings calculations applied to juniors and adjusting as necessary so Junior Ratings and related skill match the equivalent adult rating.

The old rules that applied regular ratings to junior games as short as 30 minutes were made on the realization that for most juniors that is a "long" game. The strategy, like the USCF's was to get juniors inetgrated into the system early and develop them into adults.

Getting rid of the 30 minute rule was the first shot at the under rated junior "problem". It is along the same thought lines of "deprive them of a rating until they play only adults". Unfortunately it may have had the opposite effect. With fewer rated games the kid's ratings are even slower to adjust, and more likely to be out of date.

We will never be able to compensate completely for the effect of intensive study and progress at the chess board which young kids can demonstrate. No more than we can stop them from growing three sizes in a year.

Ken Jensen

[Paul Leblanc]

We looked at all players with 25 to 50 rated games from 1 April 2007 to the present. There were 9,100 data points. We were looking to see if an outstanding performance in one event was an indicator that we had a fast-rising player on our hands worthy of bonus points.
We set the criteria at a 50 point gain in a single event as well as the attainment of a personal high rating and followed those players to their 75th rated game. It turned out that 83% of them did indeed continue to a higher rating and the average gain was a further 71 points.
The control group, all players with between 25 and 50 rated games, regardless of whether they had an outstanding single event, also gained further rating points. 71% of them went upwards and the average gain was 50 points.
This is against a background where the average rating of all players in the CFC remained static.
So, the tentative observation is that this group is under-rated. I have much more work to do on this before translating it into a recommendation to change the bonus point system.