In 2011, USA Cycling revamped its scoring system to better rank riders against one another in local series, regional UCI weekends and Nationals.
Last year we did a deep dive on the scoring system, which averages a rider’s best 5 race scores over the previous 12 months. Although many of us might not care about our specific numbers, they are important because they are used to stage amateur riders at U.S. Cyclocross Nationals and for many local races.
Right now, USA Cycling is in the middle of a massive IT overhaul. The overhaul includes a new web platform and the phasing out of many legacy features on the usacycling.org website.
“The results and rankings system currently runs on our legacy IT system,” USA Cycling’s Chuck Hodge told Cyclocross Magazine. “As you are probably aware, we are in the midst of an extensive IT overhaul. We are scheduled to move our rankings over to this new system in 2019.”
The Results and Rankings page currently points to the legacy USAC website.
USAC Rankings Investigation
One of the many walks of life USAC license holders come from is useful when it comes to numbers. Jim Cochran is a professor of mathematics who also is an accomplished Masters cyclocross racer.
Cochran did some digging on the USAC rankings and reached out with some findings of discrepancies in the USAC rankings.
“I have been studying USAC cyclocross points for nearly a decade,” Cochran wrote. “I have noticed countless errors throughout the years. This year the system has gone utterly astray, especially in the Northeast, Texas, and Colorado. Be assured that I do not have a grudge against any racer or region, and just have an interest in numbers. This year riders have been climbing through the rankings at an unusual rate, and I started digging deeper into the race results and formulas behind USAC’s ranking system to check the legitimacy of their results.”
In an editorial Cochran sent to us, he identified several issues with the USAC rankings. Below are some examples he included.
Scores averaged incorrectly:
Races entered multiple times:
And in states such as New York and Texas, extremely low (good) ranking scores for some races. For reference, a score of 100 is good enough to be the top-ranked Cat 1 in some states.
[In the first example, finding a rider’s top five scores is kind of a chore. Go to Results & Rankings –> Rankings –> Click on a rider’s name –> Click on the “spread” versus any rider]
Both Cochran and Cyclocross Magazine reached out to USAC about the issue.
Cochran received the following response from USAC:
“We think we have identified the cause of this. Our IT department is going to experiment with a fix on our development server so that we don’t create errors we cannot fix in the main system. I am working with IT to solve and rectify this issue.
We appreciate the investigative work you are doing, and it is very helpful. I can promise you this is a priority but can’t put an exact time/date we will have an answer.
As I have answers, I will pass them along to you. Our first attempt is to re-run the rankings from July 1 on, only using prior results, with the hope it clears up the problem. This will allow us to see if a bug in the system happened, or if we need to dig even further into and determine what broke.
Please understand I am not taking this lightly and will be working to find an answer and update the rankings (correctly) as soon as possible.”
We have done a check of some of Cochran’s examples. The ones above with averaging errors and multiple race entries appear to have been resolved—at least in the specific cases Cochran cited.
The low ranking scores for races in New York and Texas, however, are still present. Cochran also said that he has found scores from Colorado to be “off” as well.
Impacts for Louisville
The obvious near-term impact effect of these ranking issues is call-ups for Louisville Nationals that start in less than a month.
Cochran wrote, “If USAC is going to use this system to stage their National Championship, maybe they need to make sure it is working. Riders are beginning to register for the USAC National Championships, and it is important for USAC to release an official notice that the rider ranking system is in disarray. Riders should not just ‘bank’ on their call-up order.”
He also provided a helpful suggestion for all license holders our there. “I have shared only a few of my many examples, so USAC members, please do some of your own investigative work, as I am sure there are many errors I have missed and reach out to USAC and share your findings. Today, I have been informed by USAC that cleaning this up is priority number one.”
We are currently in contact Hodge and will keep our eyes on the continued fixes to USAC’s rankings. For now, it appears the legwork Cochran has done has at least helped fixed at least some of the low-hanging issues with the rider rankings.
If you do notice additional discrepancies, you can reach USA Cycling at rankings [at] usacycling [dot] org.
