As we near the 2023 USA Cycling Cyclocross Nationals, the USAC ranking system attracts more scrutiny from athletes concerned about call-up positions and motivated by title hopes.
One California race promoter, Clipped in For Life, even held a race series specifically to help West Coast racers get a points boost for a better call-up at Nationals (but largely suffered from poor attendance especially from highly-ranked racers).
Before our last Nationals in Louisville back in 2018, math professor Jim Cochran wrote about problems with the USA Cycling rankings that positively impacted some riders and regions, while penalizing others.
Call-ups aren’t everything, but they matter. In Louisville, we saw multiple fast racers start from the back row to land on the podium. Eire Chen did it to take the 11-12 title, in a small field, but Vida Lopez de San Ramon suffered from more traffic in the 13-14 race of 43 starters, and finished second, wondering what could have been had she had a better call-up.
USA Cycling briefly abandoned its system in favor of crossresults.com, offering racers in areas without USA Cycling-sanctioned racing a chance at a competitive call-up position. But it has returned to its ranking system, and the scrutiny has returned.
Now, a parent of a junior racer, who wishes to remain anonymous and is not associated with CXM, has written in with his own editorial about the USA Cycling ranking system, detailing how a group of young juniors have better rankings than top pros and hoping it’s not too late for USA Cycling to implement some changes before this year’s Nationals. See the parent’s editorial below:
Does the US Have the Best 13-14 Year-Olds Juniors in the World?
Way, way back in time, 2011 to be specific, USA Cycling overhauled its points system to better rank riders. You would think that by now, 12 years later, the system would be fully understood and working perfectly. Unfortunately, that is not the case.
What Is the USA Cycling Ranking System?
As explained in an FAQ issued by USA Cycling, for cyclocross, a rider’s rankings are determined by averaging their best 5 rankings achieved in any race within the last rolling 12-month period. To calculate rankings in an individual race the system takes the five best-ranked riders from the top 10 finishers, averages their pre-race rankings and multiplies that by 90 percent to obtain the “Race Quality. The winner of the race earns the “Race Quality” as their points then each subsequent finisher earns the Race Quality points plus a number calculated according to a specific formula. No one can earn more than 590 points provided they finish the race.
Why Is it Important?
As any cyclocrosser knows, your position on the grid will help or hinder your finish position. The ranking system is used to place riders on the grid at the national championship, coming in less than a month, and in some local races.
What’s Wrong Now?
In 2018, ironically the last time cyclocross nationals were in Louisville, CXM ran a lengthy article with the help of Jim Cochran, a math professor and avid cyclocrosser. In this article he highlighted some of the basic errors that were occurring at the time, specifically averaging errors and races entered multiple times. Those simple errors were fixed at the time but other errors highlighted five years ago have continued.
In an article on its website, USA Cycling explained that it found a calculation error in its ranking algorithm, although it didn’t identify the root cause. It goes on to explain that the error “led to riders in certain regions being unfairly granted lower (better) rankings” and “also impacted several Elite athletes who were missing from the top 50 rankings completely”. It’s hard to imagine how one error might cause both those things but I’ll take them at their word. Either way, the solution, “to re-calculate rankings from the 2021 season forward” with all athletes reset to baseline ranking points (600) as of January 1, 2021, does not seem to have fixed matters. It would seem to make better sense to go back as far as you can with the recalculation to avoid creating “bubbles” in the data.
Let’s look at an example by asking the question “Who do you think wins: Tobin Ortenblad or the 5th best 13-14-year-old?” According to USA Cycling it is the 14-year-old. Clearly, something is not working the way it was hoped. Take a look at the table below which shows the points and rankings of the top 5 13-14 year olds and a sample of pro/semi-pro riders. Bear in mind that these numbers are dynamic and were accurate as of 11/8/2023 and may not be exactly the same now. By comparison, the crossresults.com points (although calculated differently) seem to give a much more accurate idea of who is better. Crossresults.com data in the table below is also as of 11/8/2023.
Top pros are bested by 13-14-year-old racers in Colorado in USA Cycling’s ranking system.
Top pros are bested by 13-14-year-old racers in Colorado in USA Cycling’s ranking system.
In 2018, people found problems in the regional variations with races in some states scoring points below 100, an extremely low score. Now even our national champion Curtis White only has 270 points although he will be glad to know he is at the top of the rankings! Have our riders gotten that much worse over the past 5 years or is the system just trending higher and higher? Is the beast in control or do we need some new/different AI to calculate the rankings or do we already have one in crossresults.com.
Looking at the regional and other variations now indicates there is still some kind of problem. Is it possible that 30 out of the top 100 male riders in the country are from Colorado, including 11 Coloradans over the age of 48? This was an issue that Jim Cochran specifically highlighted with Colorado racers in CXM’s article back in 2018. In addition, the USA Cycling points ranking is telling us that 22 of the top 100 racers in the USA Cycling system are over the race age of 45 and that 18 of the top 100 are race age 16 and under.
Of course, all of this is entirely possible (although highly unlikely) but without the tools to recalculate it no one will ever know and that is another issue with the USA Cycling rankings: transparency. USA Cycling, unlike crossresults.com makes it exceptionally difficult to verify the rankings calculations for a particular race after the fact. See below to see the data that crossresults.com makes visible:
CrossResults shows some of its math to rank racers at a non-USA Cycling cyclocross race.
There is no similar thing on the USA Cycling website. Perhaps that is because it is still running on very old technology. In 2018, USA Cycling said, “As you are probably aware, we are in the midst of an extensive IT overhaul. We are scheduled to move our rankings over to this new system in 2019.” Judging by the fact that all the USAC rankings and results web pages still start with the word legacy.usacycling.org, I am guessing that transition never happened.
What Is Happening with Rankings Points Now?
What is likely happening is that the rankings points are currently favoring people who race regularly (Colorado?, 13-14-year-olds?) against each other, in their own bubble so to speak, because the 90% factor applied to get to race quality drives points down in consecutive races with the same riders. When these riders race against other bubbles (e.g. 13-14 year olds in a Cat 4 race) they tend to fare not so well but that does not affect their points because only the top 5 best races in the last 12 months count. They don’t even have much positive impact on others in the Cat 4 race because if they are not in the top 10, their “good” (low) points do not affect race quality. On the other hand, the top-level riders in their bubble who are more selective in when they race are penalized for that fact. In the end, these ranking differences might eliminate themselves but that process might take several years.
What to do Now?
While many (most?) cyclocross racers may care little about these points, they are important for anyone thinking of going to Nationals and in many parts of the country where USA Cycling cyclocross races are prevalent because they are used to grid racers. And, as you know, getting stuck at the back of a grid can be very frustrating because no matter how fast you are overtaking can be a time-consuming business, something you don’t have in a 20-60 minute race.
But Cyclocross Nationals is coming up soon and there is time to fix the problem with the USA Cycling system. One possibility would be to perform a recalculation doing a much longer time period i.e. much earlier than 2021 (possibly penalizing newer but fast racers), or possibly re-evaluate the algorithm itself as it seems to have had problems going back a long time. Perhaps AI could aid in this?
Given that we seem to be in a similar position as we were back in 2018, perhaps neither of those solutions are a realistic proposition. Another alternative would be to license the crossresults.com rankings and use that going forward. Recall that USA Cycling used crossresults.com rankings for Nationals in 2021 with no apparent downside. The advantage of doing this is that it would not penalize the racers who live in parts of the country (think the West Coast) where none or very few of the cyclocross races are sanctioned by USA Cycling.
If you have similar concerns with the ranking system, let USA Cycling know, as I have.
