Ranking the Rankings

Today seems like a good time to check in on one of our major initiatives for the 2010 cycling season -- our Podium Cafe World Rankings! Note, these rankings and numbers are identical to the data we use to calculate the VDS rankings, on the theory that our little game is a pretty good reflection of reality. But the bigger point here is that we have tried to come up with a definitive ranking system, for a sport that doesn't really have one. The final word on this project will have to wait til October, but let's take a peek in on our rankings and see how they stack up against some of the other alternative methodologies. Top ten only:

This isn't the full list of rankings, no doubt, and if you know of a better one, feel free to do more comparisons. But I've been a CQ fan for a while, and the UCI is the governing body of the sport. I added Cycling Fever because I really like their work.  So what do we see?

The first thing to do here is to eliminate the right and left hand columns from serious consideration. Sorry, Cycling Fever, but any methodology that calls Andre Greipel the #1 rider in the world is flawed. Now, the Cycling Fever folks do excellent work in general, so I am open to correction here, but this data comes from the Cycling Fever Ranking page, so I don't think I am misreading this. My hunch is that they are not weighting wins heavily enough, so that Greipel's "shit races" (thank you for this endless source of humor, Mark Cavendish) are on the level with Contador's grand tour exploits. That doesn't tell me enough, IMHO.

The UCI's rankings must also get the heave-ho. We've discussed this before, but their "methodology" is less science than marketing. No sentient being would give the Tour of Poland more weight than Paris-Roubaix, and yet there it is. But hey, I'm not a UCI basher. For example, I think they have a wonderful rulebook. Oh, and killer offices.

That leaves the Podium Cafe and CQ Ranking. There are two main differences in our methodologies: the point scales and how we handle races before Het Nieuwsvolk. CQRanking hits the ground running on January 1, meaning the TdU and the Arabian Peninsula stuff gets counted. They also include more minor races than we do, though I don't think those results are affecting the top ten individual rankings. Our decision not to include every minor race had to do in part with how those results might skew the team and country rankings. Save that topic for later.

As for the scalings, I can see a couple points that illuminate the differences. Joaquim Rodriguez got 240 points from CQ for winning the Volta a Catalunya and 200 points for finishing 8th in the Tour. From us, he got 150 for the Volta win and 225 for 8th in the Tour. Between the two, our Tour points were pretty close but they gave JRod 60% more points for the Volta than we did. I definitely think their Volta scores are too high -- for example, second in the Volta is scored equally to 9th in the Tour de France, and I personally think the latter is far more important and difficult -- but I guess you could question why our Volta rankings are so low. [Answer: because the 2009 Volta was thoroughly uninspiring, so we downgraded it. Overraction??]

Another big difference is the treatment of Luis Leon Sanchez. Of the 1424 points awarded LuLu in CQ, 35% of those were in races we didn't count. Twenty-two percent came from races too early on (TdU and Algarve) while we declined to include the Circuit de la Sarthe, where he cleaned up, in favor of some other French races. I'd have to go back and review the thinking, but I know we talked that one over a good bit, so I stand behind how we made our decision. In any event, while LuLu is a great racer, he has utilized races of questionable quality to vault ahead of guys like Cancellara, Andy Schleck, Farrar, etc., in their poll. For me, I'll take the historic cobbles double and E3 Prijs over Paris-Nice, San Sebastian and some smaller wins.

Someday we will find a way to give some credit to a few of the earlier races, but otherwise I am feeling pretty solid about our ranking system compared to the rest. Thoughts? I am very open to constructive criticism here. And not in a Chairman Mao Hundred Flowers I am taking down all your names for future retribution sort of way.