Nature: Call for caution in PED testing
No doubt many of you have seen this already, an article in velonews reporting on commentary in Nature about problems with the current rather sorry state of testing procedures in athletic competitions.
Especially important in Dr. Berry's article is that there's dreadfully little know about the false positive rates of any of the tests. Such assessment requires actual rigorous scientific testing, far beyond what actually passes for 'validation' in current practices. While I suspect strongly that the recent CERA tests (and supported confessions) indicate that it can detect it, an unvalidated test really should bother many people.
1 recs |
35 comments
Comments
I thought
it was the suitcases full of drugs in the rider’s hotel rooms and their following confessions that really counted as positives?
Blame my wife!
by sir eccles on
Aug 6, 2008 7:30 PM EDT
reply
actions
0 recs
Holy god I can't wait for there to be racing again...
... I grow, and I don’t mean any disrespect to you for posting this Jae nor do I suggest you shouldn’t post this and anything else you run across, but I do grow so very weary of this topic.
I know, it said "als" instead of "ist"... don't give me any crap...
by crashdan on
Aug 6, 2008 7:36 PM EDT
reply
actions
0 recs
I was just reading...
this very article. It’s an interesting take on the technology of testing – I wonder how to factor this in with the many confessions that have followed positive tests and the many admissions that have been made.
by nickel17 on
Aug 6, 2008 7:43 PM EDT
reply
actions
0 recs
Now don't go spoiling our witch hunt
I’ve been duly appointed the MF_ing ELVIS of Doping… Do you know what this means?
False positives my ass! I knew Sella was doped the minute I saw him!
"I won! I won! I don't have to go to school anymore." -- Eddy Merckx, after winning his first bike race
by ELVISGOAT on
Aug 6, 2008 7:56 PM EDT
reply
actions
0 recs
Interesting in the abstract,
but I guess I’d only really be bothered if I thought there had ever actually been a case of a false positive drug test on a cyclist. The majority of cyclists who test positive eventually confess that they did, in fact, ingest the banned substance. The few who have maintained the position that their test results were false positives are not credible, in my opinion.
The evidence against Hamilton, for example, was overwhelming—the repeated warnings about signs of blood doping earlier in 2004, the positive A test at the Olympics in addiiton to the positive A and B at the Vuelta, the detailed doping calendar and bill to Hamilton’s wife found in Dr. Fuentes’s files.
Landis had five separate samples test positive for exogenous testosterone, yet no one else at that Tour or since has falsely tested positive for exogenous testosterone.
Who else has refused to confess after testing positive? Vinokourov? Anyone here think he was the victim of a false positive? How about Heras? Any takers?
by Tifosa on
Aug 6, 2008 7:59 PM EDT
reply
actions
0 recs
I think the gist of the article is that
we don’t know what the percentage of false positives are… hence we can’t independently trust the veracity of the test results.
I know, it said "als" instead of "ist"... don't give me any crap...
by crashdan on
Aug 6, 2008 8:07 PM EDT
up
reply
actions
0 recs
which is a very nice academic argument
And Dr Berry’s CV is now looking that much better.
But Tifosa’s point is well-taken: If the testing is so bad, why so many confessions, or in the absence of confessions, corraboratin’ evidence? Also, built into the test regimes are a margin of error in any case, it’s skewed in the athletes’ favor, which is likely the explanation of the absence of a Peipoli positive. That is, he likely showed evidence of CERA, but did not pass the threshold into a true positive. The ‘99 samples case graphics did a great job of showing this part of the process actually, you could see from the records that l’equipe reproduced that there were samples where EPO was present, but below “positive-ness.” Then, there were the samples that actually crossed to the positive zone. These thingies aren’t black and white – it’s not a binary, but rather a process of analysis.
Eh, I dunno, I’ve already written more about this kind of thing this week than I wanted to.
by gavia on
Aug 6, 2008 8:31 PM EDT
up
reply
actions
0 recs
The article didn’t say it was “so bad”. He never claimed that the majority of positives were false positives, nor did he weigh in on ANY actual cases, save that the handling of Landis’s case was so bad that he has no way to actually weigh the results as meaningful or not (a position I’d fully agree with and have voiced to friends over the last year).
Why so many confessions? Because not all positives are false positives and clearly it has some ability to detect.
Berry’s point on statistics is that the prior probability does weigh in. If there’s suspicion from abnormal results (either on the course or in the blood, or perhaps in a bizarre paperwork trail) that this does and should influence the interpretation. But with a low prior probability, the chances of a result being falsely incriminating goes up. The result on its own should be questioned. The tests are not absolute. They are not infallible. Now of course determining this probability is anyone’s guess, but what they’re doing right now is NOT science.
by jae on
Aug 6, 2008 9:28 PM EDT
up
reply
actions
0 recs
Dude's built a strawman
Of course, the tests aren’t absolute, and I doubt very much any of the phd’s who have developed them would claim such. I would also challenge you – or anyone else who cares to – to tell Dr Catlin at UCLA that what he has spent his career doing is not science. I think the encounter would not be a pleasant one. Berry over-states the degree to which tests are “proclaimed,” and are run with. That’s a crap argument and vastly over-simplifies the amount of science the anti-doping experts around the world are doing.
by gavia on
Aug 6, 2008 10:20 PM EDT
up
reply
actions
0 recs
I have, on several occasions, publicly (under sworn oath) voiced that I thought a forensics test, as performed in the criminal justice system, was not science when, as it was applied, it was not science. I have said this in the presence of individuals who, like Catlin, had spent their careers saying otherwise. I am quite familiar with what happens when a scientific test becomes institutionalized and passed off into the hands of agencies with designs on screening for a particular examination.
by jae on
Aug 6, 2008 10:32 PM EDT
up
reply
actions
0 recs
The points raised in the article are . . . .
. . . . . some of of the reasons Don Catlin left his position at UCLA and became a key member in the founding of ADR.
This is what he said about the current system in 2005 – “People are following this old model—run ‘em down, chase ‘em, find ‘em, assume they are guilty, drag them into testing,” he says. “And athletes still get away with stuff, and I maintain you can get away with stuff with everybody looking right at you.” Outside online
This month, he was apart of a group of scientist who published their research in Journal of Internal Medicine. Though I have not had a chance to read their report, an article discussing it stated the following as part of the reports conclusions -
“It identifies failings of the overall anti-doping system – most of them out of the laboratories control – and describes a number of challenges to be confronted.
“And it also shows us that there are five elements that are necessary for a successful anti-doping programme. These are: a strong commitment to – and sufficient funding for – research, a smart sampling strategy, adequate analytical capacity, a trustworthy adjudication process and a solid foundation of clear principles and transparent process.”
- e! Science News
The quality of the article posted on Velonews aside, it is in line with what Don Catlin has been advocating for many years and is now pretty much what all the top scientists are recommending to address this chronic problem.
by Ryan_Liles on
Aug 7, 2008 2:10 AM EDT
up
reply
actions
0 recs
I haven't read it yet
But judging that Ricco says all 10 or 12 of his should’ve been popped, is it not more likely due to them having a higher number they’ve set to state a postive. Anything under that, “we’re pretty damn sure at this level” is declared unclear or non-postive even though we hear many stories of questionable numbers being seen. I would think, maybe it’s me hoping, that at this point in time, if they’re calling a rider out on a result, it’s because they knew they have them. And the increased testing pool is because of results just below that (which of course destroys the idea of them all being anon – or is it just while being tested that they remain anon?
"The most wasted day is that in which we have not laughed."
by nikki on
Aug 6, 2008 8:42 PM EDT
up
reply
actions
0 recs
blood profiling seems like a better approach
than the gotcha type tests for blood values, which seem to be more potentially error prone, and hence more open to less-than-satisfactory outcomes in court (like the Landis dealio).
But, I think the arguments against drug testing that are posed in the Velonews article are pretty weak. The DNA test example in the article doesn’t seem to be relevant to any of the dope tests I’ve read about.
by KevinK on
Aug 6, 2008 9:00 PM EDT
reply
actions
0 recs
Now this I don’t understand. The article was not against testing. It was saying that the current state of affairs stinks. It’s a system where we don’t actually know how reliable the tests are in either direction, and the call was for some actual science to be done rather than pronouncing that a test exists and running with it.
by jae on
Aug 6, 2008 9:30 PM EDT
up
reply
actions
0 recs
against test (as currently practiced)
I thought the DNA test example in the article, as an example of a false positive, was quite flabby.
I will use the same type of argument here.
The DNA test example in the Velonews article is pretty bad and hardly seems applicable to the drug tests that I read about during the course of the Landis case. Hence, I will assume the dope testing article in Nature is equally lame, and won’t waste any more time reading it.
by KevinK on
Aug 6, 2008 9:40 PM EDT
up
reply
actions
0 recs
Well, it’s so nice to see someone to conclusions about the content of an article based on a summary written by someone else. Sheesh.
by jae on
Aug 6, 2008 9:53 PM EDT
up
reply
actions
0 recs
That paper fails to mention the other 4 positives
Landis had when his samples were all tested for synthetic testosterone. Guess that’s what happens when you get your information from the Landis defence team.
by mysterion on
Aug 6, 2008 9:53 PM EDT
reply
actions
0 recs
The article didn’t claim to weigh in on Landis’s guilt or innocence, but if the test is flawed such that something can cause a false positive, it really doesn’t matter if you test and retest and retest. I guess I have much less faith that because someone says a test works and always tells the truth, that it always tells the truth.
by jae on
Aug 6, 2008 9:57 PM EDT
up
reply
actions
1 recs
The entire paper is based on information provided by the Landis defense team. Viewing antidoping science through such information is very problematic.
by mysterion on
Aug 6, 2008 10:00 PM EDT
up
reply
actions
0 recs
Honest question: Have you read the paper or have you merely read the velonews summary?
by jae on
Aug 6, 2008 10:08 PM EDT
up
reply
actions
0 recs
clarification
“retest and retest” suggests working with the same sample; the point here is that 5 SEPARATE Landis samples tested positive (and iirc, the re-tests occurred after Landis’s defense team asked for all of the samples to be tested, not just the stage 17 sample). I’m sure to be corrected on that one.
(And I’ve not read the Nature paper, but I have read darned near every page of Arnie Baker’s ex post facto defense of Landis, and found it suggestive, but not conclusive of the case he was trying to prove.)
by R Mc on
Aug 6, 2008 10:18 PM EDT
up
reply
actions
0 recs
You missed the point.
The point is that the doping labs haven’t done the homework to demonstrate that their “positive” criteria are worth a shit. Just restating what jae said.
The news here is that this is an article in Nature, one of the most respected scientific journals; and the article was endorsed in an editorial in that issue.
So, bring on your l’Équipe “science”; I’ll stick with Nature.
by hughw on
Aug 8, 2008 8:00 PM EDT
up
reply
actions
0 recs
Paper Ok not super applicable
One of the things the paper is talking about is what is called the prosecutors fallacy. This is where even if a test has a low false postiive rate if the actual incidence of true positives is low, a positive test may still be unlikely to really indicate a positive result. Unfortunately in cycling the incidence of true positive known by other means over the last 20 years is high so a positive test does likely mean a true positive.
I do absolutely agree with the call for TOTAL openness in testing procedures and the reason WADA or whoever believe in the test. If the test is only giving an 80% chance the person is truly using a drug, well we should know that. If it is 299 in 300 we should know that and how it was reached. There is certainly room (continents worth) for improvement by the drug agencies about that.
by Markk on
Aug 6, 2008 10:09 PM EDT
reply
actions
0 recs
the problem with open-ness
The problem with open-ness is that it offers the Santuccione’s of the world a chance to change their plans. Do you really think it would have been a good idea to announce that a CERA test now existed? Me, not so much.
by gavia on
Aug 6, 2008 10:21 PM EDT
up
reply
actions
0 recs
but *no one's* earned the trust in this sport
to say “trust us”, and not be laughed at. Not the cyclists, not the sponsors, not the UCI, and not the doping authorities. Total openness takes away any need to trust any of them. Which is a good start, for me.
by Sui Juris on
Aug 6, 2008 10:54 PM EDT
up
reply
actions
0 recs
Heck Yeah announce
If they had a good test yes announce it. Might have deterred some people from taking it. Might have allowed sneaky use also, but I don’t see it as a net loss. Keeping it secrets helps the first wave. Now it is announced and people can try to work around it. So we are now in the same situation.
Thinking about what you said, I think that it illustrates a problem with how many people (actually probably not Gavia from your previous posts) see drug testing. It is like, if we wipe out the current users and suppliers we have fixed the problem. No until rules are changed (likely never) to allow reasonably safe documented use of PED’s we will ALWAYS have a drug problem – always. Every day there may be a new athlete and doctor combo. So get all the testing in the open. Get all the processes which continuously monitor out in the open, and you might get to a somewhat stable place where the big hitter drugs and doping doesn’t skew things much. That is one big reason I like the passport idea – it is a continuous process running forever, not a raid type thing which doesn’t really change things over time.
by Markk on
Aug 7, 2008 8:59 AM EDT
up
reply
actions
0 recs
Lots of self satisfied witch hunters
commenting on this here. An article in Nature, as well as an accompanying editorial, so what: what the hell do they know?
by hughw on
Aug 8, 2008 11:49 AM EDT
reply
actions
0 recs
No-one cares any more
Floyd had his chance and every time people who know how to read this stuff looked at it they called him out as a cheat and a doper. So then he goes and cries to someone else and comes up with pages and pages of technical looking stuff written by people with lots of letters after their name. It’s all very baffling at first glance, but if you bother working through one of these documents it’s also largely full of horseshit. If he couldn’t be bothered to put this one before CAS, why should I bother wading through it.
by Monty. on
Aug 8, 2008 12:10 PM EDT
up
reply
actions
0 recs
That is not exactly true. At some issue is whether or not the people reviewing actually do know how to read “the stuff.” My own experience in criminal forensics is that there are far, far more problems than the institutional experts want to admit, that there’s a party line and it’s what gets put forward and it is very, very difficult to challenge because it’s been institutionally accepted. There is very much an “old boys” club that controls the bulk of dialog. I suspect strongly from what I’ve read that the anti-doping industry is very similar. [This is NOT a suggestion that most cases are in fact frameups or bogus, but rather that it can happen and the absolute certainty is dogma rather anything that has been investigated or confirmed through experiment.]
The popular notion is that the results are unambiguous, that positive means positive means user. The original test that raised the red flag, the T/E ratio has been demonstrated to be largely worthless as it was applied. It takes a statistical interpretation more complex than “it’s above 4” [with 4 being an entirely arbitrary number]. This has been published recently from research done by the Swiss anti-doping agency using exactly the statistical methodologies that Berry indicates are necessary. The isotopic carbon indicative of an exogenous testosterone probably will indicate exogenous testosterone, but whether or not it’s possible to get the same readings from some other entirely innocent cause. Since it’s been traditional that only individuals with the skewed (but potentially meaningless) T/E ratios have been subjected to testing for isotopic traces of exogenous testosterone, the data have been terribly, terribly distorted.
by jae on
Aug 8, 2008 4:43 PM EDT
up
reply
actions
0 recs
It's all them pointy headed professors
just stirring up trouble, is all it is.
by hughw on
Aug 8, 2008 12:20 PM EDT
reply
actions
0 recs
Wernstrom!!!!!!!!
I know, it said "als" instead of "ist"... don't give me any crap...
by crashdan on
Aug 8, 2008 4:56 PM EDT
up
reply
actions
0 recs











