Certainty about Uncertainty

I have written about this before.  But as of today there is a BIG difference.

There was an interesting dialogue that occurred in our ISO working group in the crafting of the standard ISO15189 on Quality and Competence in the Medical Laboratory.  Supposing a doctor did a blood test and then repeated it a month later and got a result that was 1 or 2 percent different from the first value and called the laboratory and ask if that difference, in the opinion of the laboratorian was a “real” difference.  The idea was that the laboratorian would respond by saying that we have done an uncertainty measurement (UM) and we think there is a 50% possibility that the numbers have the same value.  Armed with that information the doctor would make a clinical decision. 

Well that is a nice idea, but of course is nonsense.  That is not what would happen, and anyone having the slightest understanding of the clinical dynamic would understand that.  What would happen would be that the doc would probably hang up with a dazed glaze, mutter something about pin-heads and then get another test done and look at a 3-point curve for a trend.   

And this is all about to get a lot worse because now some patients have the ability to receive a results directly (direct –to-patient DTP or direct-to-consumer DTC) would be totally mystified and who knows what will happen. Without diminishing the knowledge and understanding of the general public,  UM is not a concept that the folks will understand or appreciate.  One can all but guarantee as series of bad decisions and bad outcomes.

Some of us have been arguing and roiling about this for years.  What some folks have lost along the way was that the notion of uncertainty, while an interesting concept was designed and intended for physicists working quantitative measures to distinguish between detection of ultrafine differences and noise.  As much as certain statisticians have thought they could intervene in the arena of medical laboratory interpretation, it was a nice but nonsense concept, at best, and a genuinely stupid one at worst.

There are so many variables that go into laboratory testing that can not be taken into account by a simple formula: diet, time-of-day, collection vials, temperature, transportation issues, waiting times, subtle equipment variations.  All can have some type of impact.  And worst among the worst is when the first sample is sent to one laboratory and the second goes to another.  All this is far beyond simple direct formulaic mathematical models where you punch in a couple of numbers and a computer spits out an answer.

So why do I raise this again.  Isn’t this just a matter of difference of opinion?  Just because I think it is stupid doesn’t mean it is stupid.  Right?

Well last week the ISO Committee on Reference Materials (ISO REMCO) the committee structure that is responsible for UM circulated the following:

“The present GUM is focused mainly on the situation of a physical quantity which “…can be characterized by an essentially unique value”. This has probably inhibited the wide use of the GUM in some scientific disciplines, such as chemistry and biology, in which a definition of the measurand according to the requirements of the present GUM can be impossible."  (note the italics and bold and red color are mine!)

Yes!  In your face! I was right and those @#$%^&*s were wrong!

So now there is a plan to try and fix it.  The REMCO committee will try to put something together maybe by 2014. 

In all due respect, this goes back to one of the concepts that I think is really critical to the standards development process.  Standards should be written by folks that are engaged in the field.  They should not be written by folks with solely a good idea about what they think other people should be doing.   Kind of like the US constitution “for the people – by the people”.  

But before standards become the rules they need to be veted or authorized by people with a vested interest to protect the public… the watchdogs, so to speak.  Legislatures, regulators, accreditation bodies, professional colleges need to be able to speak to the benefits and detriments of the documents to ensure that the writers had the best interests of the public in mind.  That is what we call collaboration and partnership.  Each partner doing what they do best to come together with a better product: improvement through synergy.

I have to say that I am feeling really encouraged because the system working as it should.  A piece of nonsense has been recognized for what it is, and now some knowledgeable folks are going to try and fix it.  Maybe they will be successful, maybe they won’t.  Either way works for me. 

So now the question will be what do the folks responsible for oversight of medical laboratories do now?  Do they concede that this is an issue that needs fixing and back off until a better solution is found, or do they just continue on?

And the ISO REMCO folks need keep in mind that all of us are simple folks. Developing a nuanced sophisticated solution that only hypergeek mathematicians understand is of little value for the laboratorian, the clinician and especially the patient.