When Standards Bodies Get it Wrong.

Today I saw an interesting article on the front page about a new requirement for the quality of drinking water in British Columbia.  As measured in nephalometric turbidity units (NTUs) our current requirement for water shall be improved from a clear level of 5 NTUs down to 1 NTU.  Sounds good and healthy and easy.


The story reminded of a few personal experiences.  A few years ago two countries looked at developing a new standard for protecting health care workers by requiring the thickness and density of the plastic walls to be increased so they would be able to resist a force of 20 N on a standardized Penetration Test.  The existing requirement required a thickness or density enough to resist 15N.  Making containers tougher has got to be better.


Again in the creation of our medical laboratory quality standard, a group of metrologists required the implementation of a expectation that laboratorians would analyze their testing studies for “uncertainty of measurement”.  This would provide a statistical tool that would provide clinical doctors with information to better interpret laboratory tests.  


All of them sound good.  A definitive and forceful statement was made.  Lives have been saved.  The only problem is that in all three situations, there was another point of view that was either not taken into consideration or was rejected. 


According to the newspaper article, it is internationally accepted that that 5 NTU provides safe clear water and the technology to get down to the 1 NTU level will cost millions of dollars to open new filtration units.  According to the public water purveyors, the water value of the new level in terms of safety and drinkability is trivial at best.  This has more to do with politics than quality.


With the sharps containers, the sharps container manufacturers had information that demonstrated the increase in plastic would not materially make the containers safer, but would result in a major equipment retooling and cost millions of dollars.  The consequence would be containers that are more expensive, but not better.  


And with the uncertainty issue, most laboratorians had no understanding or experience with uncertainty but they were pushed into adding it as a requirement anyways.  Even today, 10 years later, the only reason laboratories spend any time on this measurement estimate is to satisfy accreditation teams who don’t know or understand it any more than the laboratorians.  I was talking earlier this week to a group of accrediters who ignore the clause on “uncertainty of measurement” in ISO17025 because they recognize that it makes no sense in the laboratories they assess.  


Three stories all with a common theme, that the developers of these documents broke the first rule of what constitutes a standard; consensus.  Without wanting to flog the point, (see Standards Development and Social Media Tuesday, April 17, 2012), the problem with all these examples was can go wrong when there is clearly a fissure in standards development committee process.  


Any time one group strongly asserts that they are “right” and the other guys are “wrong” the chances of ending up with a consensus is near impossible.  And this impacts on the credibility and applicability of the document.  These sorts of problems should not be come up if the standards committee does its job properly.  
There are a few reasons for ending up with flawed or controversial clauses.  There is poor group selection, where bias is introduced even before the document got written.  The second is the power of a persuasive participant with a singular point of view.  And the third is when documents don’t get a sufficient public review process.  These may make for faster documents, but not better ones.


My advice is to read the documents thoroughly and sceptically.  If you are knowledgeable in your field and find something that seems like nonsense, you almost assuredly are not the only one who feels that way.  Write a letter to your professional body and to the organization that wrote the standard and point out the problem.  And find out which organization is responsible for standards in your country and write to them.  In Canada it is Standards Council of Canada (SCC).  In the US it is the American National Standards Institute (ANSI).  In UK it is the British Standards Institute (BSI).


Make sure your point of view gets heard, because without that you can expect no action.