Researcher beware - change is coming

The problems with research

It’s always interesting when things come together.  

Over the last few days I was at an international meeting where the discussion about the potential value of a Best Practice Guideline document for Researchers was raised because of a general experience that too many research dollars are wasted on non-reproducible and irrelevant studies.  An interesting proposal I thought, but perhaps a tad overstated.  

Then I get to the airport and pick up the Economist (March 15-21 2014) and turn to the Science and Technology section and there is an article called Metaphysicians which mentions John Ionnidis ( Why Most Published Research Findings are False ). I have discussed this author and this paper before [see: http://www.medicallaboratoryquality.com/2012/12/quality-and-medical-research.html ] because of his interest in doing “research on research”.  According to the article, a scourge of modern science is the “waste of effort”.  In 2010 “$200 BILLION (85% of total expenditure on medical research) was squandered on studies that were flawed in their design, redundant, never published, or poorly reported”.  Assuming those numbers are true, that would certain confirm there is a clear need for help.

According to Ionnidis too many researchers staggeringly over interpret statistical significance in studies that are far too small.  Further they have a lack of insight in proper study design and manifest “publication bias” where positive data gets written up for presentation and negative data gets ignored or worse.

Over the years we have observed that graduate students working in research laboratories seeming lack knowledge and interest and respect in running essential common equipment such as autoclaves.  Commonly the problem is that they are required to use the equipment, but did not receive appropriate training.  In essence they were just pushing buttons.  ( I suspect in the minds of their Principle Investigators, autoclaves are mundane compared to DNA analysis.)  

Furthermore common procedures of quality control and quality assurance are often incomplete or inadequate and generally not understood.   

What they do not seem to understand is that in the absence of basic Quality principles NO research can or should be trusted.

Apparently Ionnidis has doing something about it.  He has opened up a Center for meta research with the name Meta-Research Innovation Center at Stanford and the even more appealing acronym METRICS.  That will help to define the problems and perhaps develop some answers.  

Here are some questions that I would like to see addressed:

   A.               Would increased training in laboratory Quality result in reduced non-reproducible research and increased value for dollar spent?

   B.               Could accreditation of research laboratories result in reduced non-reproducible research and increased value for dollar spent?

   C.               Would introduction of proficiency testing into research laboratories result in reduced non-reproducible research and increased value for dollar spent?

I was intrigued by the notion of a Best Practices in Research Guideline however I also understand that in the current environment such a document would be pursued by those few laboratories that recognize the concepts of standardization and continual improvement.  The vast majority would, I suspect, never become aware of its existence, or never purchase it, or never read it, or read it but never consider it as relevant or appropriate to their laboratory.

I know this sounds a tad cynical, but for how long have we taken that approach in the health sciences. “It is not our problem, we are too smart, if it is a guideline then we can ignore it and if it is a regulation we can obscure it.”

This may well be the time when far more aggressive research oversight becomes a reality.  If a highly qualified author writes something in Accreditation and Quality Assurance (an excellent journal) its impact is strictly limited.  But if articles end up in the Economist, that is a different story.  Business folks read the Economist, as do lawyers, and politicians, and University presidents, and many of the general public.  

Sooner or later, the right (wrong?) person is going to start putting 2 and 2 together; wasting $200,000,000,000 is an insult to the public purse.   And then the regulations start happening, and some people lose their jobs, and some people end up in jail.

Even if we spend $2B on setting up a stringent oversight we are way ahead of the game.  And the impact on jobs would be negligible because for every research laboratory that is shut down, some would likely move into consultancy or oversight. 

Seems like a win.        

Does Laboratory Accreditation Improve Patient Safety?

Does Laboratory Accreditation Improve Patient Safety?

Module 3 in our Certificate Course in Laboratory Quality Management is my favorite because it addresses the critically relevant issues of the modern medical laboratory including the Costs of Poor Quality and the Role of Quality Partners.  

For reasons unclear, many (most?) modern laboratorians have brainwashed themselves into disparaging all the groups that have been created to HELP laboratories meet customer requirements.  “We” are suspicious of accreditation bodies, and disparage proficiency testing programs.  “We” trivialize educators, and demean suppliers for being in it only for the money (as if making money is a bad thing!).  I am not sure why we do this.  It has been the order of the day ever since I got involved in laboratory medicine in the 1970s.  But through our course, the evidence for increasing awareness of the importance of Quality through  literature and conferences and activities is very present.  I think that we are recognizing our friends better but it has been a slow process. 

One of the questions that I pose to the group in Module 3 is “Does laboratory accreditation improve patient safety, and how can you prove it?”.

I will say that I come to this question with a point of view.  I wrote an opinion piece now a long time ago [see  Noble MA. Does external evaluation of laboratories improve patient safety? Clin Chem Lab Med. 2007;45(6):753-5] at which time I opined that it is impossible to demonstrate by evidence, but it would be a mistake to stop accreditation just because you can prove it helps patients.  I have also written in this on-line journal [see: http://www.medicallaboratoryquality.com/2013/03/making-your-quality-partner-your-enemy.html and http://www.medicallaboratoryquality.com/2013/03/quality-and-accreditation-partnership_18.html ] 

It is still a relevant question to ask, especially with this group of experienced laboratorians who are sufficiently committed to laboratory quality that they would put a lot of money out and spend a lot of time thinking about and discussing relevant Quality.  

Of note and interest, all participants who engaged in the conversation wanted to believe that accreditation would increase patient safety for all the same and reasonable reasons.

First and second, accreditation would increase the interest in maintaining standards, and accreditation would increase the strength of the Culture of Quality.  Some did a PubMed search and found Peter’s aspirational and speculative  “Impact of Laboratory Accreditation on Patient Care and the Health System” (Am J Clin Pathol 2010) which says but doesn’t demonstrate the relationship.  

One person, wrote that on balance accreditation had more advantages than disadvantages.  That is a great sentence.

One of the challenges to writing more definitive statements is that the value of accreditation is very difficult to prove by study.  We talked about studying laboratories before and after accreditation looking for parameters like turnaround, and error, and report quality, but that sort of study could take years, and even with that looking for those markers would not help.  We talked about going to a native jurisdiction where there is no accreditation and then preparing one group while leaving the other group alone.  Aside from the moral perspective, this approach would likely fail as well because the point of Quality is not so much to stop errors, but to pick them up faster, and that would be very difficult to demonstrate.

All this taken aside, the important first step is that we have a group of laboratorians who believe accreditation is a positive step forward, and to my way of thinking that is indeed the most important step forward.  In this group the scepticism about accreditation is replaced with confidence and trust.  That is brilliant and a huge step forward. 

Now if we can figure out how to block health authority and government and capitalist bean and profit counters who all want to throw out the baby with the bath water and actually kill medical laboratories with “redundancy is bad” insanity, we actually have laboratorians  who have insight and care.

But I have my doubts.

The Power of On-line Courses

Once again I am in the middle of my on-line teaching commitment for the UBC Certificate Course in Laboratory Quality Management, and once again, I am loving it.

When I graduated from medical school and residency, I enjoyed working as an Infectious Disease specialist because it created the opportunity for a 1-to-1 relationship with patients.  It was satisfying to provide service and care in a truly direct fashion.  

But as my career progressed and I became more engaged in the laboratory and infection control, I found that my opportunities for impact and decision influencing were on a wider level – groups of patients distributed over wards and institutions.  What I sacrificed in terms of satisfaction with direct patient contact I picked up with system influence.  

With proficiency testing, if the direct patient impact was even further away, the system influence was much broader and covered health authorities and provinces and countries, and I was addressing quality assessment, quality improvement and education all at the same time. And as time marched on the education component grew from critiques to newsletter articles, to international training.  And the level of satisfaction grew even larger. 

But with the Program Office and the Certificate Course, the interaction scale has a whole new dimension to the dynamic.  While there is another layer that separates the laboratorian from patients, and sense of connection is much greater because the amount of interaction with laboratorians is much more intense.  Often we are communicating on a daily basis about Quality and Education and Laboratory improvement with folks taking the course are from across Canada and around the world.  The intensity and intimacy combined with the level of system influence can almost seem overwhelming.  

Over the years it has become very rewarding to attend meetings in many places and have people come up to me and say, “You may not remember, but I participated in your course in 2008 (or whatever) and I want you to know that it opened up a whole new career path for me.  Thank you so much”.  In the first 2 months of this year, I have run into people in 3 meetings with this same story.  And then there are the people who have contacted me through Linked_in.

It is very satisfying to know that so many people are pleased they took the opportunity to learn about laboratory Quality through our course.  I almost feel like Mr. Chips, if you know who that was.  

Recently we have been approached to consider developing a companion course which would cover another whole set of Quality related concepts not covered in the first course.  It is an intriguing idea.  As the whole arena of medical laboratory Quality grows (Thank you Institute of Medicine and To Err is Human.  Thank you ISO 15189) the body of essential knowledge continues to expand.  This is not information that is addressed in medical residency, nor in health administration, nor in the traditional MBA.  Increasingly people want it and institutions need it.  

I don’t know if this is a career path for everybody.  But it is a great path for me.