Teaching Points about EQA.

 

I was going through some of my past teaching files and came across a presentation that I put together for a guest presentation in a class at the local Institute for Technology in 2016. I had already been active in proficiency testing for many years.  Often when I I look at back at my teaching  wish I had done it differently.  This time is different.  I think the comments were actually spot on, and are still both relevant and insightful. 

These were  students in their first year of training to become laboratory technologists.  Some had some university background, but most were direct from high school.  I am not sure where or how they picked up the idea that working in a medical laboratory might be a good job. 

Few had any prior to medical education or laboratories, and I’m pretty sure that none of them had any prior education or training in the concepts of Quality. 

My first task thus was to establish a familiar foundation to which which they could relate.

“Most science laboratory students are familiar with Proficiency Testing, but probably under another name.  Often they are given samples by their teacher, which  they are expected to test, to prove that they have learned a procedure properly and are able to perform tests correctly.  This is usually referred to as receiving and testing “unknowns”.  If “unknowns” are important during training, they are even more important during professional clinical practice.”

Then I tried to make the point the more often than not when mistakes do happen, they are usually not the person’s fault.  They happen.

“Proficiency testing is used for detecting a broad array of potential errors.  Sometimes there are human errors that are made as a result of slips, or distractions, or mistakes as a result of misinterpretations or misunderstandings.  Sometimes, people forget procedures or are unaware of changes because they have been away because of illness or extended vacation or are off regular laboratory rotations.  Other times errors are more systemic, the result of silent problems with equipment or reagents, errors or confusions within the procedure. “

Students need to understand that while making errors can happen, they are always a big deal, and can put lives at risk.

But, physicians and patients don’t care about laboratory problems; they  want, indeed expect and demand the information they receive to be the right answer, accurate and interpretable and relevant and on-time.  Quality assessment  tools, including proficiency testing allow the laboratory and appropriate authorities to be aware of competencies on a regular basis.”

Next I introduced to the students the concept of Quality through Assessment, Prevention and Improvement. 

“Proficiency Testing is an important component of the total Laboratory Quality Assessment  which also includes accreditation, internal audits,  quality indicators and customer satisfaction surveys.  Each Quality Measure gives a view of an aspect of the whole organization.  Proficiency Testing can challenge every aspect of laboratory performance including  every phase of laboratory performance (pre-examination, examination, post-examination) as well as activities that surround testing such as safety, quality control (peri-examination)… every phase of laboratory performance”

I wanted to warn them about the consequences of not taking Proficiency Testing seriously. 

“With regret, some laboratories staff do not understand or appreciate why they are expected to participate in proficiency testing by authorities or accreditation bodies.  Some of them may even try to suck the value out of PT/EQA through deception (sending the sample to a reference laboratory for investigation) or overwork (not processing the sample with their usual procedure or with their usual workers), and worst, failure to Investigate why and how errors occurred.”

 

I summed up with…
“In Summary, Proficiency Testing is an internationally embraced Laboratory  Quality Assessment measurement tool similar in purpose as a grown-up extension of student “unknowns”.   It is a  technique that lets laboratories  demonstrate their quality and competence and at the same time provides the opportunity to discover silent system problems that lead to laboratory error.  Proficiency Testing can and should be a valuable  part of laboratory life for the full extent pf a laboratory based career.”

As I look back now, I’m pretty pleased with the message I put forward, now some 7 years ago.  As I recall the class response was positive.   I wish I had had the opportunity to keep in contact and later survey the students over the intervening years, to see if any part of the message actually stuck.

The renaming of Quality Improvement?

 

I have had the opportunity to be actively involved with the American Society for Quality (ASQ).  I am the current Chair of the Healthcare Quality and Improvement Committee (HQIC).  I truly enjoy being  actively involved  because Healthcare Quality has become a personal passion.  Through HQIC I collaborate with like-minded people with a similar sense of passion. 

Last month the World Conference on Quality and Improvement was held in Philadelphia.  Unfortunately, I missed it because of another commitment.  Next year. 

But wait a second… in the last 3 sentences I used a phrase “Quality and Improvement” twice.  Prior to the last little while, I don’t think I have ever said or written Quality AND Improvement before.  I have talked and taught for years about Quality Improvement and Continual Quality Improvement and Continuous Quality Improvement, but never has the phrase Quality AND Improvement been part of that process.   Joseph Juran wrote volumes about Quality Improvement as did W. Edwards Deming, and Philip Crosby and while all of them were involved in ASQ they never used the interloping AND.

So where did it come from?

In 1947 the American Society for Quality Control (ASQC) was created as a union of some seven small organizations.   Each year they would hold an annual meeting (the ASQC Congress).

In 1997, on the 50^th anniversary of ASQC, the name of the organization was shorted to American Society for Quality (ASQ), I imagine because the organization appreciated that Quality had become more diverse and involved many more aspects than just Quality Control.

Along with the conference name change, the  meeting name similarly changed to Annual ASQ Congress.

Then eight years later in 2004, ASQ joined forces with several Quality organizations for other countries around the world, and they collectively decided there was value in merging all their congresses into a single world event which they called the World Congress on Quality AND Improvement;  and the interloping "AND" first appeared!

I wonder perhaps that was done to accommodate a common English translation for a conference including primary English and non-English speaking  countries.

In any event, jump forward anther 15 years to 2019, the ASQ goes through another modernizing name exercise.  Within the Healthcare Division there is a committee named Healthcare Technical Committee that needed a name revision.  They decided that a good name would  not be Healthcare Quality Improvement Committee (HQIC) but rather Healthcare Quality and Improvement Committee, still with the acronym HQIC.   

I don’t think any other committee in ASQ followed a similar path.   I did a global check of ASQ.org and found only one reference to an ASQ Committee with the phrase Quality and Improvement in its title, the aforementioned HQIC

Five years ago is not that long ago, however none of the people I have asked can remember why they opted for Quality AND Improvement.  Was there an ASQ organizational directive (now lost and forgotten) that recommended that all committees addressing Quality Improvement harmonize  their names with the conference name? 

So in the absence of a human memory, I decided to go the non-human route,  specifically  ChatGPT.  If AI doesn’t know the answer, then no answer exists.

So I asked the following… “What is the difference between ‘Quality Improvement’  and ‘Quality AND Improvement’? and in about 2 seconds, I got the following:

"Quality improvement" is a systematic, formal approach to the analysis of performance and efforts to improve it. In healthcare, for instance, it involves methodologies to improve care, patient outcomes, and other systems processes with an aim to reduce errors.

On the other hand, "quality and improvement" isn't a recognized term in the same sense but can be understood literally. It refers to the two distinct concepts of "quality" (the standard of something as measured against other things of a similar kind) and "improvement" (the act of making something better). This phrase might be used in a general context where one is talking about both the quality of something and the improvement of something, but they may not necessarily be related.

In my opinion this is an amazingly intuitive and nuanced response.  Scary that it comes from a machine.   Even if it is only the computations of a non-thinking machine, I don’t think it can be disregarded.  Somewhere in its algorithm  the machine recognized that Quality and Improvement are two separate processes that can function either together or  independently.  Just because an organization implements a Quality Improvement exercise doesn’t necessarily ensure there will be a measurable or meaningful improvement in the organization's  Quality. 

Seems to me there are two explanations for the interloping "AND"; either the folks back in that meeting in 2019 were truly visionary and  insightful, or it was a typographical error that never got caught. 

Or maybe they were just being smart alecs.

What I find interesting is that the term Quality and Improvement rings true in my ear, and I find the argument of recognizing the two activities as distinct compelling.  Maybe it is time for all references to Quality Improvement to adjust to the new AI standard.

I suspect Joseph and Edwards and Philip would enjoy the discussion.

Giving Back and Paying Forward - 2023

In the sciences, Quality is NOT about Elegance or Specialness or Brilliant Ideas. Quality, especially in laboratories, is defined by the specifics of meeting requirements and performance. Quality provides the cornerstone for confidence through error awareness and prevention for precision, accuracy, and reproducibility, and relevance. 

The Quality Tools are essential in medical laboratory research and performance.  People's lives may depend on laboratory information.  Done well, laboratory research promotes new knowledge, new understandings, new vision.  Done poorly, it wastes time, wastes money, wastes resources, and creates harm.

I would like to think that understanding the importance of Quality is innate, but that would be expecting too much, but it most certainly is a skill that can be learned and passed on from one-to-another. 

Our university department is quietly worked at promoting laboratory quality improvement through the creation and management of 3 separate laboratory quality assessment (PT/EQA) programs, one in microbiology, one in clinical chemistry and one in tumor marker testing, all of them active in across Canada and international. We created an intense program for assisting countries already with EQA programs how to upgrade their offerings to link their programs with the expectations cited in ISO15189 (ISO15189 does NOT specify the requirements of EQA, but it does point out what medical laboratories should be expecting and looking for in EQA programs). And most recently we have been working with another program to help developing countries develop more EQA tools for their local laboratories, thus freeing themselves of the foreign programs with large costs and little relevance. 

This was not the career that I envisioned when I started my laboratory training in 1975, but I personally feel much pride in carving out was has turned out to be a very successful path for myself and for laboratory quality. I am thankful for having had the opportunity to work in a truly inspiring field,  and am proud of what I have been able to do to influence and (hopefully) inspire hundreds of students. 

So now I am starting on a new path of giving back and of paying forward. Over the last year we have developed the Michael A Noble and Family Award in Laboratory Quality which is founded on a sustained endowment to our department to highlight and reward Quality focused projects and presentations of graduate students and medical residents and course takers in our department. Last night we presented our first award to a student who has been looking at factors that can impact the survival of donor platelets, necessary for people with bleeding due to severely low platelet counts (thrombocytopenia). 

By highlighting projects and presentations that lead to new understanding and knowledge and insights in Laboratory Quality and Improvement we can help lead to further advances in Pathology and Laboratory Medicine. More importantly (in my opinion) through this recognition, we can spawn lifelong commitments to addressing and highlighting and embracing laboratory quality as an essential component of laboratory discovery. 

We look forward next year's Pathology Day 2024.

Human Performance Errors (HPEs) and EQA

 

I am in the midst of an interesting career that started simply enough by being a laboratory physician, but over time has taken me to into areas that I did not anticipate which included working in laboratory quality from multiple directions. 

In the early 1980s I got involved in developing an External Quality Assessment or EQA program (also known as proficiency testing) which was designed to help laboratorians and accreditation bodies be better aware of human performance errors (HPEs).  That led me towards an active career in all aspects of laboratory quality from standards development to quality management training and then to teaching.  Some would say that sounds like a backwards direction.

 In any event, HPEs are a big problem, not only in laboratories, and usually all work sites.  They either result from slips or distractions, or from mistakes associated with knowledge or procedures failure.  The former are frequent, but often intermittent and inconsistent.  The latter are more likely to be repeated or at least repeatable.  These repeatable HPEs are often associated with poorly written Standard Operating Procedures, or unclear unmonitored training which can result in gaps in knowledge and understanding.  These are Human Performance Error, but while the error may be detected in the work of the technologist/performer, but the REAL PROBLEM is the person who did the insufficient training or wrote the insufficient SOP, i.e. these are management caused systemic errors.  This is not a new finding.  Almost all people who look at error (Deming, Juran, Crosby, Dekker) all say the same thing, management errors cause more harm and more significant harm, but rarely get included in error reports.  (The Pen is More Dangerous than the Wrench)

HPEs differ from equipment or reagent errors.  Equipment and reagent errors can be detected and prevented by a solid Quality Control program.  HPEs can be detected by EQA, especially when they are repeated or repeatable, and silent, meaning that they didn’t show up during an bad event investigation. 

When people are presented with samples that closely simulate typical samples whose content is unknown to the tester, but known by the provider, and error can be captured by the external results evaluator.  This is clearly better than having the error detected after a patient is harmed.

The external evaluation is an essential part of the exercise, because it shines a light on the problem in a way that internal evaluation by itself may not.   When the reviewer is external (as with EQA) there is no motivation to not report the problem… there is no reason to hide missed performance.  It makes EQA the most objective form of testing to detect HPEs in laboratories.

Not everyone likes the idea of being monitored for error and may find ways to escape scrutiny.  During a recent presentation that I gave, a person in the audience said that laboratories would do better if instead of developing programs like EQA, they would develop better techniques so that that errors would not happen.  I disagreed.  I don’t think there is any plan that will prevent all errors.

I noticed with interest that the International Organization for  Standardization (ISO) does not agree either.  The newest version of ISO15189:2022 on Quality and Competence in the medical laboratory expects laboratories to enroll in EQA programs that not only look for human performance during examination testing, it also specifies looking at performance in the pre-examination (between the collection of the sample and it being received in the laboratory) as well as the post-examination  (starting to creating the record or report and ending with contacting the physician).  At the present time I would guess that only about 20 percent of EQA organizations can provide today, so good on the ISO for writing standards that should push program development.