World Antimicrobial Resistance Awareness Week 2022

Twenty-five years ago I gave a presentation on Canada Health Care and Laboratory Quality Management and reminded the audience there are NO clinical isolates of Methicillin Resistant Staphylococcus aureus (MRSA) or Vancomycin Resistant Enterococcus (VRE)… until a sample is collected and tested and isolates are recovered, tested and results reported by a clinical laboratory.

Today, as the number of antimicrobial agents has continued to grow, so too has the depth and range of bacteria, fungi, parasites, and even viruses with increasing antimicrobial resistance concerns, affecting not only in human patients, but in farm animals and pets and the environment in the wild.

More of the world community is aware and engaged, but what was true then is still true today, the cornerstone for detection and awareness is based on the skills and talents of an informed and engaged and diligent laboratory staff, surrounded by a committed team of Laboratory Quality Partners that provide better guidelines and information a wider array of testing methods, and valuable education and performance monitoring and assessment and accreditation.   

Better laboratories through stronger Quality Partnerships.  

 

Keeping Proficiency Testing Relevant (Team 2)

 

Understanding the Laboratory Proficiency Testing Cycle
(External Quality Assessment – EQA)

This presentation is part 4 of a series highlighting 
Proficiency Testing as a Laboratory Quality Partner.

This series is put together by Dr. Michael A Noble BA MD FRCP the material founder and now retired chair of CMPT – Canadian (formerly Clinical) Microbiology Proficiency Testing Program.  It is based on his national and international experience and expertise in Proficiency Testing (PT/EQA).  Dr Noble continues on the advisory committee of CMPT and collaborates with other EQA programs including Oneworld Accuracy

 

In the previous missive (https://www.medicallaboratoryquality.com/2022/10/understanding-laboratory-proficiency.html) I introduced my version of the PT/EQA cycle and the Teams that work through the cycle, talked about the importance of starting and closing every cycle and event with Quality.  It is hard for PT/EQA providers to promote laboratory quality if they are not prepared to run their own Quality-Committed-Program (QCP). 

For this reason it made sense to designate the Quality Team as Team 1.  But let it be clear this is first in sequence, in the creation of an effective PT/EQA program ALL the teams are of equal and first priority.  A failure in any step along the way results in a failure for all.

This entry is on Team 2: the Organizer, the Planner, the Research and Development and Production team, which I think is the most fun.  Team 2 is all about planning and introducing new and changing challenges that are of high quality and relevant to what is happening today... and tomorrow!

Medical laboratories change all the time.  New analytes get introduced as do new equipment and reagents.  When Sunderman was looking at laboratories he created challenges for Glucose, Calcium, Salt, Urea, Protein and Hemoglobin.  Today the battery of tests is in the hundreds.   To ensure that laboratories are performing all these tests correctly PT/EQA has be change and adopt all the time.

Consider for the moment just one analyte that has consumed medical laboratories, everywhere in the world… looking for laboratory diagnostic evidence for carriage or infection with COVID 19.  In the last 3 years there have been billions of tests performed, with millions near every month. I suspect that no other laboratory test has exploded onto the world stage in as short a time ever before.

As the demand for test results rapidly increased, the number of laboratories testing for COVID around the world rapidly expanded, with many of the laboratories having little prior experience.  Laboratories performing assays at super-human capacity, all over-stretched and exhausted.  The chance that all the tests results were correct and accurate was zero. 

What was so interesting was how rapidly people at large just accepted getting their COVID test done and assumed that is was always right.  The number of positive tests was reported on a daily basis through television and internet.  Companies and institutions and governments predicated all sorts of decisions based on the results they received. With little or no consideration about accuracy or relevance. 

For those interested, here is a short presentation on what happens when tests get introduced too quickly, before sufficient training can be implemented.

https://cmpt.ca/covid-19-proficiency-testing-one-year-later/

Fortunately many countries appreciated that quality assessment of performance was essential if anything constructive  was going to come from the testing.  The PT/EQA providers need to generate new ways to measure testing performance.  They needed to get access of COVID RNA or COVID spike protein, and other materials.  They needed to figure out how much material was needed for each sample, dilutions, how long the material would be stable, whether the samples should be pure or were essential additives required.  Would the samples travel, how long would the last, and many more questions.  And they needed to do that all in a very short time in order to reduce the impact of poor performance.  Research and Development in PT/EQA was the cornerstone of protecting laboratories, patients, families, friends, and the community.  

Putting that together requires a team of laboratory folk; the ones who see and envision the challenge, those who plan the strategy to keep relevant and move forward, the scientists who work through the R&D details, and the ones to take the products of R&D and create it into a program challenge production.  That is the group that I call Team 2. 

 

Personally I love being part of that team; without it the PT/EQA program loses its relevance and  performance falls apart;  and worse, without relevant monitoring of performance, the whole of the laboratory exercise goes will slip back to what Sunderman was railing against 80 years ago. 

More on Team 3 next.

Understanding the Laboratory Proficiency Testing Cycle - Part 3

Understanding the Laboratory Proficiency Testing Cycle (External Quality Assessment – EQA) 

This presentation is part 3 of a series highlighting Proficiency Testing as a Laboratory Quality Partner. 

 This series is put together by Dr. Michael A Noble BA MD FRCPC,  the material founder and now retired chair of CMPT – Canadian (formerly Clinical) Microbiology Proficiency Testing Program. It is based on his national and international experience and expertise in Proficiency Testing (PT/EQA). Dr Noble continues as a member on the advisory committee of CMPT and collaborates with other EQA programs including Oneworld Accuracy

 In 1946 F. W. Sunderman returned to the United States after serving in the military to find that community confidence in medical laboratories was so low that it became common practice to send samples to multiple laboratories and mean results rather than trust the results of any single laboratory. 

He was so concerned that he set up the first Quality Assessment study for common chemistry analysis in medical laboratories. He learned, with concern, the community was right in their concern on laboratory reliability 

(see: Belk WP, Sunderman FW. Am J Clin Pathol. 1947 Nov;17(11):853-61). 

Shortly after, the College of American Pathologists began its national Proficiency Testing program. Much has changed in medical laboratories over the intervening 75 years, but as long as laboratorians test and report on samples, there is a continuing need for monitoring performance using Proficiency Testing, also referred to as External Quality Assessment (EQA) laboratories regardless of industry. It is specified in both ISO/IEC 17025:2017 (General requirements for the competence of testing and calibration laboratories) and in ISO15189:2012* (Medical laboratories — Requirements for quality and competence).

*Note: ISO Standards 15189 and 17043 are about to be republished as :2023  The new standards are not currently available

PT/EQA process of quality assessment is also a subject to accreditation published as ISO/IEC17043:2010* (Conformity assessment — General requirements for proficiency testing).   I suspect that the process of assessment has not fundamentally changed substantially since Sunderman’s first early works, but it is now much better defined. 

To better visualize the process I have created a cycle graphic 

From my perspective PT/EQA must keep in mind the Eight Positions of a performance cycle. I have named these positions by the key person or persons holding that position, starting with the Organizer and ending with the Quality Team.  The graphic indicates this is a cycle that repeats with each test challenge and event.

These positions do not work in isolation silos; they work in teams, depending on the task.  I see four teams:

Team 1: The Quality Team
Team 2: The Preparation Team
Team 3: The Distribution Team
Team 4: The Assessment-Education Team, and then back to Team 1.

Depending on the size and complexity of the organization, each position may represent a unique person or group. In a smaller organization, especially if they are just getting started, they may well find that number of people involved is fewer, although I doubt that any group would be able to succeed with fewer than four.   In the olden days, it was possible to start with little specific knowledge about the process of EQA and learn as you go. I doubt that is the case any longer. 

 PT/EQA cycle. 

Team 1- The Quality Team.  Organizer and Quality
The Organizer

As with all things laboratory, the PT/EQA Cycle begins and ends with Quality. (Indeed Quality is integral throughout). Among many activities, the Organizer is the person responsible for setting the mission and vision and quality policies of the program. The organizer sets the annual goals and objectives, and reviews quality performance and has the sign-off responsibility for the program Quality Manual.

The Organizer is the one responsible for the program's Annual Report, including the review of non-conformances and opportunities for improvement.  They report on if the organization met its expected goal and objectives, and presents its expectations for coming year.   The Organizer is, in ISO terms, the lead of Top Management. If the Organizer is not Quality engaged, the program will ultimately fail. 

Quality

At the end of every test event the Quality Team needs to review not only the results of laboratory performance, but even more importantly, the performance of the EQA scheme.

Did the samples meet the expectations of scheme? Did they meeting homogeneity and stability requirements? Did the get sent at the right time and did they go to the right places? Did they make clear the contents with respect to instructions and safety? Was the expected date for reporting clear? Were the results reviewed prior to sending out. Did the scheme receive any complaints, concerns, or comments or compliments on the results? Were any-and-all addressed as required? Did the results of this test event lead to a concern about any suppliers, or about any staff members? Were there any opportunities for improvement identified, and if so was an investigation pursued? Did this initiate an internal audit, and if so was it completed and reviewed?

Does the Quality Team and the Organizer, have their eye on their own Accreditation status? Are they in-step with their own requirements? Are they ready to consider whether or not they want, need or require to taking their first important step towards their own Accreditation? 

PT/EQA has a position of incredible power and influence that can impact on laboratories reputation and accreditation. It holds that position because of its close attention to its own quality status. 

The next post will highlight
Team 2: Organizer- Planner- Research&Development- Production

 

 

The “Guiding Principles” of External Quality Assessment – PT/EQA

 

                               

This presentation is part 2 of a series highlighting 
Proficiency Testing as a Laboratory Quality Partner.

This series is put together by Dr. Michael A Noble BA MD FRCP the material founder and now retired chair of CMPT – Canadian (formerly Clinical) Microbiology Proficiency Testing Program.  The content of the series iis based on national and international experience and expertise in both providing and teaching about Proficiency Testing (EQA). 
Dr Noble continues to serve on the advisory committee of CMPT and collaborates with Oneworld Accuracy.

The International Organization for Standardization (ISO) includes in one of its essential standards ISO9001:2015 (Quality Management) a list of what is calls the Principles of Quality Management that includes: customer focus; leadership;  engagement of people; process approach; improvement; evidence-based decision making; relationship management.

In my opinion citing of Principles should be an essential part of every standard.  With disappointment, ISO does not include similar preambles to either ISO15189: 2012 (Medical Laboratories – Requirements for quality and competence) or ISO17043:2010 (Conformity Assessment – General Requirements for Proficiency Testing). 

The later document, with particular interest in Proficiency Testing (PT/EQA) does however contain a preamble that speaks about a series of what it calls ten “typical purposes for interlaboratory comparisons”.  I have not included the full list here, but I strongly encourage readers to check the Introduction section of the standard.

I agree that in the widely diverse universe of laboratory testing, PT/EQA can have a wide number of “purposes”.  But within that list of purposes there are some that go beyond purpose and are core to what we do.  I argue they could and indeed should be separated out as more than “purposes” and would be better defined as “guiding principles”, which I think are qualities based on key values that help businesses and customers make beneficial decisions and create a distinct culture.

I don’t know if this list is going to be seen in the next version of ISO17043 (2023), but in my opinion I think they are worth consideration, even if they are not included going forward.  The list includes:.

A.   Evaluation of the performance

·       The evaluation of processes including stability and reliability of reagents, supplies, equipment and the environment is readily covered in the laboratory by Quality Control and internal audits. 
The external evaluation of performance can be done by direct observation by accreditation bodies but can only be done at scale through PT/EQA. 

B.   Identification of problems and initiation of actions for improvement

·       Highlighting problems  (non-conformances or opportunities-for-improvement) is done, not in search of penalties, but to light errors that potentially can harm customers.  
PT/EQA clients and providers alike should understand that laboratories that have a perfect track record with EQA/PT may still have errors, but at a frequency below the detection limits of PT/EQA.  More importantly, laboratories having even a few deficiencies in their PT/EQA challenges is making many more errors that are not being identified.  Laboratories that investigate their EQA/PT including in clinical samples, often find the same errors have occurred before.

C.   Provision of additional confidence to laboratory customers

·       Let me tell you a story.  Increasingly laboratories are putting their accreditation certificates out in the front lobby of their institution where the public can see it, but their PT/EQA certificates hang on the walls in the laboratory.  We in the laboratory know that we are doing well; what we need to do is share that information with the clients that use the laboratories that we assess.  It is the customers who benefit from knowing that the laboratories are performing well.

D.   Education of participating laboratories based on the outcomes of such comparisons

·       Education is more than a purpose; without an educational foundation PT/EQA is only serves as an error detection system, which can prompt some facilities to finding opportunities to cheat and collude. 

·       Another story… in our program we include an educational narrative in all our reports.  It is usually not more that 400 words but provides perspective and learning principles.  In a number of surveys of clients over the years, the smaller laboratories without access to journals and conferences  identified our critiques as their most available and most important educational information for continuing education.

E.   Establishment of the effectiveness of test or measurement methods

·       I include this within my list of PT/EQA principles, but I can concede that not all would agree.  In fields where there are a variety of choices on measurement methods, PT/EQA can offer some unique opportunities for study.  When samples are both homogeneous and stable, and there is a strong informatics foundation for analyzing performance, and there is a certain homogeneity in the testing laboratory pool, there can be value in looking at test methodology on side-by-side comparisons.  Only PT/EQA studies can create information on studies done by multiple laboratories looking for a common answer.  This can be an important part of the educational value-added through PT/EQA.

 

In summary I argue there are some guiding principles that should be central to every PT/EQA program.  I applaud the original crafters of ISO17043 in identifying these. I only wish they had identified them as more than just “purposes”.

But while I am on the topic let me offer a sixth Principle which probably should not only be included, but should be first on the list…   Quality First.  I think this speaks for itself.