EQA and Continuing Education: saving lives?

One of the challenges of the clinical laboratory is that most samples received in the laboratory are routine, beyond routine.  In the microbiology laboratory the vast majority of samples are urine samples from women and the elderly, throat swabs from young kids, and genital samples mainly from the folks hoping they have dodged an unappreciated bullet (most of them have).  The reality is that within this group there are few samples of life-altering significance.   


The challenging samples where a stain analysis reading error or a faulty culture can potentially have a lot of significance include Cerebral Spinal Fluid (CSF) or joint fluid and maybe blood cultures, but these samples are not very common; indeed they are rare, especially in the smaller laboratories.  And in the rural laboratories what makes the challenge worse is that the technologists (and pathologists) tend to have very little exposure and experience to samples from patients with true examinable and interpretable diagnostic samples.


What can make this situation worse is the intervention of politics.  In these days of laboratory consolidation there are those that see great advantage in moving all the microbiology testing to a single site, and if a critical specimen (as in CSF or Joint Fluid)  comes up in the middle of the night, well the available Chemistry technologist will stain and examine it or ship it down to the main facility.  Take the least trained person and put them in the situation of having to stain, read and interpret and report the most critical sample.  That always seems to make a lot of sense (said with great sarcasm!)


So it was in that context that we at our proficiency testing program set up a supplemental program to provide samples that simulate joint fluid and CSF as a way to increase their amount of exposure to these samples.  After 20 years (plus) of creating these samples we have become really slick at creating really good simulations with the right number and type of cells and appropriate concentrations of bacteria, some easy to read, and some not so much.  On a scale of 1-10 (where 1 is you don’t actually need a microscope to read the slides and 10 is that skilled persons still find some of them very difficult). I would estimate the challenge value of the samples is close to 8 or 9.  There are not enough samples that the laboratories will become over-burdened, but enough that when the real samples come in the have enough exposure and expertise that they can read and report them with more confidence.  Where there may be 1 or 2 samples a year, we can raise the number up closer to 2 every 3 months.


So the program has now been running for about 2 years and we have just completed our first customer satisfaction survey; and the message appears to be “thank you”.   That by itself is a pretty big accomplishment.  Folks don't tend to say a lot of positives about proficiency testing. 


Most of the slides are rated as  "acceptable" or better and the readers see them as looking like and staining like real samples.  The educational material appreciated and rated highly.  And they would like us to extend the program to other uncommon but high risk samples.  


What I didn’t ask, but probably should have, was along the line of if the program has increased personal confidence in reading true critical slides.  In truth the responses would have been highly subjective and difficult to analyze, but I think it will have to be a part of a follow-up questionnaire, perhaps examinable within a word cloud.


So here is my bottom line: providing practice materials for laboratorians put in the terrible position of having to read high pressure samples with little experience and expertise is providing a service to the laboratorians and their patients.  The laboratorians comment positively about the program and I think it reduces the risk of reading error.  


While I can’t prove it, in my opinion and bias is this is a very important service, and it might even in some instances save a life.  


To see a critique with performance, evaluation, education and photograph visit: 


http://www.cmpt.ca/critiques/2011/gs112_1.pdf


More later.

Proposal: Radical Change to ISO 9001

ISO 9001:2008 is soon to have a new iteration created.  Arguably this document is one of the world’s most successful international standards, and much (most?) of industry and service success in progress in Quality is directly attributable to the creation of this standard, now 25 years ago.  I know that the standard has its critics and they are certainly free to have and express their opinion, but in my opinion, the criticism falls mostly in the category of “what have you done for me lately?”.  

The document is what it is: an excellent codification of Quality principles with strong elements of Shewhart and Deming and Crosby and Feigenbaum, all brought together through strong background documents like CSA Can3-Z299 (Quality Assurance Program).  It provides a coherent plan that says: follow these rules and your chances of success rise dramatically.  Ignore them at your peril.

If there are problems that surround ISO9001 it seems to me that they have less to do with the document per se, and more to do with its application.  Boiral and Amara (the Quality Management; Journal 2009) have previously pointed out that in recent history only about 25 percent of certifications prove down the road to be successful and effective, either due to internal barriers that interfere with the standards application or poor performance on the standard’s requirements.  They talk about managerial certifications where management gets engaged but no one else does, or ceremonial certifications where management gets engaged and starts to implement action but runs into a wall of resistance.  

What this speaks to is the company that moves to implementing a Quality System without putting any effort or energy into how impart organizational Culture of Quality as part of its plan.  If the organization has a Culture of Quality, the odds on effective implementation are much greater.  If an organization moves to certification before it is really ready, the odds for success diminish.  

So how does ISO build this concept into the next iteration of ISO 9001:20XX?

One very appealing suggestion was made in recent discussion during the CSA Annual Meetings.  Full registration of an organization’s Quality system should be done on a progressive scale.  
Putting my own spin to the concept, when a company seeks certification for the first time, and on external audit it is deemed to have Quality fundamentals in place it will receive Certification-BRONZE.  When the company seeks recertification, and on external audit the certification body can see evidence of implementation and evidence for successful application, the organization’s status is raised to Certification-SILVER.  When the company is audited the next time and there is clear evidence for implementation growth and established improvements they receive Certification-GOLD.  And when they show evidence of implementation and provision of mentoring or some other special activities, they can receive Certification-PLATINUM.  

What this type of approach would do is support all the organizations that are expected to start along the pathway, but would distinguish between those that want to get started (BRONZE) and those that want to progress ultimately to stellar levels.  

Sounds simple, but I can see reasons why it would be a challenge to implement, but I think the system would have merit.  

There would need to be some accommodation to change and some rules to implement.  That would be OK because it would demonstrate that ISO internally recognizes the value of continual improvement.  There would need to be some objective specifics on how to measure “successful application” “implementation growth” and “special activities”.  I don’t think these would be particularly difficult to develop. 

Perhaps the hardest set of rules would those that address progression.  From my perspective, since Culture is implemented through slow growth, I would personally argue an organization could only start at the BRONZE level and could only progress one step at a time and could only progress as a consequence of external assessment.  

(While I strongly support all organizations of adopting and implementing Quality Management standards on a voluntary and internal basis, I would reserve the terms BRONZE, SILVER, GOLD and PLATINUM to those that have had external evaluation.  I would go one step further.  I would further require organizations that implement ISO 9001 based on their own efforts and energies to designate themselves as “meeting the requirements of ISO9001:20xx by Self-Assessment).

So that is a plan that I would propose.  I know I am not the first to suggest/recommend change, but I also know and understand that Standards Development bodies are usually uncomfortable with change and absolutely abhor radical change.  But sometimes radical change is what organizations need to do.

Consider this a gauntlet thrown. 

Deming did not create PDCA

Thanks to Google there is a fun game for entertainment and insight.  Create a sentence in English, and pass it through Google Translate and convert it to another language, and then retranslate it back to English and compare what you started with and what you got.  Sometimes the results take an interesting twist.  Recently I heard about a company that has a commercial interest  in the phrase “come alive!”.  Unfortunately when that phrase was translated into Japanese it became “may you ancestors return from the dead”.  Gives a whole different tone to your soft drink pleasure.  Another is the German translation of “knapsacks” or “fanny packs” in “body bags”, or In Italy, a campaign for Tonic Water translated the name into Toilet Water.  It reminds me that we are not all that far away from the biblical Tower of Babel.  

But if you think that translations always go from good to bad, there seems to be an important Quality story that suggests the contrary.  The source for this comes from Ronald Moen and Clifford Norman who wrote an interesting article “Evolution of the PDCA Cycle”.  You can find it on the web.  It was originally published as Circling Back in Quality Progress in November 2010.

According to these two well informed Qualitologists, when Shewhart first raised the notion of a structured quality cycle in 1939, it was in a manufacturing framework.  The notion was that first one creates the Specification, and then creates the product by Production based on the specification and then tests the product through Inspection.  

By the time Deming adopted this and took the concept to Japan in 1951, Shewhart’s three phase cycle evolved to four phases:  first there is a product Design which is followed by Production to the design.   Then the design and product are studied through the customer’s response at the time of Sales.   If the customer is not happy and registers a complaint, then the process from design to sales (or release) is Researched, and if necessary revised to improve customer satisfaction.  One can see that this clearly has the flavour of the current cycle.

What happened next is the point of my story.

According to Imai (through Moen and Norman) Japanese executives, taking Deming’s notes from his lecture with the Japanese Union of Scientists and Engineers (JUSE) and translated Design-Produce-Sales-Research (see above) as PLAN-DO-CHECK-ACT.  So while the concept may have come from Shewhart and Deming, the words came from the English-Japanese-English translation. 

This is a perfect example of innovation [see: http://www.medicallaboratoryquality.com/2012/04/invention-and-innovation-and-new.html] which in this case may have been a consequence of intent (if the executives knew the potential power of going from the manufacture specific language to more generalized terms) or serendipity (if they did not). 

Either way, the creation of a generalizable science concept for Quality contributed largely to the wide application of PDCA in education, service, and health, and government in a way that the Shewhart or Deming manufacture specific language could not. 

Of interest the first place that Deming actually published the use of the term PDCA (which he subsequently changed to PDSA) was not until he published his book Out of Crisis in 1986, over 30 years later.  

I am not trashing Deming or Shewhart.  Both were clearly  leaders in expressing Quality in scientific terms.  But the term PDCA cycle was not their invention, it was the creation of the Japanese executives responsible for the translation.  And I think that we do history a disservice to continue to attribute the cycle as we do.

  I think my son Steven has said it best.  Life loves a good story.  While mythology needs to have heroes and clean stories,  history rarely works out that way.  

Sometimes you have to decide what you want - the story or the fact.  

Deming did not create PDCA.  Deal with it. 

National Standard Academic Challenge

Canadian Standards Association (CSA) has always been an international leader organization in the arena of Quality.  Created in 1919 it was focused on ensuring the quality of bridge construction for the railroad.  As mentioned here before, early (Z299) documents from the CSA were used as the seed documents for the creation of ISO 9000.  CSA has active in the environmental industry, nuclear industry and the nanotechnology industry.  So it should not be a surprise to anyone that they would be always be so very well positioned to take the step forward in innovation and engagement.

Look across the standards world and you will see a lot of grey hair.  Granted that interest in standards development is a pre-occupation that people grow into; first you need to be engaged in an occupation or profession or field of activity long enough to know that there are such things as good practices and not-so-good practices, and the source of good practices can usually be found through reading and understanding consensus based standards.  In my experience that usually takes 5 to 10 years (maybe more – maybe less) to reach that level of insight.  But the experience is also that once in the standards enthusiasts get energized, eventually they become the standards fogies.  I would guess the mean age in the standard arena is well above 60, which means for every person age 50 there is one at age 70, and for every person age 40 there are two near 74.   As one person said, in a lot of committees it is not necessary to set 6 or 8 year term limits on chairs because lots of us won’t make it anyways.

So the answer is to start younger.  

There are all sorts of younger people who would be appropriate to engage in standards awareness and standards development.  Standards awareness should be a component of every training program in electrical, mechanical, healthcare trades and professions.  Every student should be able to know the difference between an effective standard and an ineffective one.  Every student should know which organization in their region or country is engaged in consensus based objective relevant standards.  Every student should understand how standards are the basis for consistent quality practice.  And in time once they have got their hands and head dirty in their field of interest, those folks who want to, should have the opportunity to get engaged in creation of the next generation of documents.  These are the kids who will become our next group of standards developers and quality creators.

So the CSA started with a new pilot “National Academic Challenge” this year associated with its Annual General Meeting with some young undergraduates from an Ontario university.  The presentations were appropriately enthusiastic and well done.  The subjected were perhaps young and naïve, but that’s OK. The pilot was a great success and good for the students and good for a first shot by the organization.  The students loved it and the members loved it.  The exercise was clearly a winner and well worth repeating.  

It pointed out that as a way to recruit some young blood into CSA, this model has some legs.  The yield to CSA will likely be stronger when the challenge involves participants with a little more age and experience, and a more of a focus on a career choice, but that will evolve over time.

On another level it shows that Canada is well positioned to move forward on the world stage through hosting a World Standards Cooperation Academic Day.  We have the organizations, the academics and students at a wide variety of levels.  

There is every reason for Canada to play on the world stage.

More to come. 

Measuring Quality Knowledge

People grasp information in many ways.  They may observe, or perceive, or even describe information, but it can be argued that they do not really know the information until they can integrate facts from their past to use in the present or in the future.  The process of discovery is taking information from many observations and sources and integrating it into novel concepts.  It is the aspiration of teaching that people not only learn facts but discover how to integrate them for broader application and insight; the creation of “new knowledge”.

It was with this in mind that on the final examination for our Certification for Laboratory Quality Management course I asked the following:  Phillip Crosby defined Quality as “meeting requirements”.  Explain what is meant by the phrase “meeting requirements”.  Provide examples of meeting requirements as they might apply to (a) laboratory turnaround times, (b) laboratory accreditation, and (c) writing of standard operating procedures (SOPs).

OK, maybe it wasn’t the best question to put on a final examination, but there was a lot of choice and answering this was not compulsory.  That is probably why there were so few takers.  What I wanted to see was some evidence of information integration happening.  

When it comes laboratory turnaround times, this is pretty straight forward.  A laboratory has choices.  It can perform and complete analysis at its own convenience (how often do we hear that complaint!) or we can take into consideration the needs of the clinician and patient.  We can arbitrarily set time limits such as 1 hour or 1 shift or 1 day, and then measure how often we meet that goal.  Or we can go a talk to the clinicians and understand what their needs are.  Do they usually need the information the next the same day, the next day or the next week?  Does getting the information to them too late cause problems?  Does getting the information out very early create other problems?  

Having gone through the exercise we can say that we understand better the needs of our clients, and then develop strategies that will help us meet those needs.  That approach integrates “meeting requirements” and “turnaround times”. 

In a related way, if you write a Standard Operating Procedure (SOP) and nobody can read it or understand it, does it actually exist?  Well it may exist but it certainly doesn’t meet the criteria for being a Quality document.  It does not meet the requirements of your main internal customers, ie the people that work with and for you.  SOP developers should always take the opportunity to have their documents reviewed by the personnel before it gets finalized and integrated and signed off into the document control process.  Does it meet the needs of clarity and specificity and achievability?  Does it make sense to the people that are going to implement it?  If it does that’s great; that’s called “meeting requirements” and it becomes a useful Quality document.  If it does not, they we call that an “opportunity for improvement”.  

Integrative knowledge questions are a common part of the examination process, whether it is done in form of long or short paragraph or in multiple choice.  A recent paper by Stuart H. Jones and Michael Wright (University of Calgary) – “Does Cognitive Style Affect Performance on Accounting Examination Questions?”  in Global Perspectives on Accounting Education Volume 9, 2012 describes the ability to organize information either as being field dependent (ability to organize familiar structured information) versus field independent (able to organize information that is less familiar and more complex).  Field dependent students (FD) performed the same as field independent (FI) ones on straight forward examination questions, but did not perform as well as the FI students on questions that required more information integration.  FI students would not only tend to do better on this style of question, but that does not mean they would be more likely to choose complex questions. 

When researchers looked at the total group, neither age nor gender is strongly linked although men tend to do better than women and older children tend to do better than younger children or adults.  Furthermore FI people’s brains tend to work differently in that if given a complicated picture, they are  more able to see embedded figures and shapes inside the picture using a test called the Group Embedded Figure Test or GEFT. 

To date the cognitive style research papers that I have found have tended to be focused on individual groups and do not give any information about the proportion of FI people there are in the general population, but in the absence of any information to the contrary I am going to make the claim that people interested in Quality are by definition very bright, very intuitive, and very field independent.  Our collective Quality-oriented brains are just better wired.  But given a choice, we, as with our FD colleagues, do not tend to select more challenging questions when less complex ones are available.

By the way, you might notice that the examination question asked for 3 examples, one on the application of “meeting requirements” in the context of turnaround times, and another in the context of writing SOPs and the third in the context of laboratory accreditation.  For purpose of this entry, I purposely left that one unaddressed.  

I would be interested in knowing if anyone in the context of anonymity (or not) wants to take a shot at this one.  

June is Quality Month

There is so much (maybe too much?) happening on my Quality front in the few weeks.  I don’t think that I could have avoided this log jam through better planning or calendar management.   Sometimes, and the expression goes, things happen.


It is final examination time for our on-line Certification Course for Laboratory Quality Management.   I am marking answers on standards development, quality partners, quality associated definitions, and the strengths and weaknesses of proficiency testing.  The other faculty are all doing their fare share as well.  Fortunately Maggie and Doug, our coordinator and IT specialist have figured out a way to automate and improve our ease of electronic access to the responses.  It is probably saving me at least 2 hours on the grading process which likely will mean less stress and a more enjoyment in marking.  So far, I am really impressed with the answers that I have graded.  It is interesting and encouraging to see how much Quality information the participants have accumulated over the last 6 months.


Next week is all back-to-back-to-back meetings.  First is the Annual meeting of the Canadian Standards Association and there are going to be some very interesting meetings for leaders of Strategic Steering Committees to look at ways that we can benefit from each others experiences and perhaps find ways to harmonize activities across some pretty diverse industry boarders.


At the same meeting CSA is holding a first Canadian Academic Challenge, which will in some way I suspect connect with the World Standards Academic Day that was just held in Bali.   There is no doubt that CSA needs to find ways to better and more strongly connect with academia and the student and young faculty pool.  The reality of the currently active  standards community is that it is getting older, and the gomers (me included) need to start looking to the next younger group if we want to sustain a Quality presence.


The day after the CSA meeting there will be two days of meetings as the Canadians involved with ISO Technical Committee 176, the committee responsible for ISO9001:2008 start to prepare proposals to revise and amend that document.  This will be a very interesting exercise because the document is very mature.  But if the document does not undergo its own continual improvement process, then what will that say about its role as the very cornerstone of Quality.  Change and Improve, or Die.


And the day after the ISO TC 176 meeting is the joint meeting of the Canadian Society for Clinical Chemistry (CSCC) and its Quebec sister society (QSBC).  This meeting is being focused on the changes that are happening in Quality for medical laboratories across Canada.   There will be a substantial national and international presence and I am looking forward to being one of the keynote speakers, hoping that I remember to not burn myself out at all the other meetings being held throughout the week.


And then when I get back we start with the new cycles for our CMPT proficiency testing Annual Meeting (October) and our celebration of our 30th anniversary of provision of high quality proficiency testing services and 11th year of the Program Office for Laboratory Quality Management, and hopefully receive some positive news about our current grant proposals.  


In the next short while I will be announcing some pretty exciting changes that will start next year in our Certificate Course.


And we need to get started preparing the ISO Technical Committee 212 meeting in Berlin, and for a new PhD student intending to focus his research and study on medical laboratory Quality management.  He will be arriving in Vancouver in August.  I am pretty sure that he will be the first person in Canada to take on Quality management as a focus of primary interest.   I suspect I will hear more when I attend the Canadian Academic Challenge meeting.


My personal opinion is that this sort of pace of activities is about to become our new normal.  After a brief break in July we will be working at this sort of pace for the next 6 months, probably longer.  It may indeed go on for years.   


It seems the light has turned on in Canada and we are new starting to see all the huge benefits of being engaged in laboratory Quality at every level; a Quality explosion perhaps.


I call that a good thing.