Training and consultancy for testing laboratories.

Posts tagged ‘profiency testing’

Homogeneity and stability of PT samples

Importance of Homogeneity and stability of PT samples

To run a successful proficiency testing (PT) program, the importance of homogeneity and stability of PT samples prepared for an inter-laboratory comparison study cannot be over emphasized, as these two factors can adversely affect the evaluation of performance.

The PT provider must ensure that the measurand (i.e. targeted analyte) in the batch of samples is evenly distributed and is stable enough before laboratory analysis at the participant’s premises. Therefore an assessment for homogeneity and stability for a bulk preparation of PT items must be done prior to the conduct of the program.

Checks for sample stability are best carried out prior to circulation of PT items. The uncertainty contributors to be considered include the effects of transport conditions and any variation occurred during the PT program period.

A common model for testing stability in PT is to test a small sample of PT items before and after a PT round, to assure that no change occurred through the time of the round.  One may check for any effect of transport conditions by additionally exposing the PT samples retained for the study duration to conditions representing transport conditions.

A simple procedure for a homogeneity check

The homogeneity check aims to obtain a sufficiently small repeatability standard deviation (sr) after replicated analyses.  The general procedure is as follows:

  1. Select a competent laboratory to carry out this exercise
  2. Take a number k of the PT samples from the final packaged bulk preparation through a random process
  3. Prepare at least m = 2 test portions randomly from each PT sample
  4. Take the k x m test portions in a random order and carry out single measurements for the targeted analyte concentrations
  5. Use 1-way ANOVA to analyze the data generated from (4)
  6. Homogeneity of the prepared bulk is achieved when the F test ratio of mean square between samples against the mean square within samples is smaller than the F critical value with the given degrees of freedom, i.e. (k-1) and k(m-1), respectively.

The assigned value of PT program

lab-picture-1

The assigned value of PT program

A critical step in the organization of a proficiency testing (PT) scheme is specifying the assigned value for the participating laboratories.  The purpose is to compare the deviation of participant’s reported results from the assigned value (i.e. measurement error) with a statistical scoring criterion which is used to decide whether or not the deviation represents significant cause for concern in its performance.

ISO 13528:2015 defines assigned value as “value attributed to a particular property of a proficiency test item”.  It is a value attributed to a particular quantity being measured.  Such an assigned value will have a suitably small uncertainty which is appropriate for this interlaboratory comparison purpose.

Where do we obtain an assigned value?

A.  Assigned value obtained by formulation

A specified known level or concentration of the target analyte is added accurately to a base material preferably containing no native analyte. The assigned value is then derived by calculating the analyte concentration from the masses of analyte used.  By this way, the traceability of the assigned value can usually be established.

However, there may be no suitable base material (blank material) or well characterized base material available. Ensuring homogeneity in the prepared bulk material before distributing to the participants may also be a challenge. Furthermore, formulated samples may not be truly representative of test materials as the analyte may be in a different form from the less strongly bound to the matrix.

B.  Assigned value is a certified reference value

In this case, the test material is a certified reference material (CRM) made by a reputable organization and the assigned value is therefore the certified value and its uncertainty are quoted on the CRM certificate.  The limitations of using this assigned value are:

  1. Generally the CRM can be expensive to provide every participant with a unit of such CRM
  2. It is important to conceal the identity of commercial CRM from the participants as the testing outcome may be compromised by the participants
  3. Some certified value uncertainty may be high.

C.  Assigned value is a reference value

The assigned value is determined by a single expert laboratory using a suitable primary method of analysis (e.g., gravimetry, titrimetry, isotope dilution mass spectrometry, etc.) or a fully validated test method which has been calibrated with a closely matched CRM.

D.  Assigned value from consensus of expert laboratories

This assigned value is obtained from the results reported by a number of expert laboratories, with demonstrated proficiency in the measurements of interest, which analyze the material using suitable methods.  However, it must be cautioned that there may be an unknown bias in the results produced by the expert laboratories.

E.  Assigned value from consensus of PT scheme participants

This is the result from all the participants in the proficiency testing round. It is normally based on a robust estimate to minimize the effect of extreme values in the data set.

 

 

What is proficiency testing?

Lab Picture 3

What is Proficiency Testing?

The quality of your analytical measurements and your laboratory’s technical competence can be enhanced if you participate in testing schemes satisfactory with a number of laboratories simultaneously.  Such a scheme is known as proficiency testing (PT) scheme.

The primary aim of PT is to allow laboratories to monitor and optimize the quality of their routine analytical measurements.

A PT scheme is usually organized by an independent body, which can be a national standard body, a learned professional organization or even a business enterprise.  Basically, in chemical analysis, aliquots from homogeneous and stable test materials are distributed to a number of laboratories for analysis to be carried out at a stated window of time. Each participant is given an unique identification code.

After the participants have analyzed the samples using either a test method of their choice or a stated standard method, the scheme organizer will carry out statistical analysis of all the data submitted and provide a performance report, detailing each participant’s statistical ‘score’ that allows them to judge their performance in that particular round of testing.

In other words, the participating laboratory can gain information on how their measurements compare with those of others, how their own measurements improve or deteriorate with time, and how their own measurements compare with an external quality standard.

There are a number of different scoring systems used in PT programs; the majority involve comparing the difference between the participant’s result (x) with a target or assigned value (xa) with a quality target, which is usually a standard deviation for proficiency assessment, denoted by xp.  Each scoring system has acceptability criteria to allow participants to evaluate their performance.

Generally we do expect some divergent results to arise even between experienced, well equipped and well-staffed laboratories. If so, this PT scheme helps to highlight such alarming differences, and to suggest to these laboratories look into their own analytical process in order to improve the quality of their test results.  Hopefully a better comparison is achieved in the next round of the PT testing.

Amusingly, there are reports that some participating laboratories have been caught to have colluded in reporting their test results to the scheme organizer, particularly when the PT scheme does not involve a large number of laboratories and some of these laboratory operators are known to each other, such as being subsidiary laboratories of an organization group.  Such collusion act is undesirable as it defeats the noble purpose of carrying out the PT scheme and renders the outcome of this round of PT testing meaningless.

A simple explanation to this incident is that these laboratories are not confident of their own testing and need to compare results of others before submitting the results to the organizer. However, the outcome of a statistical graph may show a bunch of results grouped at one corner when these results are questionable, meaning they are significantly different from the assigned value of the test material analyzed.

Of late, some scheme organizers try to overcome this malpractice by preparing and sending out at least two labelled test samples with close but significantly different analyte concentrations (not duplicates) to the participants. The originality of these samples is only known to the organizer.

Actually we must not treat PT samples with extra care and attention, but run the PT samples like any other routine samples.  It is quite common to see that a participant would repeat the analysis as many times as possible until no more sample left for future reference!

A good source of reference on statistical techniques applicable to a PT scheme is ISO 13528:2015  Statistical methods for use in proficiency testing by interlaboratory comparison