In metrology, error is defined as “the result of measurement minus a given true value of the measurand”.
ISO 3534-2:2006 (3.2.5) states that “Value which characterizes a quantity or quantitative characteristic perfectly defined in the conditions which exist when that quantity or quantitative characteristic is considered.”, and the Note 1 that follows suggests that this true value is a theoretical concept and generally cannot be known exactly.
In other words, when you are asked to analyze a certain analyte concentration in a given sample, the analyte present has a value in the sample, but what we do in the experiment is only trying to determine that particular value. No matter how accurate is your method and how many repeats you have done on the sample to get an average value, we would never be 100% sure at the end that this average value is exactly the true value in the sample. We bound to have a measurement error!
Actually in our routine analytical works, we do encounter three types of error, known as gross, random and systematic errors.
Gross errorsleading to serious outcome with unacceptable measurement is committed through making serious mistakes in the analysis process, such as using a reagent titrant with wrong concentration for titration. It is so serious that there is no alternative but abandoning the experiment and making a completely fresh start.
Such blunders however, are easily recognized if there is a robust QA/QC program in place, as the laboratory quality check samples with known or reference value (i.e. true value) will produce erratic results.
Secondly, when the analysis of a test method is repeated a large number of times, we get a set of variable data, spreading around the average value of these results. It is interesting to see that the frequency of occurrence of data further away from the average value is getting fewer. This is the characteristic of a random error.
There are many factors that can contribute to random error: the ability of the analyst to exactly reproduce the testing conditions, fluctuations in the environment (temperature, pressure, humidity, etc.), rounding of arithmetic calculations, electronic signals of the instrument detector, and so on. The variation of these repeated results is referred to the precision of the method.
Systematic error, on the other hand, is a permanent deviation from the true result, no matter how many repeats of analysis would not improve the situation. It is also known as bias.
A color deficiency technician might persistently overestimate the end point in a titration, the extraction of an analyte from a sample may only be 90% efficient, or the on-line derivatization step before analysis by gas chromatography may not be complete. In each of these cases, if the results were not corrected for the problems, they would always be wrong, and always wrong by about the same amount for a particular experiment.
It can be easily estimated by measuring a reference material a large number of times. The difference between the average of the measurements and the certified value of the reference material is the systematic error. It is important to know the sources of systematic error in an experiment and try to minimize and/or correct for them as much as possible.
If you have tried your very best and the final average result is still significantly different from the reference or true value, you have to correct the reported result by multiplying it with a certain correction factor. If R is the recovery factor which is calculated by dividing your average test result by the reference or true value, the correction factor is 1/R.
Today, there is another statistical term in use. It is ‘trueness’.
The measure of truenessis usually expressed in terms of bias.
Trueness in ISO 3534-2:2006 is defined as “The closeness of agreement between the expectation of a test result or a measurement result and a true value.” whilst ISO 15195:2018 defines trueness as “Closeness of agreement between the average value obtained from a large series of results of measurements and a true value.”. The definition of ISO 15195 is quite similar to those of ISO 15971:2008 and ISO 19003:2006. The ISO 3534-2 definition includes a note that in practice, an “accepted reference value” can be substituted for the true value.
The difference between ‘accuracy’ and ‘trueness’ is shown in their respective ISO definition.
ISO 3534-2:2006 (3.3.1) defines ‘accuracy’ as “closeness of agreement between a test result or measurement result and true value”, whilst the same standard in (3.2.5) defines ‘trueness’ as “closeness of agreement between the expectation of a test result or measurement result and true value”. What does the word ‘expectation’ mean here? It actually refers to the average of the test result, as given in the definition of ISO 15195:2018.
Hence, accuracy is a qualitative parameter whilst trueness can be quantitatively estimated through repeated analysis of a sample with certified or reference value.
ISO 3534-2:2006 “Statistics – Vocabulary and symbols – Part 2: Applied statistics
ISO 15195:2018 “Laboratory medicine – Requirements for the competence of calibration laboratories using reference measurement procedures”
In the next blog, we shall discuss how the uncertainty of bias is evaluated. It is an uncertainty component which cannot be overlooked in our measurement uncertainty evaluation, if present.
Conducting a laboratory analysis is to make informed decisions on the samples drawn. The result of an analytical measurement can be deemed incomplete without a statement (or at least an implicit knowledge) of its uncertainty. This is because we cannot make a valid decision based on the result alone, and nearly all analysis is conducted to inform a decision.
We know that the uncertainty of a result is a parameter that describes a range within which the value of the quantity being measured is expected to lie, taking into account all sources of error, with a stated degree of confidence (usually 95%). It characterizes the extent to which the unknown value of the targeted analyte is known after measurement, taking account of the given information from the measurement.
With a knowledge of uncertainty in hand, we can make the following typical decisions based on analysis:
The figure below shows a variety of instances affecting decisions about compliance with externally imposed limits or specifications. The error bars can be taken as expanded uncertainties, effectively intervals containing the true value of the concentration of the analyte with 95% confidence.
We can make the following observations from the above illustration:
In short, we have to make decisions on how to act upon results B,C and E. What is the level of risk that can be afforded to assume the test result is in conformity with the stated specification or in compliance with the regulatory limit?
By making such a decision rule, we must be serious in the evaluation of measurement uncertainty, making sure that the uncertainty obtained is reasonable. If not, any decision made on conformity or compliance will be meaningless.
Data analysis is a systematic process examining datasets in order to draw valid conclusions about the information they contain, increasingly with the aid of specialized systems and software, leading to discovering useful information to make informed decisions to verify or disapprove some scientific or business models, theories or hypotheses.
As a researcher or laboratory analyst, we must have the drive to obtain quality data in our work. A careful plan in database design and statistical analysis with variable definitions, plausibility checks, data quality checks and ability to identifying likely errors in data and resolving data inconsistencies, etc. has to be established before embarking the full data collection. More importantly, the plan should not be altered without agreement of the project steering team in order to reduce the extent of data dredging or hypothesis fishing leading to false positive studies. Shortcomings in initial data analysis may result in adopting inappropriate statistical methods or making incorrect conclusions.
Our first step of initial data analysis is to check consistency and accuracy of the data, such as looking up for any outlying data. This can be visualized through plotting the data against time of data collection or other independent parameters. This should be done before embarking on more complex analyses.
After having satisfied that the data are reasonably error-free, we should get familiar with the collected data and examine them for any consistency of data formats, number and patterns of missing data, the probability distributions of its continuous variables, etc. For more advanced initial analysis, decisions have to be made about the way variables are used in further analyses with the aid of data analytics technologies or statistical techniques. These variables can be studied in their raw form, transformed to some standardized format, categorized or stratified into groups for modeling.
Sampling is a process of selecting a portion of material (statistically termed as ‘population’) to represent or provide information about a larger body or material. It is essential for the whole testing and calibration processes.
The old ISO/IEC 17025:2005 standard defines sampling as “a defined procedure whereby a part of a substance, material or product is taken to provide for testing or calibration of a representative sample of the whole. Sampling may also be required by the appropriate specification for which the substance, material or product is to be tested or calibrated. In certain cases (e.g. forensic analysis), the sample may not be representative but is determined by availability.”
In other words, sampling, in general, should be carried out in random manner but so-called judgement sampling is also allowed in specific cases. This judgement sampling approach involves using knowledge about the material to be sampled and about the reason for sampling, to select specific samples for testing. For example, an insurance loss adjuster acting on behalf of a cargo insurance company to inspect a shipment of damaged cargo during transit will apply a judgement sampling procedure by selecting the worst damaged samples from the lot in order to determine the cause of damage.
Field sample Random sample(s) taken from the material in the field. Several random samples can be drawn and compositing the samples is done in the field before sending it to the laboratory for analysis
Laboratory sample Sample(s) as prepared for sending to the laboratory, intended for inspection or testing.
Test sample A sub-sample, which is a selected portion of the laboratory sample, taken for laboratory analysis.
Generally speaking, random sampling is a method of selection whereby each possible member of a population has an equal chance of being selected so that unintended bias can be minimized. It provides an unbiased estimate of the population parameters on interest (e.g. mean), normally in terms of analyte concentration.
“Representative” refers to something like “sufficiently like the population to allow inferences about the population”. By taking a single sample through any random process may not be necessary to have representative composition of the bulk. It is entirely possible that the composition of a particular sample randomly selected may be completely unlike the bulk composition, unless the population is very homogeneous in its composition distribution (such as drinking water).
Remember the saying that the test result is no better than the sample that it is based upon. Sample taken for analysis should be as representative of the sampling target as possible. Therefore, we must take the sampling variance into serious consideration. The larger the sampling variance, the more likely it is that the individual samples will be very different from the bulk.
Hence, in practice, we must carry out representative sampling which involves obtaining samples which are not only unbiased, but which also have sufficiently small variance for the task in hand. In other words, we need to decide on the number of random samples to be collected in the field to provide smaller sampling variance in addition to choosing randomization procedures that provide unbiased results. This is normally decided upon information such as the specification limits and uncertainty expected.
Often it is useful to combine a collection of field samples into a single homogenized laboratory sample for analysis. The measured value for the composite laboratory sample is then taken as an estimate of the mean value for the bulk material.
It is important to note also that the importance of a sound sub-sampling process in the laboratory cannot be over emphasized. Hence, there must be a SOP prepared to guide the laboratory analyst to draw the test sample for measurement from the sample that arrives at the laboratory.
Today, sampling uncertainty is recognized as an important contributor to the measurement uncertainty associated with the reported results.
It is to be noted that sampling uncertainty cannot be estimated as a standalone identity. The analytical uncertainty has to be evaluated at the same time. For a fairly homogeneous population, a one-factor ANOVA (Analysis of Variance) method will be suffice to estimate the overall measurement uncertainty based on the between- and within-sample variance. See ../assets/uploads/2018/02/19/a-worked-example-to-estimate-sampling-precision-measuremen-uncertainty/
However, for heterogeneous population such as soil in a contaminated land, sample location variance in addition to sampling variance to be taken into account. More complicated calculations involve the application of the two-way ANOVA technique. An EURACHEM’s worked example can be found at the website: ../assets/uploads/2017/10/10/verifying-eurachems-example-a1-on-sampling-uncertainty/