“NIST-traceable” is often invoked as though it has some sort of magical metrological power, imparting accuracy to all it touches. In reality, it is only one of several elements of good calibration. What’s worrisome in the practice of equating NIST-traceability to accuracy is the perception that it supersedes the other, equally important elements of calibration.
The National Institute of Standards and Technology (NIST) is the USA’s federal agency responsible for ensuring that we adhere to a common measurement system. NIST, along with the world’s other National Measurement Institutes (NMIs), describes our shared metrological language, without which many critical industries—life sciences, meteorology, aerospace, and more—would live in a Tower of Babel situation, metrologically speaking. This is because, without global reference standards that are shared, agreed upon, and ratified, for instance, a component madeto- measure in Malaysia might cause a catastrophic failure when assembled as part of an airplane in the U.S. We need to agree on a standardized “vocabulary” of weights and measurements; in effect we agree to accept a reference.
BY ANY OTHER NAME
But, isn’t a meter always a meter? Unfortunately, no. There is no absolute truth in measurement. Measurement is really about probability. The percentages of times the measurement will be different are the uncertainties of that parameter. What we are left with is “close enough” or, the nearest we can currently get with the technologies and equipment available. Fortunately, for most applications, close enough is good enough. So long as measurements are standardized and communicated internationally—by our National Measurement Institutes (NMIs) like NIST,1 PTB,2 NPL,3 etc.—we have a global “mother tongue” of accuracy. But a shared language does no good unless it is applied. This is the goal of traceability. Traceability is what allows us to use international standards as the references for calibrating measurement instruments, so that mètre, even when called a “meter,” is still the same length.
TRACEABILITY V. ACCURACY
What is traceability? According to NIST, “[it] requires the establishment of an unbroken chain of comparisons to stated references.” NIST stands by its own measurements and can provide standard reference materials and calibrations that we can be assured conform to other NMIs around the world. Since it’s not possible to have NIST calibrate every instrument, traceability allows us to propagate measurement standards.
So we create a chain that can be traced back to the reference. If Cal Lab A has a thermometer calibrated by NIST, then Cal Lab B can have their thermometer calibrated by Cal Lab A, and be NIST-traceable. Cal Lab B can then calibrate thermometers for Lab C, and so forth.
What is important to understand, however, is that as you move farther and farther from that original calibration of Cal Lab A to the national standard reference, the uncertainties increase. Each link has its own uncertainties. Accuracy is really about uncertainties, and each instrument—that which is calibrated and that which is the reference— has discrete uncertainties. NIST-traceability cannot equate to accuracy in calibration; it is, rather, a starting place.
In reality, the procedures, equipment used, and skills of the calibration technicians all have a greater impact on the quality of any calibration. Without using a reference that has been calibrated to the international standard, it is entirely possible that a calibration lab could still perform an extremely accurate measurement by using the best available methods, instrumentation, equipment, and skilled technicians. But who would know other than the lab itself? So we see that a shared reference and traceability to that reference is only a starting place and cannot supersede the other elements of accurate measurement. Just as you can measure accurately without a reference linked to NIST, you can use a traceable reference to perform a poor calibration.
Keep in mind that there is no auditing process for using the term “NIST-traceability.” Any calibration lab can claim it, and it may well be true. But how many calibration labs down the chain of traceability are you? If your device is calibrated to a reference that has degraded in its accuracy, you may not know it until you have a catastrophic failure of a product or process.
ANALOGIES OF TRACEABILITY
Consider a lock and master key. If you cut a copy of a master key it will likely open the same lock that the master was designed for. Now, cut a new key from that copy. You may get a somewhat sticky key—not quite as accurate as the original, but still a working copy. If you keep making copies of copies, eventually a key will fail because the accuracy degrades with each copy. Another analogy of traceability is the “Telephone Game.” By the time the last person in the line hears the whispered message, it little resembles the original. This is traceability; with each copy or transmission of a message, the uncertainties increase. Accuracy cannot be transmitted from an accurate reference as if by magic, it can only be effected by measurement that accounts for all known uncertainties, performed by someone with skill, using good methods and equipment. NIST-traceable reference instruments can’t magically undo the harm of calibration poorly performed.
The NIST itself does not attribute magical powers of precision to traceability, quite the opposite. NIST’s policy on traceability4 states exactly what it is, and is not:
“It is important to note that traceability is the property of the result of a measurement, not of an instrument or calibration report or laboratory…Merely having an instrument calibrated, even by NIST, is not enough to make the measurement result obtained from that instrument traceable to realizations of the appropriate SI unit or other stated references.” 5
ACCREDITING BODIES & STANDARDS
If we start with NIST and build outward toward accuracy in our measurement instrumentation, what comprises good calibration? Accreditation by an organization that evaluates the laboratory, equipment, and staff against established standards (such as ISO/IEC 17025, ANSI Z540) is the next step. An accrediting body (AB) forces the calibration to include a quantification of the length of its traceability chain in terms of all known uncertainties. The processes used and competency of people performing the calibration are also key elements of ISO/IEC 17025,6 and is also a critical difference in this standard.
An AB will ensure that a calibration lab is compliant with ISO/IEC 17025, which applies to all tests and calibrations performed within the scope of accreditation using standard, non-standard, and laboratory developed methods. This standard is applicable by all organizations performing tests and calibrations including 1st, 2nd, and 3rd party labs performing test and calibrations and applies to all laboratory staff, instruments, methods, and procedures used within the scope of accreditation. If using NIST-traceable reference standards ensures we are at least starting from accurate, our accrediting bodies and standards such as ISO/IEC 17025 and ANSI Z540 ensure the words of that language are strung together in a meaningful way, which is to say accurate enough to be useful to measurement- critical applications.
While it’s beyond the scope of this article to describe ISO/IEC 17025, a basic understanding of it helps us understand why it’s useful to seek calibration that complies with it. Largely used to create management systems for quality, regulatory authorities, and accreditation bodies, ISO/IEC 17025 is applied to determine a laboratory’s competence. It’s often mistakenly thought that ISO/IEC 17025 is used as a basis for certification, but it isn’t. Specifically used to create accredited quality management systems to assure the competence of a laboratory, the standard does not provide certification of products or services.
Two key sections (out of a total of five)7 of ISO/IEC 17025 are “Management Requirements” and “Technical Requirements.” The first section provides direction for the operation and effectiveness of the quality management system within the laboratory and to provide compliance with ISO/IEC 9001.8 The “Technical Requirements” section addresses the competence of staff, methodology, and test/calibration equipment and also distinguishes this standard from other quality management standards. The ISO/IEC 17025 standard tries to ensure that there is neither a dull blade in the hand of a brilliant surgeon, nor a razor sharp scalpel in an unskilled hand.
THE WHOLE TRUTH (OR CLOSE ENOUGH)
We share a common language of metrology in order to apply the best current standards worldwide. However, it’s important to understand the scope of that language. NIST makes the propagation of the standards possible, but does not make those standards a requirement. It publishes its knowledge, but does not regulate the use of that information. It compares and works with other global NMIs to produce accurate references, but does not certify traceability;9 nor does it imply any level of uncertainty in that traceability. NIST is like the dictionary, defining terms. How we use the words depends on our message and our context.
To step away from the language analogy to some real-world examples, consider the implications for stability testing with inaccurate devices, or biological samples for transplantation stored a few degrees below the optimal temperature. A monitoring device for a refrigerator that is inaccurate by only a few degrees will not alert you of temperatures that will allow freezing that can lower the efficacy of drugs or vaccines, ruin expensive samples, or destroy high-value product. We need to go beyond thinking that NIST-traceability can mitigate poor calibration to protect critical products and processes that are vulnerable to the conditions we are trying to measure.
A shared language in measurement through traceable references is only the beginning. Given, it’s an important first step, especially when we know that a meter is only a mètre in a certain percentage of measurements (and depending where you are on the planet). The next step is to ensure that we don’t underestimate the impact of accuracy in measuring our environment. An effective quality system requires that calibration of instruments goes beyond using a reference that is NIST-traceable.
References
- National Institute of Standards & Technology is the NMI for the USA. See: http://www.NIST.gov/index.html
- Physikalisch-Technische Bundesanstalt (PTB) is the NMI for Germany. See: http://www.ptb.de/index_en.html
- The National Physical Laboratory is the NMI for the UK. See: http://www.npl.co.uk/
- http://ts.NIST.gov/Traceability/Policy/NIST_traceability_policyexternal. cfm
- See NIST: Traceability Policy > Supplemental Materials > “What is traceability?” http://ts.NIST.gov/Traceability/supplmatls/suppl_matls_for_NIST_policy_ rev.cfm#def15
- ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories available online at: http://www.iso.org/iso/Catalogue_detail?csnumber=39883
- 1. Scope, 2. Normative References, 3. Terms and Definitions, 4. Management Requirements, 5. Technical Requirements. See: [http://www.a2la.org/requirements/req17025.pdf]
- For further information on the International Organization for Standardization, see: http://www.iso.org/iso/home.htm
- See: See NIST: Traceability Policy > Supplemental Materials > [http://ts.NIST.gov/Traceability/supplmatls/] “Although the measurement results in a calibration or measurement certificate can be considered to be certified by NIST to be traceable to NIST reference standards at the time the measurements were performed, NIST cannot certify that those measurement results are valid after an instrument or artifact or reference material has left NIST.”
Bruce McDuffee is the Marketing Manager of the Americas Controlled Environments division, of Vaisala, which provides measurement, monitoring, and validation solutions for critical en - vironments to the pharmaceutical, medical equipment, food processing, and semiconductor manufacturing industries. He holds a BSc in Civil Engineering and served as a Surface Warfare Officer in the US Navy for several years before entering the test and measurement instrumentation industry. (Watch a Humidity 101 webinar at cemag.us/company/veriteq-vaisala)
