Infrared Thermographs- Methods for Evaluating Laboratory Performance
Catalog of Regulatory Science Tools to Help Assess New Medical Devices
Technical Description
This tool consists of a series of test methods for objective, quantitative assessment of infrared thermographs (IRTs) performance based on IEC 80601-2-59: 2017 [1]. Methods for evaluating stability and drift, image uniformity, minimum resolvable temperature difference, and radiometric temperature laboratory accuracy of IRTs are described in detail in peer-reviewed publication [2].
IRTs have been used for initial temperature assessment and triage of individuals for elevated temperatures in medical and non-medical environments. To achieve good accuracy, qualified IRT systems should be used, and appropriate procedures should be consistently applied. This tool provides methods for testing IRT laboratory accuracy which can enhance the methods in the international standard [1].
The tool may improve the utility of IRTs and aid in comparing IRT performance, thus improving the potential for producing high-quality countermeasures during infectious disease outbreaks.
Intended Purpose
The tool is intended for evaluation of IRT laboratory accuracy. It enables, streamlines, or otherwise improves verification, validation and quality control during design, development and manufacturing of IRTs. The testing described in this tool should be performed by persons with engineering background in thermography.
Testing
The method has been tested on two infrared cameras (A325sc, FLIR Systems Inc.; 8640 P-series, Infrared Cameras Inc.). Details are described in our publication [2].
Limitations
- This tool is for IRTs that capture whole face thermal images. Therefore, there are requirements for camera configuration (for example, pixel number).
- This tool can only be used to evaluate IRT laboratory accuracy. A complete assessment of IRT performance should also include clinical accuracy evaluation as described in a publication [3]
- This tool is not intended for evaluating laboratory accuracy of some temperature scanner kiosks that identify the region of interest (ROI) with a visible light camera and to measure the ROI temperature with an infrared sensor with limited number of pixels. However, methods for evaluating those devices might be derived from this tool.
Supporting Documentation
- This tool enhances the test methods found in the IEC 80601-2-59: 2017 standard [1] for evaluating the stability and drift, image uniformity, minimum resolvable temperature difference, and radiometric temperature laboratory accuracy of IRTs. The standard provides an overview of test methods as well as background information such as scope, definitions, testing equipment, procedure, and hazard.
- Scientific details about the tool can be found in our publication [2].
- A comparison of the enhanced methods provided by this tool with the methods specified in the standard can be found in the appendix.
Contact
Tool Reference
In addition to citing relevant publications please reference the use of this tool using DOI: 10.5281/zenodo.7883676
For more information:
Note: This tool was previously listed in the catalog as “Laboratory accuracy evaluation of infrared thermographs.”
Appendix: Method for evaluating laboratory accuracy of infrared thermographs
Acronym List
Acronym | Full name |
---|---|
ASTM | American Society for Testing and Materials |
BB | blackbody |
CS | calibration source (usually an extended area blackbody) |
ETRS | external temperature reference source (usually a blackbody with known temperature and emissivity), often part of a ST |
FOV | field of view |
IEC | International Electrotechnical Commission (standards organization) |
IR | Infrared |
IRT | IR thermograph (also called IR camera), often works with an ETRS |
ISO | International Organization for Standardization (standards organization) |
MRTD | minimum resolvable temperature difference |
NCIT | non-contact IR thermometers |
ROI | region of interest |
SD | standard deviation |
ST | screening thermograph, often composed of an IRT and an ETRS |
WTP | workable target plane (can be equal or less than the IRT’s FOV) |
Symbol List
Symbol | Definition |
---|---|
uD | standard uncertainty of the measurement drift |
uS | standard uncertainty of the measurement stability |
uU | standard uncertainty of the measurement uniformity of the WTP |
uER | standard uncertainty of the ETRS temperature |
uMRTD | standard uncertainty caused by the MRTD |
uST | uncertainty of the ST u_ST^2=u_D^2+ u_S^2+ u_U^2+ u_ER^2+u_MRTD^2 |
uCS | standard uncertainty of the CS used in calibration of the ST |
u | combined standard uncertainty of the laboratory accuracy u^2=u_CS^2+ u_ST^2 |
TST | temperature of the CS measured by the ST |
TCS | temperature of the CS |
M8h | mean value for 8 hours measurement |
M8h,min , M8h,max |
minimum and maximum of the mean value for 8 hours measurement |
SD8h | standard deviation for 8 hours of measurement |
Setup, device, and environmental requirements:
- Minimum WTP resolution in thermograms: 320 × 240 pixels
- Minimum resolution for the subject's face in thermograms: 240 × 180 pixels
- Minimum ETRS resolution in thermograms: 20 × 20 pixels
- Minimum spatial resolution of ~1 mm /pixel (i.e., one pixel can image an area of 1×1mm2 on the face)
- Ambient temperature: 18°C – 24°C
- Relative humidity: 10% – 75%
- Airflow from ventilation ducts should be deflected to minimize forced cooling or heating of the target.
- No source of IR radiation (for example, incandescent and halogen lighting) surrounds the experimental setup.
Comparison:
Our study [2] evaluated and enhanced some requirements and test method in the IEC 80601-2-59:2017 standard [1]. The following table compares the differences between the standard methods and our proposed methods.
IEC 80601-2-59: 2017 | Enhancements of RST | |||
---|---|---|---|---|
Stability and drift | Test procedure |
|
|
|
Requirements |
Stability:
Drift:
The combined stability and drift:
|
Stability:
Drift:
|
||
Notes |
|
|||
Uniformity of WTP | Test procedure |
|
|
|
Requirements |
|
uU <0.05°C | ||
Notes |
|
|||
MRTD | Test procedure |
|
|
|
Requirements | MRTD is not larger than 0.1°C | MRTD at the highest spatial frequency (≥0.2 cycles/mrad) is not greater than 0.1°C. | ||
Notes |
|
|||
Radiometric temperature laboratory accuracy | Requirements |
|
||
CS emissivity | Requirements | ≥0.998 (Annex BB) | 0.98±0.01 | |
Note | It is difficult to find a commercial blackbody with emissivity ≥0.998 in the temperature range of 30°C to 40°C (clause 201.101.2.1). While some cavity blackbodies have emissivity around 0.99, they usually operate at temperature higher than 50°C. We were unable to identify a justification for the specified emissivity in the standard (references in the standard do not mention an emissivity of 0.998). We believe CS emissivity of 0.98±0.01 should be sufficient if proper emissivity compensation is applied in the IRT algorithm. | |||
ETRS size | Requirements | Less than 10% of the face (Annex AA) | A larger size (15±20%) is also acceptable if it can be experimentally proven that the ETRS does not adversely affect the measurement. |
Note: Contents in this appendix are based on references [1, 2].
References:
[1] IEC 80601-2-59: Medical electrical equipment - Part 2-59: Particular requirements for the basic safety and essential performance of screening thermographs for human febrile temperature screening, IEC, Geneva, Switzerland, 2017.
[2] P. Ghassemi, T. J. Pfefer, J. P. Casamento, R. Simpson, and Q. Wang, "Best practices for standardized performance testing of infrared thermographs intended for fever screening," PLoS ONE, vol. 13, no. 9, p. e0203302, 2018.
[3] Q. Wang, Y. Zhou, P. Ghassemi, D. McBride, J. P. Casamento, and T. J. Pfefer, "Infrared Thermography for Measuring Elevated Body Temperature: Clinical Accuracy, Calibration, and Evaluation," Sensors, vol. 22, p. 215, 2022.
[4] ASTM E1213-14: Standard Practice for Minimum Resolvable Temperature Difference for Thermal Imaging Systems, ASTM, West Conshohocken, PA, USA, 2014.