XVI. Appendix to Class II Special Controls Guidance Document: Antimicrobial Susceptibility Test (AST) Systems
TABLE 1: Recommendations for Antimicrobial Susceptibility Devicesa
MIC/BP | Fastidiousb | Additional methods of | ||
---|---|---|---|---|
Number of Sites (including 1 in-house) | 3 | 3 | 3 | |
Organisms | Freshd Clinical/stocke | 100/site | 75/site | 0 |
CDC Challengef | 75/one site | 50/one site | 75 or 50/one site | |
Reproducibilityg | 25/site or 10x3x3/site | 25/site or 10x3x3/site | 25/site or 10x3x3/site | |
Interpretive Standards | FDA | FDA | FDA | |
Stability (3 lots) | Real time (on file) | Real time (on file) | Real time (on file) | |
QC Reference and Test Device Results | CLSI Strains | 20 results/site | 20 results/site | 20 results/site |
(Other Mfg. Recommended) | Optional | Optional | Optional | |
On-scale | At least 1 | At least 1 | At least 1 | |
Inoculum density checkh | QC, reproducibility, fresh | QC, reproducibility, fresh | QC, reproducibility | |
CLSI Reference Method | MIC | MIC | MIC |
- See Tables 5, 8, and 9 for statistical calculation examples and evaluable results.
- For Fastidious organisms such as Streptococcus, Haemophilus, anaerobes, etc. that have an CLSI approved standard methodology, FDA interpretive criteria and quality control recommendations, refer to CLSI approved standard M100 4 Table 1A. The routine testing of rare isolates is not recommended.
- Minimal data to establish performance should be presented for each procedural option of the method of inoculation (growth, direct colony suspension etc.), reading of results (visual vs. automated), or combinations of options.
- Fresh clinical - an organism isolated from a clinical specimen which has been on an agar plate for less than 7 days and not frozen.
- Stock organisms - any organism from a clinical specimen which has been isolated greater than 7 days prior to testing. Should not include organisms for which the antimicrobial agent is not intended. Selection should be supplemental based on the listing in the FDA approved pharmaceutical antimicrobial agent package insert and should not comprise more than 50% of the clinical isolates.
- Challenge - CDC or reference laboratory source with known results (preferably on-scale) to be tested on the test system. Organisms that are intended for the testing with the antimicrobial agent as stated in the FDA approved pharmaceutical antimicrobial agent package insert (microbiology section), should be selected for testing on the test device.
- All on-scale results.
- Inoculum density check should be performed daily on the QC isolates, on reproducibility isolates, and 10% of fresh isolates. Alternate approaches may be substituted if the inoculum method uses a spectrophotometric device.
TABLE 2: Recommendations for the Removal of Limitations from Antimicrobial Susceptibility Devicesa
Items | Performance | Insufficient Resistant | Quality Controlb | Reproducibility | |
---|---|---|---|---|---|
Number of Sitesc | 3 | 1 | 3 | 3 | |
Organisms | Fresh or | 100/site | 75d | 0 | NA |
Challenge | 75/one site | as needed | 0 |
| |
Reproducibilityae | NA | NA | NA | 25/site or 10x3x3/site | |
Quality Control | Daily | Daily | 20 | Daily |
- For Statistically evaluable numbers see Tables 5, 8 and 9
- To be used to demonstrate that the QC ranges are now in the same ranges as the FDA/CLSI
- One may be in-house
- A minimum of 75 organisms either resistant or clustered near the susceptible threshold.
- On-scale
Note:
- If changes have been made to the device to alter the overall performance, the testing should include all organisms previously tested.
Refer to Table 1. - Perform testing for all procedural options.
TABLE 3: Presentation of Summary Data for Both Challenge and Clinical Data
Clinical Data
Organism Group | Total Tested | # EA | %EA | Total Evaluable | # EA of Evaluable | %EA of Evaluable | # CA | %CA | # R | # vmj | # maj | # min |
---|---|---|---|---|---|---|---|---|---|---|---|---|
K. pneumoniae | 79 | 74 | 93.7 | 64 | 62 | 92.2 | 70 | 88.6 | 20 | 0 | 1 | 8 |
P. aeruginosa | 96 | 90 | 93.8 | 91 | 85 | 93.4 | 88 | 91.7 | 40 | 0 | 1 | 7 |
C. freundii | 26 | 21 | 80.8 | 18 | 14 | 72.2 | 23 | 88.5 | 10 | 0 | 0 | 3 |
E. aerogenes | 22 | 21 | 95.5 | 21 | 21 | 95.2 | 21 | 95.5 | 2 | 0 | 0 | 1 |
E. cloacae | 57 | 53 | 93 | 50 | 48 | 92 | 51 | 89.5 | 13 | 1 | 1 | 4 |
M. morganii | 15 | 14 | 93.3 | 12 | 12 | 91.7 | 15 | 100 | 7 | 0 | 0 | 0 |
P. mirabilis | 34 | 26 | 76.5 | 34 | 29 | 76.5 | 34 | 100 | 33 | 0 | 0 | 0 |
E. coli | 92 | 85 | 92.4 | 62 | 59 | 88.7 | 87 | 94.6 | 43 | 2 | 1 | 2 |
S. marcescens | 50 | 49 | 98 | 48 | 48 | 100 | 47 | 94 | 32 | 0 | 0 | 3 |
Acinetobacter spp | 41 | 40 | 97.6 | 29 | 28 | 96.6 | 34 | 82.9 | 14 | 0 | 0 | 7 |
TOTAL | 512 | 473 | 92.4 | 429 | 406 | 94.6 | 470 | 91.8 | 214 | 3 | 4 | 35 |
Challenge
Organism Group | Total Tested | # EA | %EA | Total Evaluable | # EA of Evaluable | %EA of Evaluable | # CA | %CA | # R | # vmj | # maj | # min |
---|---|---|---|---|---|---|---|---|---|---|---|---|
K. pneumoniae | 10 | 10 | 100 | 10 | 10 | 100 | 10 | 100 | 6 | 0 | 0 | 2 |
P. aeruginosa | 20 | 18 | 90 | 17 | 15 | 88.2 | 19 | 95 | 15 | 1 | 0 | 0 |
C. freundii | 10 | 10 | 100 | 9 | 9 | 100 | 9 | 90 | 2 | 0 | 1 | 0 |
E. aerogenes | 5 | 4 | 80 | 4 | 3 | 75 | 5 | 100 | 0 | 0 | 1 | 1 |
E. cloacae | 5 | 5 | 100 | 2 | 2 | 100 | 5 | 100 | 4 | 0 | 0 | 1 |
Aeromonas sp. | 5 | 4 | 80 | 4 | 4 | 100 | 5 | 100 | 0 | 0 | 0 | 0 |
P. mirabilis | 10 | 9 | 90 | 8 | 6 | 75 | 9 | 90 | 4 | 0 | 0 | 2 |
S. marcescens | 10 | 10 | 80 | 9 | 9 | 100 | 9 | 90 | 2 | 0 | 0 | 1 |
Acinetobacter spp | 10 | 8 | 80 | 5 | 5 | 100 | 8 | 80 | 2 | 1 | 0 | 2 |
TOTAL | 85 | 78 | 91.8 | 68 | 63 | 92.6 | 79 | 92.9 | 35 | 2 | 2 | 9 |
Clinical and Challenge Combined
Organism Group | Total Tested | # EA | %EA | Total Evaluable | # EA of Evaluable | %EA of Evaluable | # CA | %CA | # R | # vmj | # maj | # min |
---|---|---|---|---|---|---|---|---|---|---|---|---|
All organisms | 597 | 551 | 92.3 | 497 | 469 | 94.4 | 549 | 92.0 | 249 | 5 | 6 | 44 |
TABLE 4: Example of Reporting Format for Quality Control Data
Antimicrobial agent: ___________________________
QC Organism | Expected Result |
| Reference Result Frequency | New Device Frequency | ||||
---|---|---|---|---|---|---|---|---|
E. coli ATCC 25922 | 0.25 - 1.0 mg/mL |
| Site 1 | Site 2 | Site 3 | Site 1 | Site 2 | Site 3 |
|
|
| 1 | 1 |
| |||
.25 | 14 | 18 | 14 | 4 | 14 | 14 | ||
.5 | 6 | 2 | 6 | 4 | 5 | 6 | ||
1.0 |
|
|
| 11 |
|
| ||
>1.0 |
|
|
|
|
|
| ||
E. cloacae Ref 1611 | 2 - 8 mg/mL
|
|
|
|
|
|
| |
2 |
|
|
| 12 | 14 | 4 | ||
4 | 14 | 15 | 12 | 2 | 6 | 5 | ||
8 | 6 | 5 | 8 | 8 |
| 11 | ||
>8 |
| >
| >
| >
| >
| >
| ||
Pseudomonas aeruginosa ATCC 27853 | 0.5 - 2 mg/mL
| >
|
| 2 |
|
| 2 | |
.5 | 10 | 10 | 18 |
| 2 |
| ||
1 | 5 | 8 | 2 | 20 | 18 | 18 | ||
2 | 5 | 2 |
|
|
| 4 | ||
4 |
|
|
|
|
|
| ||
Enterococcus faecalis ATCC 29212 | 4 - 16 mg/mL
| 2 |
|
|
|
|
|
|
4 | 18 | 2 | 18 | 20 | 12 | 6 | ||
8 | 2 | 18 | 2 |
| 8 | 14 | ||
16 |
| >
| >
| >
| >
| >
|
- Performed daily with a minimum of 20 per site.
- List all reference and test results including out of range results that required repeat testing.
- To be used for all procedural options.
TABLE 5: Sample Table Format for Device Performance
Antimicrobic agent: Oxacillin
| Organism: Staphylococcus aureus
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Antimicrobic agent: Oxacillin
| Organism: Staphylococcus epidermidis
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Antimicrobic agent: Oxacillin
| Organism: other CNS
|
Present all fresh and stock results for organisms in a genus/species in a separate diagonal. For example all Staphylococcus aureus results from all sites would appear in the first diagonal.
Antimicrobic agent: Oxacillin
| Organism: All
|
a Results are not considered evaluable if they fall in this box
b Essential Agreement (EA) = (# exact agreement or within plus or minus one two-fold dilution of the reference method / total # organisms tested) X 100
c Category Agreement (CA) = (# with interpretive agreement to the reference method / total # organisms tested) X 100
d Minor Discrepancy (min) = (# min discrepancies based on interpretation / total # organisms tested) X 100
e Major Discrepancy (maj) = (# maj discrepancies based on interpretation / total # susceptible organisms by reference method) X 100
f Very Major Discrepancy (vmj) = (# vmj discrepancies based on interpretation / totla # resistant organisms by reference method) X 100
To be used for:
- All fresh and stock organisms for each site presented in one 2A diagonal.
- All fresh and stock organisms for all sites combined presented in one 2A diagonal.
- All challenge organisms presented in one 2A diagonal chart.
- All challenge organisms presented in one 2A diagonal chart for each method variation.
- All organisms (challenge, fresh, stock) combined and presented in one 2A diagonal chart. Do not combine different method variations.
TABLE 6: Presentation of Reproducibility Results a by Organism
Difference in the number of wells between test result and test mode | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Off- scale | -2 | -1 | 0 | +1 | +2 | Off- scale | Site 1 result | Site 2 result | Site 3 result | Test Modeb | |
P. aeruginosa | 3 | 0.5 | 0.5 | 0.5 | 0.5 | ||||||
P. aeruginosa | 2 | 1 | 4 | 2 | 2 | 2 | |||||
P. aeruginosa | 1 | 2 | 16 | 32 | 32 | 32 | |||||
P. aeruginosa | 2 | 1 | 8 | 16 | 8 | 8 | |||||
E. coli | 3 | 2 | 2 | 2 | 2 | ||||||
E. coli | 1 | 2 | 8 | 16 | 16 | 16 | |||||
E. coli | 2 | 1 | 4 | 2 | 2 | 2 | |||||
E. coli | 3 | 1 | 1 | 1 | 1 | ||||||
E. coli | 1 | 2 | 2 | 4 | 4 | 4 | |||||
E. coli | 3 | 32 | 32 | 32 | 32 | ||||||
M. morganii | 1 | 2 | 16 | 16 | 8 | 16 | |||||
C. diversus | 2 | 1 | 0.5 | 0.5 | 1 | 0.5 | |||||
C. freundii | 2 | 1 | 1 | 2 | 1 | 1 | |||||
C. freundii | 3 | 16 | 16 | 16 | 16 | ||||||
E. cloacae | 2 | 1 | 16 | 32 | 16 | 16 | |||||
E. cloacae | 2 | 1 | 0.5 | 0.5 | 2 | 0.5 | |||||
E. cloacae | 3 | 4 | 4 | 4 | 4 | ||||||
P. mirabilis | 3 | 8 | 8 | 8 | 8 | ||||||
P. mirabilis | 3 | 2 | 2 | 2 | 2 | ||||||
S. marcescens | 2 | 1 | 2 | 1 | 1 | 1 | |||||
S. marcescens | 2 | 1 | 32 | 16 | 16 | 16 | |||||
S. marcescens | 2 | 1 | 4 | 2 | 2 | 2 | |||||
K. pneumoniae | 2 | 1 | 32 | 32 | >32 | 32 | |||||
K. pneumoniae | 2 | 1 | 0.25 | 0.5 | 0.25 | 0.25 | |||||
P. stuartii | 1 | 2 | 16 | 16 | 4 | 16 | |||||
Total | 1 | 4 | 58 | 10 | 1 | 1 | |||||
Between-site reproducibilityc | 73/75 = 97.3%d |
a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 25 on-scale organisms, tested at 3 sites.
b Most frequent new test result. If there is no mode, the median should be used.
c Total number of results that fall within 1 well (+/-1) of the mode result divided by total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all three results are off-scale they should still be included in the calculation as part of the best-worst case calculations. For this study the denominator would always be 75 whether the results are off or on-scale.
d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.
TABLE 6A: Presentation of Reproducibility Resultsa by Organism and Site
Difference in the number of wells between new test result and test mode | ||||||||
---|---|---|---|---|---|---|---|---|
Off-scale | -2 | -1 | 0 | +1 | +2 | Off-scale | Test Modeb | |
|
|
|
| Site 1 |
|
|
|
|
E. coli 1 | 6 | 3 | 0.5 | |||||
E. coli 2 | 9 | 2 | ||||||
E. coli 3 | 6 | 3 | 2 | |||||
Pseudomonas 4 | 3 | 6 | 16 | |||||
Pseudomonas 5 | 3 | 5 | 1 | 32 | ||||
Klebsiella 6 | 8 | 1 | 8 | |||||
Enterobacter 7 | 8 | 1 | 8 | |||||
Serratia 8 | 4 | 5 | 4 | |||||
Serratia 9 | 5 | 4 | 16 | |||||
Proteus 10 | 5 | 4 | 1 | |||||
Total | 20 | 52 | 16 | 1 | 1 | |||
Within-Site Reproducibilityc | 89/90 = 98.9%d | |||||||
|
|
|
|
|
|
|
| |
|
|
|
|
|
|
|
| |
|
|
| Site 2 |
|
|
|
| |
|
|
|
|
|
| |||
E. coli 1 |
|
| 3 | 6 |
|
|
| 0.5 |
E. coli 2 |
|
|
| 9 |
|
|
| 2 |
etc… |
|
|
|
|
|
|
|
|
a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 10 on-scale organisms, tested in triplicate on 3 separate days at 3 sites.
b Most frequent test result.
c Total number of results that fall within 1 well (+/-1) of the mode result divided by total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all results are off-scale they should still be included in the calculation. For this study the denominator would be ninety whether they are all on scale or not.
d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.
TABLE 6B: Presentation of Reproducibility Resultsa by Organism, Pooled Across Sites
Difference in the number of wells between new test result and test mode | ||||||||
---|---|---|---|---|---|---|---|---|
Off-scale | -2 | -1 | 0 | +1 | +2 | Off-scale | Test Modeb | |
All Sites | ||||||||
|
|
|
|
|
|
|
|
|
E. coli 1 | 9 | 18 | 0.5 | |||||
E. coli 2 | 25 | 2 | 2 | |||||
E. coli 3 | 3 | 9 | 12 | 3 | 2 | |||
Pseudomonas 4 | 6 | 12 | 9 | 16 | ||||
Pseudomonas 5 | 2 | 17 | 8 | 32 | ||||
Klebsiella 6 | 7 | 14 | 5 | 1 | 8 | |||
Enterobacter 7 | 5 | 14 | 8 | 8 | ||||
Serratia 8 | 4 | 15 | 8 | 4 | ||||
Serratia 9 | 3 | 20 | 4 | 16 | ||||
Proteus 10 | 3 | 14 | 10 | 1 | ||||
Total | 3 | 48 | 161 | 49 | 1 | 8 | ||
Between-site reproducibilityc | 266/270 = 98.5%d |
a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 10 on-scale organisms, tested in triplicate on 3 separate days at 3 sites.
b Most frequent test result.
c Total number of results that fall within 1 well (+/-1) of the mode result divided by Total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all results are off-scale they should be included in the calculation with a denominator of 270 for all calculations.
d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode
TABLE 7: Report Format for Inoculum Density
ORGANISMa | NUMBER TESTED | SOURCE | METHODb | MEAN | MINUMUM | MAXIMUM | STD. DEV. |
---|---|---|---|---|---|---|---|
S. aureus ATCC # | 20 | QC ATCC | Reference | 6 X 10 5 | 2 x 10 5 | 9 x 10 5 | |
S. aureusATCC # | 20 | QC ATCC | Direct inoculum | 5 x 10 5 | 2 x 10 5 | 8 x 10 5 | |
S. aureusATCC # | 20 | QC ATCC | Growth inoculum | 5 x 10 5 | 2 x 10 5 | 6 x 10 5 | |
MRSA | 13 | Reproducibility, clinical | Direct inoculum | 7 x 10 5 | 4 x 10 5 | 6 x 10 6 | |
MRSA | 13 | Reproducibility, clinical | Growth inoculum | 6 X 10 5 | 2 x 10 5 | 8 x 10 6 | |
MSSE | 3 | Reproducibility | Direct inoculum | 8 x 10 5 | 5 x 10 5 | 7 x 10 5 | |
MSSE | 3 | Reproducibility | Growth inoculum | 7 x 10 5 | 4 x 10 5 | 12 x 10 5 | |
MRSE | 19 | Reproducibility | Direct inoculum | 6 X 10 5 | 2 x 10 5 | 8 x 10 6 | |
MRSE | 19 | Reproducibility | Growth inoculum | 7 x 10 5 | 5 x 10 5 | 7 x 10 5 | |
Enterococcus | 4 | Clinical | Direct inoculum | 9 x 10 5 | |||
Enterococcus | 4 | Clinical | Growth inoculum | ||||
MSSA | 15 | Clinical | Direct inoculum | ||||
MSSA | 15 | Clinical | Growth inoculum | ||||
Direct inoculum | |||||||
Growth inoculum |
- Data should be available upon request for “by site” evaluation, by organism, etc.
- Inoculum density should be performed on all methods of inoculation unless a standardized method (photometric device) is used.
TABLE 8: Number of VMJ Discrepancies as a Function of the Number of Resistant Organisms Tested
Number of Resistant Organisms | Acceptable Number of Discrepancies | Estimated Ratea | 95% Confidence Intervalb for True VMJ Rate |
---|---|---|---|
48 | 0 | 0.00 | (0.00, 7.40) |
50 | 0 | 0.00 | (0.00, 7.11) |
60 | 0 | 0.00 | (0.00, 5.96) |
70 | 0 | 0.00 | (0.00, 5.13) |
72 | 1 | 1.39 | (0.04, 7.50) |
80 | 1 | 1.25 | (0.03, 6.77) |
90 | 1 | 1.11 | (0.03, 6.04) |
94 | 2 | 2.13 | (0.26, 7.48) |
100 | 2 | 2.00 | (0.24, 7.04) |
110 | 2 | 1.82 | (0.22, 6.41) |
120 | 3 | 2.50 | (0.52, 7.13) |
130 | 3 | 2.31 | (0.48, 6.60) |
140 | 4 | 2.86 | (0.78, 7.15) |
150 | 4 | 2.67 | (0.73, 6.69) |
160 | 5 | 3.13 | (1.00, 7.20) |
170 | 5 | 2.94 | (0.94, 6.78) |
180 | 6 | 3.33 | (1.21, 7.16) |
190 | 7 | 3.68 | (1.48, 7.48) |
200 | 7 | 3.50 | (1.40, 7.12) |
250 | 8 | 3.20 | (1.38, 6.24) |
300 | 9 | 3.00 | (1.37, 5.64) |
400 | 11 | 2.75 | (1.37, 4.88) |
500 | 13 | 2.60 | (1.39, 4.41) |
600 | 15 | 2.50 | (1.40,4.09) |
700 | 19 | 2.43 | (1.42,3.86) |
- Est. Rate = estimated vmj rate = number of vmj discrepancies divided by number of resistant organisms.
- Exact confidence intervals based on the binomial distribution.
TABLE 9: Essential Agreement as Function of the Number of Evaluable Organisms Tested
Number of Evaluablea Organisms | Acceptable Number of Disagreements | Estimated Essential Agreement (EA)b | 95% Confidence Intervalc for True EA |
---|---|---|---|
35 | 0 | 100.00 % | (90.00, 100.00) |
54 | 1 | 98.15 | (90.11, 99.95) |
55 | 1 | 98.18 | (90.28, 99.95) |
60 | 1 | 98.33 | (91.06, 99.96) |
65 | 1 | 98.46 | (91.72, 99.96) |
70 | 2 | 97.14 | (90.06, 99.65) |
75 | 2 | 97.33 | (90.70, 99.68) |
80 | 2 | 97.50 | (91.26, 99.70) |
85 | 3 | 96.47 | (90.03, 99.27) |
90 | 3 | 96.67 | (90.57, 99.31) |
95 | 3 | 96.84 | (91.05, 99.34) |
100 | 4 | 96.00 | (90.07, 98.90) |
110 | 4 | 96.36 | (90.95, 99.00) |
120 | 5 | 95.83 | (90.54, 98.63) |
130 | 6 | 95.38 | (90.22, 98.29) |
140 | 6 | 95.71 | (90.91, 98.41) |
150 | 7 | 95.33 | (90.62, 98.10) |
160 | 8 | 95.00 | (90.39, 97.82) |
170 | 9 | 94.71 | (90.19, 97.55) |
180 | 10 | 94.44 | (90.02, 97.30) |
190 | 10 | 94.74 | (90.53, 97.45) |
200 | 11 | 94.50 | (90.37, 97.22) |
- Evaluable (e.g. on-scale) organisms are those that fall within the test range of the reference and have the opportunity for a result on the test method that could also be on-scale. Any reference result that falls in the category is considered not evaluable.
- Estimated Essential Agreement = percent agreement = number of evaluable test results that are equal to or with in one dilution of the expected result divided by number of organisms that are evaluable.
- Exact confidence intervals based on the binomial distribution.