Forensics Laboratories Underassess Uncertainty in Blood Alcohol Determinations

In the scientific publication process, studies lacking appropriate validation and quality control are regularly rejected during peer review. Similarly, in forensics, measurements that have not been supported by widely accepted criteria for validation and quality control should not be relied upon in litigation, especially considering that someone’s civil liberties may be at stake.

The uncertainty (or error) associated with a reported value is an important criterion to assess the reliability of a measurement. Accuracy can quickly be called into judgment when uncertainty becomes elevated. Uncertainty should also be assessed regularly, during the course of routine measurement of samples, because instrument performance does not remain constant over time. Instruments have to be regularly maintained and repaired, because their performance will eventually deteriorate with use.

Uncertainty should also be comprehensively assessed on the instrument in question. Performance results obtained from one instrument should not be used to indicate the performance of a different instrument. This statement is obvious to the readership of LCGC, but such assessments—using results from one instrument to validate performance of another—has been commonly encountered in our review of forensic laboratory documentation.

When a blood alcohol concentration is reported, it is usually accompanied by a value for uncertainty at the 99.7% confidence interval. In a large collection of cases we have reviewed, this level of uncertainty has been declared to be 4.3% (for example, 0.188 ± 0.008 g/dL). This assertion claims that the “true” result for this blood alcohol determination has a 99.7% chance of being between 0.180 and 0.196 g/dL, and only a 0.3% chance of being outside that range.

When you look in these cases to see from where the 4.3% uncertainty value is derived, you find that it has been assessed solely based on a) the repeated analysis of calibration and control standards in neat aqueous solution with internal standardization; and b) the manufacturer’s indicated uncertainty in the certified reference ethanol standards that they provided. In the end, they ascribe more than 70% of the assessed total variability in a reported blood alcohol determination to the variability coming from the repeat analysis of pure standards, with the remainder being attributable to the variability in the concentration of the certified reference materials, as assessed by the manufacturer.

In our opinion, this is a gross underassessment of uncertainty, especially for a method that is intended to measure a chemical substance from a biological fluid. Additionally, this uncertainty evaluation is only performed semi-annually, and the assessed uncertainty determined (4.3% at the 99.7% confidence interval) is applied across all instruments in, and results from, laboratories in the forensic laboratory system for blood alcohol determinations. Such a level of uncertainty can hardly be expected to be consistent for every instrument and operator in a large system of forensics laboratories, nor does it contain an assessment of uncertainty arising from biological matrices.

Overall, the level of uncertainty provided by most forensic laboratories for reported blood alcohol results has been woefully underassessed. The methodology that has been used to estimate uncertainty does not capture changing variability amongst different instruments and instrument types, as they are used over time. It does not capture variability associated with the preparation and measurement of complex biological samples, and it definitely does not capture variability in sample handling and storage. When these sources of error are not adequately assessed, then they can only be accounted by assuming reasonable levels of the variability possible for each. When those errors are propagated together with the limited assessment of variability from the forensic laboratory, then the window of “true” values represented by a reported measurement becomes much wider, such that the accuracy of the measurement, especially relative to some threshold (for example, 0.08 g/dL) becomes very debatable. Without proper assessment of the uncertainty of a method, the accuracy of the result it provides cannot be reliably established. In many of the cases we reviewed, forensics laboratories need to revise their procedures for uncertainty assessment, to be more realistic.

References

(1) Schug, K. A.; Hildenbrand, Z. L. Accredited Forensics Laboratories Are Not Properly Validating and Controlling Their Blood Alcohol
Determination Methods. LCGC N. Am. 2022, 40 (8), 370–371. DOI: 10.56530/lcgc.na.hz5482n7

(2) Schug, K. A. Fundamentals: Full Method Validation is Still a Glaring Deficiency in Many Forensics Laboratories. LCGC N. Am. 2021, 39 (11) 200.

(3) Schug, K. A. Forensics, Lawyers, and Method Validation—Surprising Knowledge Gaps. The LCGC Blog. 8 June 2015. http://www.chromatographyonline.com/lcgc-blog-forensics-lawyers-and-method-validation-surprising-knowledge-gaps (accessed 2023-04-26)

(4) AAFS Standards Board, ANSI/ASB Standard 036, First Edition 2019. Standard Practices for Method Validation in Forensics Toxicology. https://www.aafs.org/sites/default/files/media/documents/036_Std_e1.pdf (accessed 2023-04-26)

(5) AAFS Standards Board, ANSI/ASB Standard054, First Edition 2021. Standard for a Quality Control Program in Forensic Toxicology Laboratories. https://www.aafs.org/sites/default/files/media/documents/054_Std_e1.pdf (accessed 2023-04-26)

About the Authors

Kevin A. Schug is a Full Professor and the Shimadzu Distinguished Professor of Analytical Chemistry in the Department of Chemistry & Biochemistry at The University of Texas at Arlington, and a partner of Medusa Analytical. Zacariah L. Hildenbrand is a partner of Medusa Analytical. Direct correspondence to: kschug@uta.edu