PRIISM Seminar by Swarthmore College's Amanda Luby
This seminar will be recorded and posted on our website.
Join PRIISM and Dr. Amanda Luby to learn how using Item Response Theory can improve the analysis and interpretation of forensic science.
Forensic science often involves the evaluation of crime-scene evidence to determine whether it matches a known-source sample, such as determining if a fingerprint or DNA was left by a suspect. Even as forensic measurement and analysis tools become increasingly sophisticated, final source decisions are often left to individual examiners' interpretation. However, the current approach to characterizing uncertainty in forensic decision-making has largely centered around conducting error rate studies (in which examiners evaluate a set of items consisting of known-source evidence) and calculating aggregated error rates. This approach is not ideal for comparing examiner performance, as decisions are not always unanimous and error frequency is likely to vary depending on the quality of the physical evidence. Item Response Theory (IRT), a class of statistical methods used prominently in educational testing, is one approach that accounts for differences in proficiency among participants and additionally accounts for varying difficulty among items. Using simple IRT models, more elaborate decision tree models, and extensions, along with data from the FBI “Black Box” and “White Box” studies, Dr. Luby and her team find that there is considerable variability in print quality assessments, inconclusive rates, perceived difficulty, and minutiae identification even when examiners largely agree on a final source decision. In this talk, Dr. Luby will review some of our recent advances, outline challenges in applying IRT in practice, and discuss the implications of these findings within the criminal justice system.
Amanda Luby is an assistant professor of statistics at Swarthmore College and holds a PhD from Carnegie Mellon University. Her research is centered around developing statistically-sound methodology to better understand individual differences in decision-making behavior, often using Bayesian tools. Dr. Luby currently works with the Center for Statistics and Applications in Forensic Evidence (CSAFE) to improve the accuracy of analysis and interpretation of forensic evidence.