Authentic assessments: a method to detect anomalies in assessment response patterns via neural network

In this study, we propose a hybrid unsupervised-supervised approach to identify inauthentic assessments via anomalous response patterns. The study utilized 60-question behavioral assessment from a clinical population served by a county-operated public children ’s behavioral health services (n = 42,945). A novel hybrid unsupervised-supervised approach was developed to identify inauthentic assessment records from highly dimensional assessment data without the need for a priori record labels, which would otherwise require countless hours of record revi ew by highly trained clinical staff. A neural network trained with 75% of the labeled data recognized anomalous response patterns in the test data with 84.5% sensitivity and 97.3% specificity. The model identified 26% of records as potentially inauthentic based on anomalous response patterns. For me ntal health and behavioral health, this novel method could flag a relatively small proportion of the records for clinical review while allowing records with probable authenticity to bypass review processes, saving time and creating efficiency in care. This method is also a potential tool for improvi ng service quality through the identification of circumstances that have been overlooked but could otherwise be addressed during care. For decades, assessment data has been collected by policy and ignored by practice due to limitations of utility. With the right approaches, we could uncover the patt erns of what works for whom that ar...

Source: Health Services and Outcomes Research Methodology - November 22, 2021 Category: Statistics Source Type: research