Transcript for Using Machine Learning in Residency Applicant Screening

We reported one year in the paper, but we’ve done it two years now. And filled 20 spots of individuals who, for whatever reason, were not invited on the initial review. But the algorithm suggested, “Hey, this person could be an amazing fit for our program.” And what was probably the most interesting thing, and again, it’s sort of qualitative findings but was that when you asked the program directors to look back at those folks who were not invited initially, but the algorithm said, let’s give them a consideration it was often metrics related issues. Jesse Burk-Rafel: They came from maybe a school that was not as prestigious, or their Step scores were not as high. Those are features used in our selection process, like many other programs, no shocker there. But I am bullish, that this kind of approach could give a second look to people who might be actually a very good fit for our program. So that’s the high-level findings. The only last piece I would say is, we did do an analysis, what we’d call a sensitivity analysis, where we said, “Well, what if we took the same exact approach, but this time, we leave out all USMLE scores. Step 1 and Step 2 CK.” Jesse Burk-Rafel: We kept whether someone passed or failed, we thought that was fair, given the change to pass/fail reporting for Step 1, but we left out the actual score. And indeed the performance dropped a teeny bit, but not clinically significant amount. The model cou...
Source: Academic Medicine Blog - Category: Universities & Medical Training Authors: Tags: AM Podcast Transcript Audio AI machine learning medical education residency application resident selection RIME Source Type: blogs