Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback

AbstractThe entrustment framework redirects assessment from considering only trainees ’ competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic character istics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differe ntially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors’ themes predominantly reflected skills related to patient presentations, while trainees’ themes were broader—including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment,p <  0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1–4 scale,p <  0.05). We also found biases tied to demographic characteristics: trainees’ documentation reflected more positive sentiment in the case of male trainees (+ 1.3%,p <  0...
Source: Advances in Health Sciences Education - Category: Universities & Medical Training Source Type: research