Inter-rater Reliability of the Dysphagia Outcome and Severity Scale (DOSS): Effects of Clinical Experience, Audio-Recording and Training

This study investigated inter-rater reliability (IRR) of the DOSS. It also determined the effect of clinical experience, VFSS audio-recording and training on DOSS IRR. A quantitative prospective research design was used. Seventeen speech and language pathologists (SLPs) were recruited from an acute teaching hospital, Dublin (>  3 years’ VFSS experience,n = 10) and from a postgraduate dysphagia programme in a university setting (<  3 years’ VFSS experience;n = 7). During testing, participants viewed eight VFSS clips (5 with audio-recording). Each VFSS clip was independently rated using the DOSS scale. Four weeks later, the less experienced group attended a 1-h training session on DOSS rating after which DOSS IRR was re-tested. Cohen’s kappa co-effi cient was used to establish IRR. IRR of the DOSS presented only fair agreement (κ = 0.36,p <  0.05). DOSS IRR was significantly higher (κ = 0.342) within the more experienced SLP group, compared to the less experienced SLP group (κ = 0.298) (p <  0.05). DOSS IRR was significantly higher in VFSS clips with audio-recording (κ = 0.287) compared to VFSS clips without audio-recording (κ = − 0.0395) (p <  0.05). IRR of the DOSS pre-training (κ = 0.328) was significantly better comparing to post-training (κ = 0.218) (p <  0.05). Findings raise concerns as the DOSS is frequently used in clinical practice to capture dysphagia severity and to monitor changes.
Source: Dysphagia - Category: Speech-Language Pathology Source Type: research