Intra-rater Kappa Accuracy of Prototype and ICD-10 Operational Criteria-Based Diagnoses for Mental Disorders: A Brief Report of a Cross-Sectional Study in an Outpatient Setting - Rocha Neto, Helio G.; Sinem, Tomas Boldrini; Koiller, Luisa Mendez; Pereira, Amanda Machado; de Souza Gomes, Bianca Marques; Veloso Filho, Carlos Linhares; Cavalcanti, Maria T.; Telles-Correia, Diogo (Frontiers in Psychiatry, 2022)
Background and objectives: The use of "operational criteria" is a solution for low reliability, contrasting with a prototypical classification that is used in clinics. We aim to measure the reliability of prototypical and ICD-10 diagnoses. Methods: This is a retrospective study, with a convenience sample of subjects treated in a university clinic. Residents reviewed their diagnosis using ICD-10 criteria, and Cohen’s kappa statistic was performed on operational and prototype diagnosis. Results: Three out of 30 residents participated, reviewing 146 subjects under their care. Diagnoses were grouped in eight classes: Organic (diagnoses from F00–F09), substance disorders(F10–F19), schizophrenia spectrum disorders (F20–F29), bipolar affective disorder (F30,F31,F34.0,F38.1), depression (F32,F33), anxiety-related disorders (F40–F49), personality disorders (F60–F69), and neurodevelopmental disorders (F70–F99). Overall, agreement was high (K=0.77, 95% confidence interval (CI)=0.69-0.85), with lower agreement related to personality disorders (K=0.58,95%CI=0.38-0.76) and higher with schizophrenia spectrum disorders (K=0.91,95%CI=0.82–0.99) Discussion: Use of ICD-10 criteria did not significantly increase the number of diagnoses. It changed few diagnoses, implying that operational criteria were irrelevant to clinical opinion. This suggests that reliability among interviewers is more related to information gathering than diagnostic definitions. Also, it suggests an incorporation of diagnostic criteria according to training, which then became part of the clinician’s prototypes. Residents should be trained in the use of diagnostic categories, but presence/absence checking is not needed to achieve operational compatible diagnoses. Registration: IPUB’s ethical committee approved this research, registered under the CAAE 33603220.1.0000.5263 and the UTN -U1111-1260-1212.