PP1317 - Evaluation of Readability and Comprehensibility of PROMs for Childhood Listening Difficulties

Using patient reported outcome measures (PROMs) for evaluating the success of interventions. With increasing emphasis on using PROMs, it is important that readability and comprehensibility be considered, as respondent burden can negatively impact reliability and validity of questionnaires. This research aligns with an increasing national effort to address health literacy issues [Healthy People 2030] and we explore the readability and comprehensibility of PROMs focused on children with listening difficulties.

Summary:

Purpose: Using patient-related outcome measures (PROMs) is common practice in many medical settings and is currently considered best practice for evaluating the success of healthcare interventions (Clerehan, Guillemin, Epstein, & Buchbinder, 2016). With the increasing emphasis on the use of PROMs, it is important that reading comprehension levels be considered, as respondent burden can negatively impact both reliability and validity of the questionnaire (SAC-MOT, 2002). It is therefore imperative that PROMs are designed to yield the most accurate responses as they inform what treatment is undertaken imperative that PROMs are designed to yield the most accurate responses as they inform what treatment is undertaken primarily, as well as serve as a key patient-centered measure of treatment outcomes. This research aligns with an increasing national effort to address health literacy issues [Healthy People 2030] and explore PROMs focused on children with listening difficulties.

Method: Ten listening difficulty PROMs were analyzed using reading grade level formulas and the ELF-Q (Evaluative Linguistic Framework for Questionnaires). After obtaining the original version of each PROM, each was converted to a Word document containing the title, description, instructions, and items. These were run through the Windows-based software Readability Studio Version 2019 (Oleander Software, Ltd., Vandalia, OH, USA) to determine reading level scores using the same formulas as Atcherson, Richburg, Zraick, and George (2013). Each PROM was then analyzed by the first two authors separately using the ELF-Q (Clerehan et al., 2016), after which the two authors convened to form a consensus regarding scoring and recommendations for improvement. The ELF-Q was adapted from the Evaluative Linguistic Framework (Clerehan, Buchbinder, & Moodie, 2015) as a method of evaluating the comprehensibility of questionnaires. Nine items are identified as contributing to reader understanding. The ELF-Q tool provides a description with assessment probes for each item except factual content of text. As part of the ELF-Q, lexical density and technicality of vocabulary were examined. The lexical density was determined by using a free online text analyzer for computation. The technicality of vocabulary was determined by comparing the vocabulary used in the questionnaire to the New General Service List (Brezina & Gablasova, 2013).  

Results: This study found that most PROMs have not been revised since 2013 and readability levels remained largely unchanged. However, the ELF-Q has helped us to identify issues with various PROMs that may make it difficult for some adults to complete properly without some instruction from the professional requesting the PROM. Almost all PROMs were found to need changes, however, the changes needed were diverse in nature. Most commonly, minor changes in formatting were recommended. We will present practical solutions when readability or format cannot be altered without affecting psychometric properties of the original or revised PROMs.

Conclusion: To decrease respondent burden, readability and PROMs format should be taken into consideration. Clinicians should be aware of the respondent’s health literacy skills. Clinicians also should know how to give meaningful directions and clarification when necessary to help ensure reliability and validity of the data on the PROM.