|
|
Volume 2: No. 2, April 2005
SPECIAL TOPICS
ORIGINAL RESEARCH: FEATURED
ABSTRACT FROM THE 19TH NATIONAL CONFERENCE ON CHRONIC DISEASE
PREVENTION AND CONTROL
Response Shift: The Measurable and Desired Outcome of Chronic Disease Self-management
Programs That Violates Pre–Post Assessment
Melanie Hawkins, Richard Osborne
Suggested citation for this article: Hawkins M, Osborne R. Response shift:
the measurable and desired outcome of chronic disease self-management programs
that violates pre-post assessment [abstract]. Prev Chronic Dis [serial online] 2005 Apr [date cited]. Available from: URL: http://www.cdc.gov/pcd/issues/2005/
apr/04_0142u.htm.
PEER REVIEWED
Track: Evidence-based Programs: Research, Translation, and Evaluation
Chronic disease self-management programs are designed not only to empower participants and increase their knowledge but also to provide participants with existential insights into their health and ability to self manage. The outcome of a self-management program is traditionally evaluated by comparing pre-intervention and post-intervention questionnaires, a methodology that assumes that
participants answer questions from the same perspective before and after the program. Systematic reviews of the outcomes from self-management interventions that use self-appraisal outcomes identify small to no effects. This study of community-based self-management programs in Australia aimed to determine if changes in internal values or perspectives (termed a response shift) occurred in
participants and if response shift is measurable with a paper-based questionnaire.
The HEI-Q-Perspective Questionnaire, a retrospective nine-item post-test
questionnaire, was developed to measure potential benefits of self-management
programs. Cognitive interviews with respondents elicited spontaneous statements
about the reasons for their paper-based answers across the nine items. The sensitivity, specificity, and overall accuracy of the questionnaire were
calculated using the interview as the gold standard. A response shift can be negative (i.e., after the course, participants report they now realize that before the course they were worse than they thought they were), positive (i.e., participants now realize they were better before the course than they thought they were), or neutral (no change).
In-depth interviews (n = 39) and mailed questionnaires (n = 132) reflected that a
“true” response shift occurred in about half of the questionnaire items. Of these,
33% had a negative response shift, 18% had a positive response shift, and approximately
32% had no response shift. The presence or absence of response shift could not
be determined in approximately 17% of cases. Substantial concordance between interview
and questionnaire were observed (average overall accuracy 0.79), indicating the
questionnaire effectively identified response shift. A positive or negative
response shift was found to have profound effects on patient-reported outcomes —
even large positive or large negative program effects revealed in an interview
could be concealed in an individual’s pre–post score. This clearly demonstrates that response shift
can violate classic outcome assessment of self-management programs.
Response shift occurred in about half of the participants. This result suggests that classic outcome assessment (pre-test vs post-test) in many individuals is flawed. Response shift is a desirable outcome of courses but has not been formally measured. The strong concordance between the questionnaire and cognitive interviews indicates the HEI-Q-Perspective can detect response shift. This new
questionnaire will assist researchers and program evaluators to better estimate the impact of self-management programs and to understand the role of response shift in this and other settings.
Corresponding Author: Richard H. Osborne, BSc, PhD, The University of Melbourne Department of Medicine, Centre for Rheumatic Diseases, Royal Melbourne Hospital, Melbourne, Australia 3050. Telephone:
+61-3-9342-8561. E-mail: richardo@unimelb.edu.au.
Back to top |
|
