Title: Does participation in an intervention affect responses on self-reported questionnaires? Authors
|Allen, Diane - UNIVERSITY OF CA BERKELEY|
|Masse, Louise - UNIV OF BRITISH COLUMBIA|
|Wilson, Mark - UNIVERSITY OF CA BERKELEY|
Submitted to: Health Education Research
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: July 19, 2006
Publication Date: October 26, 2006
Citation: Baranowski, T., Allen, D. D., Masse, L. C., Wilson, M. 2006. Does participation in an intervention affect responses on self-reported questionnaires? Health Education Research. 21(Suppl 1):i98-i109. Interpretive Summary: Concerns have been raised that participation in an intervention changes the nature of response to measurement scales because of the new knowledge and insights from participating in the intervention. Concerns have also been raised that simply taking a questionnaire once may influence the responses to that questionnaire the second time it is taken. Within the conduct of a behavior change experiment, this paper reanalyzed the data to assess if there were differences in item location between treatment and control groups at post assessment, and to assess if taking the measure once influenced the second taking of the questionnaire in the control group. While there was no evidence that simply taking a measurement once changed the nature of measurement the second time a scale was used, there was some evidence that treatment and control groups located the same items at different points on the underlying distribution. Further research is needed to assess the extent to which theintervention may change the nature of the measurement scale.
Technical Abstract: There has been some concern that participation in an intervention and exposure to a measurement instrument can change participants' interpretation of the items on a self-report questionnaire, thereby distorting subsequent responses and biasing results. Differential item functioning (DIF) analysis using item response modeling can ascertain possible differences in item interpretation by testing for differences in item location between groups. The DIF for treatment versus control group differences at post-intervention assessment and the Time 1 and Time 2 differences in a control group were analyzed using data from a dietary change intervention trial for Boy Scouts. The measures included fruit and vegetable (FV) frequency of consumption, preferences and self-efficacy. Treatment–control group DIF at post-intervention assessment was detected in a higher percentage of items for FV frequency than for preference or self-efficacy. Time 1 to Time 2 differences in items for the control group were detected in one item for each of the three scales. Further research will need to clarify whether the obtained DIFs reflected true changes in frequency, preference or self-efficacy or some reinterpretation of items by participants following an intervention or merely after previous exposure to the measure.