Using cognitive interviews to improve quantitative survey data

Background

ClearWay Minnesota is an independent nonprofit organization that improves the health of Minnesotans by reducing tobacco use and exposure to secondhand smoke (SHS) through research, action, and collaboration. One of ClearWay’s Legacy goals is to reduce adult SHS exposure to 5% by 2023. The Minnesota Adult Tobacco Survey (MATS) serves as a primary source of information on SHS exposure in the state. However, the secondhand smoke items collected on MATS had been developed over a decade earlier, and the climate of smoking had changed dramatically since then. For example, the items were developed prior to implementation of Minnesota’s smoke-free law, which banned smoking in all workplaces, including bars and restaurants. It was unclear whether the existing items continued to work well within the new context. ClearWay contracted with PDA to conduct a formative research study to examine the functioning of survey items designed to assess Minnesotans’ exposure to SHS and ultimately help inform programming and policy work.

Approach

Cognitive interviews were designed to test the functioning of the current SHS items and provide a preliminary look into the nature of respondent’s SHS exposure. In cognitive interviews, participants are asked to think aloud as they answer survey items so the researcher can better understand their thought processes. Cognitive interviews are a great tool to identify sources of response error in survey items. Cognitive interviews were conducted via telephone with 17 smokers and 20 non-smokers. Interviews were transcribed and analyzed with written notes from the interviews.

Results

Key findings from the study include:

• Items measuring SHS in homes and cars appear to function well. Respondents seemed to have a clear understanding of the language used in these items and the construct assessed.
• Items about exposure to smoke at work appear to actually measure rules about smoking and sometimes result in an over-reporting of exposure to SHS.

• Smokers were often not able to accurately report their exposure to others’ SHS. Smokers report thinking less about their exposure than non-smokers and sometimes assess their exposure to others’ SHS by considering their own smoking behaviors.
• Minnesotans exposure to SHS in multi-unit housing is variable and is not currently measured. New items were tested and appear to accurately identify residence in multi-unit housing and capture exposure to SHS.
• Assessing exposure to SHS in places besides home, workplace, or car was important to measure. In Minnesota, other places, such as parks or building entrances, are often the only incidence of SHS exposure.
• Duration of SHS exposure varied between participants. Many non-smokers were briefly exposed to SHS outdoors. Assessing the duration of exposure can inform decision-makers about future SHS programming and policy work.

Some of the SHS items asked on MATS were modified based on the results of this study. Additionally, many of the items tested are or were used as part of other surveillance assessments in the U.S. and internationally. Therefore, results were shared more broadly via a poster at the 2014 Annual Conference of the Society for Research on Nicotine and Tobacco and publication in Nicotine and Tobacco Research.