The base used to calculate a percent is a common source of misrepresentation or inaccuracy in survey findings. Last month we highlighted you to these problem bases overlooked by the fact checkers at JAMA, the Journal of the American Medical Association. This month, we'd like to draw your attention to an equally frustrating issue with bases used in the American Psychological Association (APA)'s Stress in America survey.
According to the 2014 edition of the survey, "the most commonly reported sources of stress include money (64 percent report that this is a very or somewhat significant source of stress), work (60 percent), the economy (49 percent), family responsibilities (47 percent) and personal health concerns (46 percent)".
These findings are based on All qualified respondents (trend chart on page 2) but qualification was not systematic across the individual source of stress metrics. To respond and be counted for the Work metric, for example, the respondent had to be employed. To respond and be counted for the Family responsibilities or Health Concerns metrics, however, the respondent didn't have to have family responsibilities or health concerns.
If all the potential stressors were analyzed for all Americans (regardless of whether they were relevant to them or not), the report is correct in stating that the results represent "the most commonly reported sources of stress". If, however, the potential stressors were analyzed for only the Americans they were relevant to (i.e. Work for Americans that are working or Health concerns for Americans with health concerns) then the results represent the things that cause the most stress when experienced. If - as it appears to be the case in the APA report - the source of stress metrics were analyzed for a mix of All respondents and All qualifying respondents, a comparison across the metrics is meaningless.
Image via Wikimedia Commons