《context effects in the general social survey》由会员分享，可在线阅读，更多相关《context effects in the general social survey（15页珍藏版）》请在金锄头文库上搜索。
1、CHAPTER 4 CONTEXT EFFECTS IN THE GENERAL SOCIAL SURVEY Tom W. Smith NORC and University of Chicago 4.1 INTRODUCTION After a half-century of study, question order remains one of the least developed and one of the most problematic aspects of survey research. As Schuman and Presser remark in their work
2、 on survey methodology (1981, p. 77), “Overall, order effects . constitute one of the most important areas of methodological research. They can be very large and are difficult to predict.“ This perplexity is shared by Bradburn (1983, p. 302) who observes, “No topic in questionnaire construction is m
3、ore vexing or resistant to easy generalization than that of question order“ and by Groves (1989, p. 479) who notes, “(T)here seems to be no general theory that predicts when such effects are to be expected and when they should not be expected.“ In part because of our limited ability to predict their
4、 occurrence, there is some disagreement in the literature over how common context effects are. Tourangeau et al. conclude that “The literature on survey context effects may create the impression that such effects are relatively rare, involving items on a few scattered issues. These results here indi
5、cate otherwise . (Tourangeau et al. 1988, pp. 22-23).“ This impression of pervasiveness is supported by numerous instances in which This research was done for the General Social Survey project directed by James A. Davis and Tom W. Smith. It revises and expands upon GSS Methodological Report No. 55.
6、The project is funded by the National Science Foundation Grant SES-8747227. Measurement Errors in Surveys. Edited by Biemer, Grove, Lyberg, Msthiowetz, and Sudman. Copyright 6 1991 by John Wiley It Sons, Inc. ISBN: 0-471-53405-6 58 CONTEXT EFFECTS IN THE GENERAL SOCIAL SURVEY changes in question ord
7、er have upset time series or caused other undesired measurement variations (Smith, 1986; Smith, 1988d; Astin, et al., 1988; Cowan, et al. 1978; Roper, 1990; Johnston and Bachman, 1980; Turner and Martin, 1984; and Gibson, et al. 1978). Schuman and Presser, on the other hand, reach a conclusion that
8、at least differs in emphasis “Question order-effects are evidently not pervasive . but there are enough instances to show they are not rare either (1981, p. 74).“ This non-pervasive impression is supported by numerous failures to produce context effects in experiments designed to do so (Schuman and
9、Presser, 1981; Smith, 1983a; Turner and Martin, 1984) and by the ability of different survey organizations to produce similar marginals when the same questions, but different question content (as well as other variations), existed (Turner and Martin, 1984; Smith, 1978; Smith, 1982a). To date only on
10、e study has conducted a general search for context effects.1 Schuman and Presser (1981) examined the 1971 Detroit Area Study (DAS). The DAS used split ballots in order to accommodate various experiments in either question order or wording. They looked at 113 attitude items that were not the designed
11、 objects of these experi- ments, but appeared after the experiments and thus varied in context due to the prior experiments. Apparently using simple random sampling (SRS) assumptions, they found eight significant differences at the 0.05 level, just two above what chance would predict. Their inspecti
12、on of these eight suggested that three probably represented real effects and the rest were due to sample variation. In this chapter we conduct a similar analysis using the 1988 and 1989 General Social Surveys (GSS).2 While the survey methodologist would usually think of context as creating variation
13、 in the target question, most substantive analysts usually characterize context effects as creating bias or distortion. This latter perspective rests on the notion that there is a true value for the target item free from context (see Introduction and Chapter 1). One way to think about context effect
14、s is to imagine a sample of all possible contexts with the resulting distributions on the target variable ranging from low to high. We presume that something resembling a normal distribution would arise and that one might consider the contexts clustering at the middle of the distribution as neutral.
15、 Substantive analysts would prefer these contexts. 1 Also see Bradburn and Mason, 1964 who tested for 14 differences in marginals across four forms and found no statistically significant variation. 2 The GSSs are full probability samples of adults living in households in the United States. Personal
16、interviewing is used. For complete details see Davis and Smith, 1990. CONTEXT EFFECTS IN THE GENERAL SOCIAL SURVEY 59 Another (and different) way of thinking of a neutral vs. biasing context would be to consider the contexts that duplicated how the issue appeared in the public arena as the “neutral“ context. It would be neutral in the sense that the survey context would duplicate the real world context and not add any me
