《研究设计实验研究的基本问题》由会员分享，可在线阅读，更多相关《研究设计实验研究的基本问题（39页珍藏版）》请在金锄头文库上搜索。
1、School of Public Administration & PolicyDr. Kaifeng Yang研究设计研究设计:实验研究的基本问题实验研究的基本问题问题问题l研究设计的核心要诣研究设计的核心要诣l实验设计的基本过程和环节实验设计的基本过程和环节l实验设计的影响因素实验设计的影响因素Given the research question, what type of evidence is needed to answer the question (or test the theory) in a convincing way?Why?Making decisions to e
2、nsure validitylPlanning: how to collect, analyze, and interpret data in order to test hypotheses1.Who2.Where3.What4.How5.When Can results be generalized (external validity)?Are the relationships true (internal validity)?Time order if necessarySchool typeAcademic Achiv.Childs abilitySchool typeAcadem
3、ic Achiv.Parent resourceHome facilitySchool typeAcademic Achiv.Parent educationvalueChild valuationSchool typeAcademic Achiv.Key: Dealing with rival hypotheses (another example)When you observe X and Y co-vary, the reason may be:When you observe X and Y co-vary, the reason may be:xyz1xyz2xyz3xyz4xyz
4、5xyz6Coping with Rival HypotheseslOperation Fright - Juvenile DelinquencyVisit prisonmonths .70% not arrested1.Even without the program, no more than 30% would have been arrested2.Family background: Volunteered are different3.Committed crimes but not been caught4.Only temporary effects5.Participatio
5、n labels them as potential criminalsThe New Design1.Two groups: similar characteristics relevant to delinquency; one attend the program, another not.2.Interview both groups about delinquent activity and family background before the program3.Interview all families 4.One year after the visit, intervie
6、w both groups about delinquent activity and changes in family5.During step 4, also check arrest and conviction records for both groups6.Two years after the visit, repeat steps 4 and 57.Three years after, repeat 4 and 5.问题问题l研究设计的核心要诣研究设计的核心要诣l实验设计的基本过程和环节实验设计的基本过程和环节l实验设计的影响因素实验设计的影响因素Pretest Stimul
7、usPosttestRandomizationOr matchingComparabilityCompareCompareExp.ControlStaging ExperimentsPretestingSubject recruitmentIntro to experimentManipulation of indep variableMeasurement of dep variableDebriefingAcquisition of informed consentRandom assignmentManipulation checkKey Features of Experimental
8、 Approach1. Manipulation2. ControllTo test hypotheses, the experimenter deliberately introduces changes into the environments of subjects and observes or measures the effects of the changes.Random Assignment vs. Random SamplinglRandom sampling is a method of drawing a sample of cases, such as the po
9、ol of subjects in an experiment.lRandom assignment is a method of assigning subjects from the pool to experimental conditions. 17Experimental DesignlExperimental Design (Randomized Controlled)Group Random. Obser.1 Treat Obser.2 ComparisonExper.Re Oe1 X Oe2 Oe2-Oe1Control Rc Oc1 Oc2 Oc2-Oc1 lTwo grou
10、pslRandomizationlTreatment manipulationlPre and post-tests on the dependent variablelControl of the environment of the experimentPretest StimulusPosttestRandomizationOr matchingComparabilityCompareCompareExp.ControlPre-experimental vs. True Experimental DesignslPre-experimental designs are designs t
11、hat contain one or more features of true experimental design features, such a comparison group or random assignment. lTrue experimental designs are those designs that meet the basic requirements for an experiment. Pre-experimental DesignsDesign 1: The One-Shot Case StudyXODesign 2: The One-Group Pre
12、test-Posttest DesignO1XO2Design 3: The Static-Group ComparisonXO1 O2True Experimental DesignsDesign 4: The Pretest-Posttest Control Group DesignO1X O2RO3O4Design 5: The Posttest Only Control Group DesignX O2RO4True Experimental Designs (2)Design 6: The Solomon Four-Group DesignO1X O2O3 O4RX O5 O6Des
13、igns 4-6 are called between-subjects designs because different groups of subjects are compared with one another.Within subjects design: X1O1X2 O2 Each subject acts as his/her own control that is, O1 is compared with O2 for each subject.Quasi-experimental DesignslQuasi-experimental designs lack some
14、features of a true experimental design, “but permits stronger inferences about cause and effect than pre-experimental designs, by means of special design features and supplementary data.”lThe quasi-experimental design falls into four subdivided categories, including 1) the separate-sample pretest po
15、sttest design, 2) Non-equivalent control group designs, 3) interrupted time-series design, and 4) multiple time-series design. Quasi-experimental Designs1) Separate-sample pretest-posttest designO1X RX O42) Nonequivalent control group designsO1X O2 RO3 O4Quasi-experimental Designs (3)3) Interrupted
16、time-series designO1O2 O3 O4X O5 O6 O7 O84) Multiple time-series designO1O2 O3 O4X O5 O6 O7 O8O1O2 O3 O4 O5 O6 O7 O8Experimentation Outside the LaboratorylField experiments: studies that meet all the requirements of a “true” experiment but are conducted in a natural setting.lExperimental designs in
17、survey researchlUnit of analysis other than individualsEvidence of the versatility of experimentation comes from the incorporation of experimental design methodology into sample surveys and from the use of units of analysis other than individuals.Because of its greater degree of control, experimenta
18、tion is the preferred approach for testing causal hypotheses.问题问题l研究设计的核心要诣研究设计的核心要诣l实验设计的基本过程和环节实验设计的基本过程和环节l实验设计的影响因素实验设计的影响因素Experiment as a Social OccasionlThe particular cues in an experimental situation that communicate to subjects what is expected and what the experimenter hopes to find are c
19、alled demand characteristics.lSubjects in experiments often experience anxiety about being evaluated. Subjects come to experiments expecting the possibility that they will be evaluated. evaluation apprehension.lExperimenters expectations about how the experiment will turn out. experimenters may unin
20、tentionally communicate to subjects their expectations about how the subjects “should” respond so as to confirm their hypotheses.Minimizing Bias Due to the Social Nature of ExperimentationSeveral strategies for handling the problems raised by demand characteristics and experimenter bias1)To ask subj
21、ects about their perception of the experimental situation2)In a cover story, providing the subject with a false hypothesis about the purpose of the experiment.3)To measure the dependent variable in a setting different from the one in which the independent variable is manipulated. 4)Double-blind tech
22、nique: the experimenter does not know which condition a subject is in.5)Reduce the amount of contact between the experimenter and subjects (e.g., using taped recordings)Research Design ValiditylInternal ValidityWhether, in a given study or research project, the independent variable did indeed cause
23、or lead to changes in the dependent variablelExternal Validity: About Generalizing whether and to what extent the conclusions in your study would hold for other persons (or other units of analyses) in other places and at other times. Internal ValidityIVDVExample: Imagine that we are studying the eff
24、ects of a compensatory Imagine that we are studying the effects of a compensatory education program in mathematics for first grade students on a measure of education program in mathematics for first grade students on a measure of math performance such as a standardized math achievement testmath perf
25、ormance such as a standardized math achievement testselectionThreats to Internal ValiditylHistory: events other than the independent Vs (stimulus) occurred and affected the dependentslMaturation: change in subjects over time (e.g., older)lTesting: act of measuring affect dependentslRegression: from
26、extremes (deviations) to averageslMortality: loss of participantslInstrumentation: “measurement” or “process” differslSelection: composition of groups Program (independent Vs)Observation 2(dependent Vs)subjectssubjectsObservation 1WhoWhereWhenWhatHowExternal Validity: RepresentativenessExternal Vali
27、dity: GeneralizabilityNext WeeklXp#s%v)y0B3F6IaLdOgSjVmYq!t&w-z1D4G7JbMePhTkWoZr$u(x+A2E5H9KcNfRiUlXp#s&v)y0C3F6IaLdPgSjVnYq!t*w-A1D4G8JbMeQhTlWoZr%u(x+B2E6H9KcOfRiUmXp#s&v)z0C3F7IaLdPgSkVnYq$t*w-A1D5G8JbNeQhTlWo#r%u(y+B2E6H9LcOfRjUmXp!s&w)z0C4F7IaMdPhSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOgRjUm
28、Yp!s&w)z1C4F7JaMdPhSkWnZq$u*x-A2D5H8KbNfQiTlXo#s%v(y0B3E6I9LdOgRjVmYp!t&w)z1C4G7JaMePhSkWnZr$u*x+A2D5H8KcNfQiUlXo#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1D4G7JbMePhTkWoZr$u(x+A2E5H8KcNfRiUlXp#s%v)y0C3F6IaLdOgSjVnYq!t*w-z1D4G8JbMeQhTkWoZr%u(x+B2E5H9KcOfRiUmXp#s&v)z0C3F7IaLdPgSjVnYq$t*w-A1D4G8JbNeQhTlWoZr%u(y+B2
29、E6H9KcOfRjUmXp!s&v)z0C4F7IaMdPgSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOfRjUmYp!s&w)z0C4F7JaMdPhSkVnZq$u*x-A2D5G8KbNfQiTlXo#r%v(y0B3E6I9LcOgRjVmYp!t&w)z1C4G7JaMePhSkWnZq$u*x+A2D5H8KbNfQiUlXo#s%v(y0B3F6I9LdOgRjVmYq!t&w-z1C4G7JbMePhTkWnZr$u(x+A2E5H8KcNfRiUlXp#s%v)y0B3F6IaLdOgSjVmYq!t*w-z1D4G7JbMeQhT
30、kWoZr$u(x+B2E5H9KcNfRiUmXp#s&v)y0C3F7IaLdPgSjVnY(x+A2E5H8KcNfQiUlXp#s%v)y0B3F6IaLdOgSjVmYq!t*w-z1D4G7JbMeQhTkWoZr$u(x+B2E5H9KcNfRiUmXp#s&v)y0C3F7IaLdPgSjVnYq!t*w-A1D4G8JbMeQhTlWoZr%u(x+B2E6H9KcOfRiUmXp!s&v)z0C3F7IaMdPgSkVnYq$t*x-A1D5G8JbNeQiTlWo#r%u(y+B3E6H9LcOfRjUmXp!s&w)z0C4F7IaMdPhSkVnZq$t*x-A2D5
31、G8KbNeQiTlXo#r%v(y+B3E6I9LcOgRjUmYp!t&w)z1C4F7JaMePhSkWnZq$u*x-A2D5H8KbNfQiTlXo#s%v(y0B3E6I9LdOgRjVmYp!t&w-z1C4G7JaMePhTkWnZr$u*x+A2E5H8KcNfQiUlXp#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1D4G7JbMePhTkWoZr$u(x+A2E5H9KcNfRiUlXp#s&v)y0C3F6IaLdPgSjVnYq!t*w-A1D4G8JbMeQhTlWoZr%u(x+B2E5H9KcOfRiUmXp#s&v)z0C3F7IaLdPgSkV
32、nYq$t*w-A1D5G8JbNeQhTlWo#r%u(y+B2E6H9LcOfRjUmXp!s&w)z0C4F7IaMdPgSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOgRjUmYp!s&w)z1G8JbNeQhTlWo#r%u(y+B2E6H9LcOfRjUmXp!s&v)z0C4F7IaMdPgSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOgRjUmYp!s&w)z1C4F7JaMdPhSkWnZq$u*x-A2D5G8KbNfQiTlXo#r%v(y0B3E6I9LcOgRjVmYp!t&w)z1C4G
33、7JaMePhSkWnZr$u*x+A2D5H8KcNfQiUlXo#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1C4G7JbMePhTkWnZr$u(x+A2E5H8KcNfRiUlXp#s%v)y0C3F6IaLdOgSjVnYq!t*w-z1D4G8JbMeQhTkWoZr%u(x+B2E5H9KcNfRiUmXp#s&v)y0C3F7IaLdPgSjVnYq$t*w-A1D4G8JbNeQhTlWoZr%u(y+B2E6H9KcOfRjUmXp!s&v)z0C4F7IaMdPgSkVnYq$t*x-A1D5G8JbNeQiTlWo#r%u(y+B3E6H9LcOfRjUm
34、Yp!s&w)z0C4F7JaMdPhSkVnZq$u*x-A2D5G8KbNfQiTlXo#r%v(y0B3E6I9LcOgRjUmYp!t&w)z1C4F7JaMePhSkWnZq$u*x+A2D5H8KbNfQiUlXo#s%v(y0B3F6I9LdOgRjVmYq!t&w-z1C4G7JbMePhTkWnZr$u*x+A2E5H8KcNfQiUlXp#sz1C4F7JaMePhSkWnZq$u*x+A2D5H8KbNfQiUlXo#s%v(y0B3F6I9LdOgRjVmYq!t&w-z1C4G7JaMePhTkWnZr$u*x+A2E5H8KcNfQiUlXp#s%v)y0B3F6I
35、aLdOgSjVmYq!t*w-z1D4G7JbMeQhTkWoZr$u(x+B2E5H9KcNfRiUlXp#s&v)y0C3F6IaLdPgSjVnYq!t*w-A1D4G8JbMeQhTlWoZr%u(x+B2E6H9KcOfRiUmXp!s&v)z0C3F7IaMdPgSkVnYq$t*w-A1D5G8JbNeQhTlWo#r%u(y+B2E6H9LcOfRjUmXp!s&w)z0C4F7IaMdPhSkVnZq$t*x-A2D5G8KbNeQiTlXo#r%v(y+B3E6I9LcOgRjUmYp!s&w)z1C4F7JaMdPhSkWnZq$u*x-A2D5H8KbNfQiTlXo
36、#s%v(y0B3E6I9LdOgRjVmYp!t&w-z1C4G7JaMePhTkWnZr$u*x+A2D5H8KcNfQiUlXo#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1D4GbNfQiTlXo#s%v(y0B3E6I9LdOgRjVmYp!t&w-z1C4G7JaMePhSkWnZr$u*x+A2D5H8KcNfQiUlXo#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1D4G7JbMePhTkWoZr$u(x+A2E5H9KcNfRiUlXp#s%v)y0C3F6IaLdOgSjVnYq!t*w-z1D4G8JbMeQhTkWoZr%u(x+B2E5H9
37、KcOfRiUmXp#s&v)z0C3F7IaLdPgSkVnYq$t*w-A1D4G8JbNeQhTlWoZr%u(y+B2E6H9KcOfRjUmXp!s&v)z0C4F7IaMdPgSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOgRjUmYp!s&w)z0C4F7JaMdPhSkVnZq$u*x-A2D5G8KbNfQiTlXo#r%v(y0B3E6I9LcOgRjVmYp!t&w)z1C4G7JaMePhSkWnZr$u*x+A29LcOfRjUmYp!s&w)z0C4F7JaMdPhSkVnZq$u*x-A2D5G8KbNfQiTlXo#r%v
38、(y0B3E6I9LcOgRjVmYp!t&w)z1C4G7JaMePhSkWnZq$u*x+A2D5H8KbNfQiUlXo#s%v(y0B3F6I9LdOgRjVmYq!t&w-z1C4G7JbMePhTkWnZr$u(x+A2E5H8KcNfRiUlXp#s%v)y0B3F6IaLdOgSjVmYq!t*w-z1D4G7JbMeQhTkWoZr$u(x+B2E5H9KcNfRiUmXp#s&v)y0C3F7IaLdPgSjVnYq$t*w-A1D4G8JbMeQhTlWoZr%u(x+B2E6H9KcOfRiUmXp!s&v)z0C3F7IaMdPgSkVnYq$t*x-A1D5G8Jb
39、NeQiTlWo#r%u(y+B3E6H9LcOfRjUmYp!w-A1D4G8JbMeQhTlWoZr%u(x+B2E6H9KcOfRiUmXp!s&v)z0C3F7IaMdPgSkVnYq$t*x-A1D5G8JbNeQiTlWo#r%u(y+B2E6H9LcOfRjUmXp!s&w)z0C4F7IaMdPhSkVnZq$t*x-A2D5G8KbNeQiTlXo#r%v(y+B3E6I9LcOgRjUmYp!t&w)z1C4F7JaMePhSkWnZq$u*x-A2D5H8KbNfQiTlXo#s%v(y0B3E6I9LdOgRjVmYp!t&w-z1C4G7JaMePhTkWnZr$u*
40、x+A2E5H8KcNfQiUlXp#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1D4G7JbMePhTkWoZr$u(x+A2E5H9KcNfRiUlXp#s&v)C4G7JaMePhTkWnZr$u*x+A2E5H8KcNfQiUlXo#s%v)y0B3F6I9LdOgSjVmYq!t&w-z1D4G7JbMePhTkWoZr$u(x+A2E5H9KcNfRiUlXp#s&v)y0C3F6IaLdPgSjVnYq!t*w-z1D4G8JbMeQhTkWoZr%u(x+B2E5H9KcOfRiUmXp#s&v)z0C3F7IaLdPgSkVnYq$t*w-A1D5G8JbNeQ
41、hTlWo#r%u(y+B2E6H9KcOfRjUmXp!s&v)z0C4F7IaMdPgSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOgRjUmYp!s&w)z1C4F7JaMdPhSkWnZq$u*x-A2D5G8KbNfQiTlXo#r%v(y0B3E6IPgSkVnZq$t*x-A1D5G8KbNeQiTlWo#r%v(y+B3E6H9LcOgRjUmYp!s&w)z0C4F7JaMdPhSkVnZq$u*x-A2D5G8KbNfQiTlXo#r%v(y0B3E6I9LcOgRjVmYp!t&w)z1C4G7JaMePhSkWnZr$u*x+A2
42、D5H8KcNfQiUlXo#s%v(y0B3F6I9LdOgRjVmYq!t&w-z1C4G7JbMePhTkWnZr$u(x+A2E5H8KcNfRiUlXp#s%v)y0C3F6IaLdOgSjVnYq!t*w-z1D4G7JbMeQhTkWoZr$u(x+B2E5H9KcNfRiUmXp#s&v)y0C3F7IaLdPgSjVnYq$t*w-A1D4G8JbNeQhTlWoZr%u(y+B2E6H9KcOfRiUmXp!s&v)z0C3F7IaMdPgSkVnYq$t*x-A1D5G8JbNeQiTlWs&v)y0C3F7IaLdPgSjVnYq$t*w-A1D4G8JbNeQhTlWoZr%u(x+B2E6H9KcOfRiUmXp!s&v)z0C3F7IaMdPgSkVnYq$t*x-A1D5G8JbNeQiTlWo#r%u(y+B3E6H9LcOfRjUmYp!s&w)z0C4F7IaMdPhSkVnZq$t*x-A2D5G8KbNeQiTlXo#r%v(y+B3E6I9LcOgRjUmYp!t&w)z1C4F7JaMePhSkWnZq$u*x+A2D5H8KbNfQiUlXo#s%v(y0B3E6I9LdOgRjVmYp!t&w-z
