Research Design
Terms
undefined, object
copy deck
- Define Internal Validity
- Extent to which you can infer a causal relationship between the IV(s) and DV(s)
- Name and define 8 threats to internal validity
-
1. History: any external event that affects DV besides the experimental tx
2. Maturation: any internal (biological,psychological) change in S's during expt that systematically affects DV (e.g. fatigue, boredom, hunger, development)
3. Testing: Experience with pretest changes scores on post-test (e.g. test-wiseness)
4. Instrumentation: Unreliable instruments or systematic changes in measurement (i.e. improvement in rater reliability) affect DV
5. Statistical Regression: Tendency for extreme scores to regress to mean (affects studies of extreme scorers - e.g. gifted, extremely depressed)
6. Selection: Pre-existing subject factors explain changes in DV
7. Differential Mortality (Attrition): Subjects in exptl group drop out at different rate than control
8. Experimenter Bias: Experimenter unconsciously communicates expectations to subjects (e.g. Pygmalian / Rosenthal effect) or errors in scoring / data analysis bias toward hypothesis - Name and Define 6 techniques used to control for threats to internal validity
-
1. Double-Blind: controls for effects of experimenter bias
2. Random Assignment: randomly assign Ss to control / exptl groups so as to obtain equivalency among those groups.
3. Matching: Used to control for effects of specific extraneous variable. E.g. Pair potential Ss on similarity in IQ score and split the pairs between two groups. Useful with small N and Random Assignment (therefore) might not obtain equivalency on the potential confound.
4. Blocking: Turning the potential confound into another IV so you can isolate its effects on DV
5. Holding Extraneous Variable Constant: use only subjects who are similar in terms of the potential confound (e.g. use only high IQ subjects) - trade off with external validity
6. Analysis of Covariance (ANCOVA): statistical technique, analagous to post-hoc matching, to adjust DV scores so that Ss are qualized in terms of status on extraneous variables. Only controls for variables that have been ID'd and measured (like matching) - Define External Validity:
- Extent to which results can be generalized to other times, settings, or people
- Name and describe 6 threats to external validity
-
1. Interaction btwn selection and tx: effects of the tx would not generalize to other members of the population of interest
2. Interxn betw Hx and Tx: The effects of the treatment would not generalize to other places and/or times
3. Interxn betw. testing and tx: When the pretest itself accounts for changes in DV (so results can't be generalized to Ss who don't receive pretest). E.g. "Pretest Sensitization": Ss are oriented to purposes of research study or their susceptibility to tx effects are increased.
4. Demand Characteristics: Cues in research setting allow Ss to guess the research hypothesis (which might lead them to try to confrim and/or disprove it)
5. Hawthorne Effect: tendency for Ss t behave differently due to the fact that they are participating in research (and therefore are being observed).
6. Order Effects (aka Carryover Effects or Multiple Treatment Interference): occurs in repeated measures designs. The effect of one of the tx is due in part to Ss having received a previous tx. Results can't be generalized to setting in wh the client will only receive one tx. - Name and Define 7 strategies for increasing external validity
-
1. Random Selection: all members of target pop have an equal chance of participating in research (ensuring representativeness)
2. Stratified Random Sampling: Random samples are taken from several subgroups of total target pop (to ensure proportionate representation of defined subgroups)
3. Cluster Sampling: Randomly sample naturally occuring groups of individuals (rather than individuals themselves)(e.g. divide city into 32 square blocks, randomly select blocks, and use all people who live in them)
4. Multistage Cluster Sampling: Select smaller and smaller clusters (e.g. randomly select states, then randomly select school disctricts within chosen states, then randomly select schools, then classrooms etc)
5. Naturalistic Research: controls for Hawthorne Effect / Demand Characteristics but sacrifices internal validity (compared to "analogue research")
6. Single / Double-Blind Research: reduce demand charactersitcs / Hawthorne effect
7. Counterbalancing: technique for controlling for order effects. Different groups of Ss receive tx in different orders - Define "Latin Square Design"
- A counterbalanced design in which each tx appears once (and only once) in every position (1st, 2nd, 3rd)
- What are the defining characteristics of true experimental research?
- Random Assignment of Ss into Manipulated levels of IV
- When is it necessary to use quasi-experimental research:
- When IV can be manipulated, but random assignment is impossible (e.g. studying intact groups)
- What is the defining characteristic of correlational research?
- IV's are not manipulated, only measured
- In what way are the findings of correlational research limited?
- No internal validity (looking for association, not causation)
- When is it necessary to use a correlational research design?
- When IV's are not manipulable (e.g. gender, race, SES): aka subject variables, organismic variables, quasi-independent variables.
- What is the purpose of developmental research?
- To assess changes in variables as a fx of time
- Name three types of Developmental Reseasrch
- Longitudinal, Design, Cross-Sectional Design, Corss-Sequential Design
- Describe three drawbacks to longitudinal research:
-
high cost (time and $)
high drop-out
practice effects - Describe the direction of bias in longitudinal research?
- Underestimates true-age related decline (b/c low scorers drop out more, and because of practice effects) High ability Ss with repeated practice over years do not represent general pop
- What is the main problem associated with cross-sectional designs?
- Cohort (intergenerational) effects: differences between age groups are attributable more to the effects of being in a different generation vs. age itself.
- What is the direction of bias in cross-sectional research?
- Overestimates age-effects (by combining age and expereience effects)
- What are the benefits of cross-sequential designs?
-
Control for cohort effects by studying each cohort at different points in time
Less Time Consuming
Therefore Less Drop-out - What is a time-series design?
- Taking multiple measurements over time
- What is an interrupted Time Series Design?
- A time-series design in which measurements are taken before and after the administration of some tx.
- What are the advantages of time-series designs?
- Rules out internal validity threats incl maturation, regression, testing