Design and Statistics
Terms
undefined, object
copy deck
- What makes an experiment internally valid?
- Experiments can be termed internally valid ONLY when the obtained effect can be attributed CONFIDENTLY to the independent variable being manipulated by the experimenter. (Neale & Liebert, 1973)
- Is there only one type of Internal Validity?
-
No. Differing are:
1.Experimental Internal Validity
and
2. Theoretical and Conceptual Internal Validity
(Neale & Liebert, 1973) - What does Experimental Internal Validity refer to?
-
- the necessity of showing that it was the particular treatment or manipulation that produced the obttained effects
(Neale & Liebert, 1973) - What does Theoretical & Conceptual Internal Validity refer to?
-
- whether the treatment or manipulation employed produced the observed effect FOR THE REASONS SUGGESTED BY the theoretical rationale of the experimenter, or, another reason.
(Neale & Liebert, 1973) - In it's most basic sense, when does Experimental Internal INVALIDITY result?
-
- when experimental and control groups are treated differently on some dimension other than the independent variable which is of primary interest.
(Neale & Liebert, 1973) - Isn't there ALMOST ALWAYS some dimension - other than the treatment of interest - on which experimental and control groups differ?
-
Yes, Yes, Yes, Yes, Yes, But...
The experimenter's job is to eliminate "plausible rival hypotheses."
(Neale & Liebert, 1973) - Where do "plausible rival hypotheses" come from that an experimenter - considering Experimental Internal Validity - must take into account?
-
Plausible rival hypotheses come from existing knowledge. Our uncertainty is derived from unexcluded hypotheses that the current state of science / established laws/ suggest as rival hypotheses.
(Neale & Liebert, 1973) - What are seven COMMON sources of Experimental Internal INVALIDITY to ALWAYS be on guard against?
-
1. Maturation; 2. Testing; 3. Instrument Decay; 4.Selection;
Statistical Regression; 5. Differential Mortality; 7. History.(Neale & Liebert, 1973)
MNEUMONIC:
M T ID S SR DM H
"Monday. Tuesday. I did. Something. So Romantic. Dad Might. Holler" - What is meant by "Maturation" as a COMMON source of Experimental Internal INVALIDITY?
-
Processes that produce change in the subject over time that are not related to the treatment or other specific events. Any or all systemic changes in organism's biological or psychological condition over time (i.e., may include becoming tired, growing hungry.
Maturation may compete with indep. variable for producing changes - but With adequate control one may discriminate the two hypotheses.
(Neale & Liebert, 1973) - What is meant by "Testing" as a COMMON source of Experimental Internal INVALIDITY?
-
The possible effects of taking a test once upon an individuals's score when administered the test a second time.
(Neale & Liebert, 1973) - What is meant by "Instrument Decay" as a COMMON source of Experimental Internal INVALIDITY?
-
Occurs when a measurement device changes in certain of it's characteristics over time. E.G., in social science research an interviewer may become more skilled over time & collect more complete information. Conversely, systematic change in observers recording behavior may result from desensitization that occurred over time ... or increased sensitization.
(Neale & Liebert, 1973) - What is meant by "Selection" as a COMMON source of Experimental Internal INVALIDITY?
-
Biases that can be introduced when selecting subjects for comparison groups. Common problem when subjects come from already formed groups, i.e., formed for reasons other than experimental study.
(Neale & Liebert, 1973) - What is meant by "Statistical Regression" as a COMMON source of Experimental Internal INVALIDITY?
-
The extreme scores in a particular distribution will move/regress toward the mean as a fucntion of repeated testing. Regression is important source of invalidity when subjects have been chosen for study based upon their initial extreme scores on some variable.
Use of adequate control groups and adequate selection procedures may lessen this problem.
(Neale & Liebert, 1973) - What is meant by "Differential Mortality" as a COMMON source of Experimental Internal INVALIDITY?
-
Differences in the # of S's lost as a function of which group is being considered.
(Neale & Liebert, 1973) - What is meant by "History" as a COMMON source of Experimental Internal INVALIDITY?
-
Environmental events other than the experimental variable which occur between first and second testing. Major historical events may factor.
(Neale & Liebert, 1973) - If critics don't challenge that an experimental manipulation is responsible for an observed change, can they still assert there is "Theoretical and Conceptual Internal INVALIDITY?"
-
YES. Why?
They may argue that the research in question still fails to demonstrate the points it was designed to make.
(Neale & Liebert, 1973) - What are the 2 major reasons it is argued that there is a problem with Theoretical and Conceptual Internal INVALIDITY?
-
1. Most experimental treatments are multidimensional. Each detail that is irrelevant to the investigator's theoretical argument is a source of potential alternate hypothesis for explaining the obtained effects.
2. There is typically an imperfect relationship between the independent variable and the theoretical variable of interest to the investigator. How theoretical variables are translated into experimental manipulations is the crux of the issue.
(Neale & Liebert, 1973) - What is the solution to Theoretical and Conceptual Internal INVALIDITY known as "narrow operationalism?'
-
An 'operational definition' completely specifies the meaning of a term by identifying the operations involved in its' measurement. There is a serious disadvantage to this: A theoretical concept operationally defined with respect to one set of operations is 'isolated', 'specious', and its' value (generality) is entirely lost.
(Neale & Liebert, 1973) - Can 'operational definitions' be distinguished from 'operationism'?
-
YES. HOW?
The tradition of operationism shuns requirement for 1 set of operations to uniquely and exclusively define a theoretical concept. Rather, 'operationism' defines a set of operations is viewed as providing a partial definition of the theoretical term. Sush a term may apply to several different operations (e.g., referents for the term "anxiety" are many and it may be measured variously.
(Neale & Liebert, 1973) - Are there disadvantages to the partial definition?
-
Yes. A 'Partial Definition' specifies meaning under one test condition at a time, thereby leaving open the meaning of the term under other conditions. This appraoch has made the theoretical tem an "open concept."
(Neale & Liebert, 1973) - Are there simple rules and prescriptions to handle the problem of Theoretical and Conceptual Internal INVALIDITY?
-
NO. A careful and thoughtful analysis of alternativce explanations should be undertaken before beginning an experiment.
(Neale & Liebert, 1973) - Can the same manipulation be construned in different ways?
-
Yes, BUT, remember that different interpretations mmust be amenable to separation in experimental designs.
(Neale & Liebert, 1973) - What is "the experimenter bias effect?"
-
The experimental bias effect is a possible general threat to the internal validity of behavioral research. Rosenthal (1966) and others demonstrated that an experimenters expectations regarding outcome of a particular investigation may influence data collected.
(Neale and Liebert, 1973)