One of the key ways in which psychologists study human behaviour is through experiments. Experiments have the advantage that the researcher can control extraneous variables to make sure that they do not influence the behaviour of the animal or person in the study. In this way, a stronger cause and effect relationship can be established since it is assumed that it is the manipulation of the independent variable that leads to the change in the dependent variable.
But is there a cost to this level of control in a laboratory? One of the ways that we can evaluate a study is by examining its level of ecological validity.
Orne (1970) defined ecological validity as "the extent to which the setting in which research takes place is capable of producing results that are valid." Since 1974, the term ecological validity has come to be used by some authors to refer to the degree to which results obtained in the psychological laboratory "predict" circumstances outside the laboratory.
This gets us to the question of how can we determine a study's level of ecological validity? A study has high ecological validity when there is:
- High internal validity - that is, we can establish that the manipulation of the IV does, in fact, lead to changes in the DV and that any changes observed are not the result of extraneous (confounding) variables.
- High external validity - that is, when the results obtained in the study can be applied to situations outside of the laboratory. Notice, this is not the same as "generalizability" to a larger sample; the focus is on similar situations outside of the experimental setting.
When considering the level of ecological validity, researchers must consider the artificiality of the experimental setting. A study high in artificiality has low mundane realism. For example, memorization of a random list of words is low in mundane realism, but retelling a story that you have heard is relatively high in mundane realism. Ecological validity is not the same as mundane realism. Sometimes when a study has low mundane realism, it still accurately predicts what will happen outside of the lab. When the study has high mundane realism, it is more likely to accurately predict what will happen outside of the lab.
Some experiments done under highly controlled conditions have high ecological validity; some naturalistic studies have low ecological validity. We cannot assume that any study has or doesn’t have ecological validity. We must find evidence to support this. That is where critical thinking plays a key role in discussing research.
How does research stack up?
According to Anderson, Lindsay & Bushman (1999), “the psychological laboratory is doing quite well in terms of external validity; it has been discovering truth, not triviality.” (p 8)
Mitchell (2012) wanted to test Anderson’s claims. He looked at 217 comparisons of lab and field experiments from 82 meta-analyses in such areas as industrial, social and developmental psychology. He found a high degree of correspondence between findings observed in the lab and those found in the field, with an overall correlation of r - .73. However, there were major variations in the different areas of psychology.
Correlation of Lab-Field Effects by Research Classification
Psychometrics (personality and intelligence research)
It is also important to note that 30 of the 217 comparisons the results in the field were the opposite of those in the lab. 21 of those studies were in social psychology.
Mitchell argues that to determine ecological validity we need to look at the effect size – that is, the difference between the mean of the control and treatment groups divided by the standard error. If the difference is small and the sample is small, then it has low ecological validity. If the sample is large, then the standard error will be smaller – therefore, it is more likely to predict what will happen “in the real world.”
Thus, having a large sample size or several replications of the same experiment that increases its reliability is important in determining the ecological validity of a study.
Ecological validity in research
Many students believe that they are demonstrating critical thinking by writing that “because this is a laboratory experiment, it lacks ecological validity.” However, this is not always correct. In fact, many of the classic studies that we do in IB psychology are done in labs and have high ecological validity.
If you think about it, if the research we study has no ecological validity, why should we offer an IB psychology course to study it?
Let's take, for example, the classic “line test” by Asch (1951). Here we can see just how tricky the question of ecological validity can be – and why a discussion of it requires critical thinking.
The first question we have to ask is: Is the study highly artificial? At first glance, it is easy to say “yes.” It is not normal to be asked to judge the lengths of lines in front of a group of strangers. But the participant knew that this was a psychological experiment. The study is representative of a psychological experiment.
In addition, Asch has been replicated in many situations and has high reliability. Every replication adds to the sample size and thus there is a greater chance of ecological validity. Finally, when looking at field experiments in which individuals sit in a room where smoke is coming in under the door, when no one responds in a group of confederates, the naive participant also does not respond. It appears that these results can be generalized to other settings.
Another assumption students make is that if it is a field experiment it automatically has high ecological validity. Once again, we have to be careful. Let's take, for example, Piliavin's classic study of bystander behaviour on the New York City subway where he wanted to study what would happen if a well-dressed man with a crutch fell down vs. a man that was holding a bottle of alcohol.
The study definitely has high mundane realism. The event could, in fact, happen. However, a problem with field experiments is that they are more difficult to control than laboratory experiments and therefore also difficult to replicate. This limits the overall sample size. In Piliavin's study, most of the time people helped. However, the participants were also trapped inside a moving metro (subway) car. This does not reflect many studies of helping behaviour where people do not help someone who is in need of help. Although the study may be "naturalistic," it does not automatically have a high level of ecological validity.
The concept of ecological validity is tricky. In many cases, we cannot definitively say that a study is "high" or "low" in ecological validity. You have to make a clear argument for your claim. Doing so demonstrates an understanding of the terminology as well as sound critical thinking.
ATL: Critical thinking
Take a look at the following list of classic studies that are often discussed in IB psychology. For each of the studies, decide whether you think that the study has high or low ecological validity. Remember, the responses that I have given are simply examples of how you could make an argument. There is not really a perfect answer to this question - it all depends on your justification. In order to justify your response, consider the following strategies:
- Discuss the internal validity of the study.
- Discuss how the testing environment itself may have affected the results of the study.
- Discuss the mundane realism of the study.
- Discuss the extent to which the study predicts behaviour in the world outside the laboratory.
- Discuss the reliability of the study with regard to other settings.
1. Loftus & Palmer's study on reconstructive memory in a car crash.
2. Milgram's study of obedience.
3. Maguire et al's taxi cab driver study on plasticity
4. Bandura's Bashing Bobo study
5 The St Helena's study on television and aggression
6. The HM case study