This section of the report is where you describe how your experiment was designed and carried out. You are asked to demonstrate your understanding of the experiment as a quantitative methodology.
In this section you will be expected to:
- Explain the experimental design
- Explain the sampling technique used as well as characteristics of the participants.
- Explain how you controlled variables in your experiment.
- Explain the choice of materials, i.e. how they were designed or selected.
- Outline the procedure that was followed.
Research design and control of variables
Depending on what you investigate, you can choose between two basic designs: an independent samples design or a repeated measures design.
An independent samples design makes use of two different groups of participants: one group is given the experimental treatment; the other group receives no treatment. For example, one group is given a list of words to memorize and instructed in using imagery; the other group is simply given a list of words to memorize. The group in which the IV is manipulated is called the treatment group. The group that gets no treatment is called the control group. The control group allows us to see how the list is memorized under normal circumstances. Many experiments which you may wish to replicate will have two different treatment groups, where one group gets one treatment (e.g. memorizing a list of words using imagery), and the other group gets a different treatment (e.g. memorizing a list of words using rehearsal). In such an experiment, you may compare which treatment results in the highest recall or you may modify the experiment so that you have one treatment condition and one control – that is, memorization with no manipulation of an IV.
The independent samples design is used when it is not possible to use the same participants in the two conditions. For example, when asking the same participants to remember the same list of words under two different conditions (first imagery and then rehearsal), they would learn what to remember in the first trial and this would affect the second trial. This is called the practice effect.
A strength of an independent samples design is that the participants are less likely to guess the hypothesis of the study. The participants are also less likely to be bored or tired during the test - an example of order effects. A final strength is that the same materials may be used with both groups. For example, both groups could be given the same list of words to memorize under different conditions. This eliminates the possibility of extraneous variables that may result from using two different word lists in your experiment.
The limitation of using an independent samples design is that there may be participant variability—that is, the participants may differ so much (e.g. in memory ability) that the differences between the two groups may actually be due to this and not simply to the manipulation of the IV. This can be controlled by random allocation of participants to the experimental conditions; for example, by having the names of all participants in a box and then drawing their names one by one while you allocate them to conditions. The main thing here is that all participants have an equal chance of being selected to the conditions and this is considered to be a control for participant variability.
Because of the limitations of the independent samples design, researchers may use a repeated measures design. This design involves using the same participants in both the treatment and control groups. For example, in an experiment testing the Stroop effect , the same participants participate in both conditions to avoid participant variables. In the first condition, participants are asked to read a list of colour words that are written in a different colour and name the colour of the word (incongruent condition) and the time it takes them to name the colour is measured. The same participants are then are asked to read the same list of colour words now written in the same colour (the congruent condition). The time it takes them is measured again and compared to the timing of the first condition. One strength of this design is that it eliminates the problems of participant variability. Another strength is that it requires fewer participants. Ten participants would be enough for your IA.
However, a repeated measures design also has its limitations. First, doing the same task twice may result in order effects – that is, the fact that one treatment occurred before the other treatment may affect the results of the second condition. Another potential problem is demand characteristics, when participants guess the aim of the study and do not act naturally because they want to be helpful - or the opposite. Lastly, a repeated measures design does not work for all types of experiments. It is best for experiments where the researcher wants to see how an IV may change participants’ performance on a specific task—for example, in the Stroop test.
In order to reduce order effects Counter-balancing is an important control measure in an experiment with a repeated measures design. This means that with a group of 10 participants, half of your participants would first do condition A and then B (A-B). The other half of the participants would do the opposite starting with condition B and then they would do condition A (B-A). This control measure is assumed to control for order effects.
Controls are an important part of any procedure in psychology. By using standardized instructions, we attempt to make sure that the way in which the experiment was carried out does not lead to extraneous variables such as demand characteristics or order effects. Other controls include carrying out pilot tests of your materials, conducting the experimental conditions in the same place and at the same time of day; or standardizing the way in which a variable is measured. Control of the physical environment – for example, room temperature – is often irrelevant to the experiment. However, ensuring that your experiment is run in a quiet environment with a limited amount of interference is important. So, you should not carry out your experiment in the canteen during lunch time!
When writing about the design in the Exploration section of the report, remember to:
- Identify the design you have used—either independent samples or repeated measures;
- Explain why you chose the design that you did and describe the controls you have undertaken to avoid extraneous variables (e.g. by giving a standardized briefing to the participants).
- Identify the independent and dependent variables.
Sampling technique and characteristics of participants
The next section of the exploration section of your report should focus on the characteristics of your participants and your sampling technique, i.e. the nature of your sample and how it was obtained. You should have a least 20 pieces of data for your experiment – so, for an independent samples design that means 20 participants; for a repeated measures design, only 10.
The choice of participants is important in an experiment. First, you need to identify which characteristics your participants should have (for example, not colour-blind, having driving experience or English proficiency) in order to test the hypothesis. It is convention to always describe the sample in terms of the number of participants, age, and gender distribution.
You should also explain the sampling technique used. In a typical IA experiment the sampling technique is often a so-called opportunity sample. This means a sample that is chosen based on availability, that is, whoever happens to be present and willing to participate can do so. If specific characteristics are needed in your experiment, you can also advertise for participants. Then you would have a self-selected sample. The sampling technique must be explained. You must describe how the sampling was actually carried out and explain why this sampling technique was chosen. In addition, you should describe the manner in which participants were allocated to the experimental group and control conditions.
Example of description of sampling technique
The sampling technique used was opportunity sampling because this was the easiest and most convenient. The 20 participants were taken from two second year Economics classes.. The sample consisted of 10 boys and 10 girls (mean age 17). Because of the nature of the experiment, it was important that participants were fluent in English and had some experience with driving a car so only participants with these characteristics were selected. The participants were randomly allocated to one of the two conditions by the flip of a coin.
Procedure and materials
Because you are replicating a study – and it is important for every member of a psychological research team to use a standardized procedure – an outline of the procedure must be included in the report, but it is not formally assessed. The procedure includes explanation of the materials used in the experiment, that is, what was used, why and how it was used. There must be a copy of all materials in the appendices. Summing up, this means that any written materials (e.g. list of words to recall, briefing and debriefing notes) that were specially developed for the experiment should be referenced to a sample copy included in the appendices.
You must describe how the experiment was carried out,. Enough detail should be provided so that another researcher could replicate the experiment. The outline of procedure must also include reference to any ethical issues that were addressed—for example, how consent was obtained and how debriefing was carried out.
Before you write out your procedure, you should have an explanation of the materials that were used. This should not simply be a list. For example, if you are using a list of words, how were these words chosen? What was the rationale for this list? Did you carry out a pilot study to test that these words were familiar to your population? Basic materials, such as tables, chairs, paper, and pencils should not be listed.
Ethical considerations are not formally assessed; however, failure to provide evidence that all ethical guidelines were met may end up with your internal assessment not being assessed. A copy of the letter of informed consent should be included in the appendices. Make sure that the consent form is written in a way that informs the participants of the nature of the experiment. Remember that if your participants are younger than 16-years-old, evidence of parental consent is also required.
Standardized briefing notes are a copy of the script that you used in the briefing of your participants. These include the aim and instructions regarding the procedure of the study, as well as information about ethical issues. Include the standardized briefing notes in the appendices. By using standardized briefing, you ensure that you control extraneous variables that could interfere with the experiment.
Standardized debriefing notes must also be included in the appendices. This is a copy of the script that you used in the debriefing of your participants when the experimental procedure was over. The notes should state what you expect to find in your study, promise to share full conclusions when they are available, ask if the participants have any questions or concerns about the study and remind them that they have the right to withdraw their data, in keeping with ethical standards.
This experiment used an independent samples design. The independent variable is the size of the anchor value – in this case, the anchor was either “1” or “8”; the dependent variable is the answer to the problem, which is an estimated value of 8! As in both conditions we wanted the participants to estimate the product of the same numbers, it was not possible to use a repeated measures design. A sample of opportunity was used, guaranteeing that the sample was easily collected. The sample was made up of two classes of grade-nine mathematics. Both classes were in the advanced level with the hope to eliminate math anxiety as a confounding variable. The use of the sample of opportunity minimized the level of participant variability as all students would have had a similar level of mathematics skill. In the descending condition there were 18 participants (11 females and 7 males); in the ascending condition there were 16 participants (9 males and 7 females).
We created three slides for each condition. The first two slides were the same. The first slide read “52 * 52.” The second slide read “39 * 19.” To make sure that these would require estimation and could not be quickly solved, we carried out a pilot study with our classroom. By using these two slides as practice estimations, we are hoping to control for language difficulties in understanding the directions. The final slide was different for each group – with either the ascending or descending condition.
We flipped a coin to randomly allocate the participants to either the ascending or the descending condition. In each condition, participants were read the standardised briefing (see Appendix ii). Participants were given an “answer sheet” on which they wrote their estimates for all three questions. When the slides were presented, the participants had only fifteen seconds to write their response. There were two boxes on one side of the paper to write their estimations for the first questions and then a box on the back of the paper for the final question. This was done as a control to make sure that any numbers that they had written on the paper would not influence their estimation which was the dependent variable.
After the final slide, the answer sheets were collected and the participants were debriefed (see appendix iii). When results were available, the researchers returned to the classes and explained their findings
Comments on the sample
Paragraph 1: The design is identified and then explained. The sampling method is identified and then described. Then the choice of the sampling method is explained.
Paragraph 2: This paragraph describes the materials and explains how they were developed.
Paragraphs 3 & 4: This paragraph explains how participants were allocated to conditions. Controls for the study are also explained. The procedure is outlined, including how ethical standards were met.
How you are assessed
The following table is the assessment rubric used to award marks for your exploration.
|1 - 2||
|3 - 4||