Critical thinking about headlines

The following activity is adapted from Jon Mueller's great website Correlation or Causation. There is a copy of the worksheet attached below. I give this to my students in pairs. I ask them to find as many problems as they can with the short research blurbs that are included. They should not make any assumptions, but just work with what is there.

Below you will see a copy of each of the headlines. Below each section you will find some of the variables and concerns that my own students raised with regard to each of them. A good activity. Takes about 20 minutes. You may also choose to start four different classes with one of the problems in order to revisit critical thinking more regularly in the classroom.

Problem 1

Headline: Diet of fish ‘can prevent’ teen violence.

Participants were a group of 3-year-olds given an “enriched diet based on fish, exercise, and cognitive stimulation.” They were compared to a control group who did not go through this same program.

By age 23 they were 64% less likely than a control group of children not on the program to have criminal records.

  • There are three independent variables. We cannot know from this whether the diet has a more direct effect than the other two variables - or whether it has no effect whatsoever. It may also be that the three variables must all be present and need to interact in order to produce the desired behaviour.
  • The word "prevention" is problematic as no cause and effect can be established.
  • The teens had "criminal records." That does not necessarily mean "violence."
  • What type of fish was eaten? And for how long? Was it fish high in Omega-3 fats?
  • How did the researchers control whether or not the other group exercised?
  • How is exercise defined? Cognitive stimulation? How were they measured?
  • There is no control (that we know of) of the family history or environmental factors. We do not know if the three-year-olds were randomly allocated to conditions.

Problem 2

Headline: Higher beer prices cut gonorrhea rates

The research suggests “that raising the price of a six-pack of beer by 20 cents would cut gonorrhea rates by almost 9%.

Researchers considered gonorrhea rates from 2001 to 2007 among teens and young adults in states that raised increased the state beer tax.

Of the 36 beer tax increases that we reviewed, gonorrhea rates declined among teens aged 15 to 19 in 24 instances. Among young adults aged 20 to 24, they declined in 26 instances.

  • There is an assumption here that people who drink beer (versus other alcohol) make poor choices, have unprotected sex and get gonorrhea. That may be a faulty assumption. There is no evidence that there was a higher rate of gonorrhea among beer drinkers prior to the study.
  • There may have been a public health campaign some time between 2001 and 2007 that influenced the sexual behaviour of the teens.
  • There is no comparison made to states where the price of beer did not increase.
  • There is no indication whether this is a steady decline or the average decline over the seven years.
  • This is correlational research, so no cause and effect can be established.
  • It is assumed by the stats that all people who get gonorrhea drink beer.
  • The cost increase is minimal. Do the stats reflect a decrease only in the lowest classes? If not, this may not be a logical conclusion.

Problem 3

Headline: Luckiest people ‘born in summer’

An online public survey in which 40,000 people responded. People who took part in the survey gave their birth-dates and rated the degree to which they saw themselves as lucky or unlucky

Those born in May were most likely to consider themselves lucky; those born in October had most negative views of their life. The poll found there was a summer-winter divide between people born from March to August and those born from September to February. 50% of those born in May considered themselves lucky; 43% of those born in October.

  • What is the definition of "lucky?" Some people would say that if they have good health and a good job, they are "lucky." Others may only see "luck" as winning the lottery. The definition can also be culturally relative.
  • This was an online survey. So, the results may not be representative of the general population. It may, however, be more cross-cultural.
  • We don't know the number of people that responded for each of the months. If only 2 people, for example, had November birthdays, then 1 person saying that they were lucky meant that only 50% felt lucky. There are 40,000 respondents, so that is a good number, but the data may still be highly skewed.
  • The differences in the percentages is rather small and may not be statistically significant.
  • Could the month in which this was asked have affected the results?
  • Peak end rule may play a role here. What if the day they answered the survey they had just recently gotten engaged or a promotion at work? Or what if they had just paid their taxes or found that someone had scratched their car in the supermarket parking lot?

Problem 4

Headline: Fear of hell makes us richer, Feds say.

Researchers studied 35 countries including the United States, Japan, and Turkey. They found that religion shed some “useful light” on how successful a country may be.

In countries where a large percentage of the population believe in hell, there seems to be less corruption and a higher standard of living. For instance, 71% of the U.S. population believe in hell and the country boasts one of the world’s highest per capita income.

  • First and foremost, not every religion believes in hell. Jews, for example, do not have hell as part of their religion. Neither do Buddhists. Or Atheists.
  • There is a question of how the researchers determined whether people "believe in hell." Did they simply count the number of Christians in the US? Did they do an online survey and if so, who answered it? How many people were actually surveyed in order to extrapolate to 71% of the population?
  • There is the problem of generalizing from smaller samples to target populations with surveys in diverse populations like the United States.
  • The study is correlational and cannot establish causation.
  • There is no discussion about the resources available to the country. This could be another important variable.
  • Only 35 countries were reviewed. There are currently 192 countries represented in the United Nations. Were these countries randomly chosen?
  • The study mentions "less corruption" but there is no indication how this is measured.

Critical thinking worksheet

All materials on this website are for the exclusive use of teachers and students at subscribing schools for the period of their subscription. Any unauthorised copying or posting of materials on other websites is an infringement of our copyright and could result in your account being blocked and legal action being taken against you.