Experiments

We can all think back to science class in high school. Or even elementary school. Let’s take the classic science experiment. What happens when we mix vinegar and baking soda? We simulate a volcano. The independent variable, mixing two chemicals, creates a reaction, the dependent variable. Social scientists also conduct experiments.

The premise is relatively straightforward: we want to know what the relationship of an independent variable on a dependent variable is. There are a few ways we can go about this, but first a reminder.

In quantitative research, we are making an assumption that the data we collect can be applied to the entire population. This is a bit easier with the volcano experiment: if you take any box of baking soda and add vinegar from any bottle, we’ll get the same reaction. If we’re going to conduct an experiment on people though, does this hold? If were to take 10 of your best friends to conduct an experiment and then take 10 of my best friends to conduct the same experiment, would we get the same results? This is highly doubtful (and violates reliability!). Therefore, we have to make the assumption that in any experiment, we are taking a random sample.

Let’s say we randomly select 100 students at The University of Texas to conduct an experiment: Does eating pizza every day help a person get better sleep at night? Do we have a random sample? Well, this depends on the population. If our population is all the students at The University of Texas, we do. But, if our population is all residents of Austin, the data are not random.

There are two main types of experimental designs. The difference in each design relates to how participants are exposed to the conditions of the experiment (which functions as our independent variable). For the purpose of this discussion, I am going to speak in the simplest of terms: our independent variable will have two levels (conditions): the treatment (received the experimental condition of interest) and the control (did not receive the condition of interest, which could be a placebo).

1. Independent measures. This is also known as a “between groups” design. In this case, a random sample are randomly assigned to one of the two levels. So, about half of the sample will receive the treatment, and the other half will receive the placebo. Because we took a random sample, we can compare the results of the two groups to determine if there is an effect of the treatment.
2. Repeated measures. This is also known as a “within groups” design. In this case, everyone in the random sample receives both treatments. For example, at the first timepoint of the experiment, they receive the placebo. The dependent variable is measured. At the second timepoint, everyone then receives the treatment, and the dependent variable is measured again. Because we took a random sample of the population, and everyone received both treatments, we can still use a simple comparison test to determine the effect.

There are other types of experiments that can be conducted as well. For example, let’s say we know that students of different hair colors have different patterns of sleep. It might be useful to do a matched pairs experiment. This is similar to an independent measures design, but rather than randomly sampling to determine the groups, we match the pairs based on hair color to conduct the experiment.

Let’s take a real life example. Devah Pager wondered what the cost was of having a criminal record on finding employment. Read Chapter 4 of Marked: Race, crime, and finding work in an era of mass incarceration (the link takes you to an e-book, which you can access through the library with you UT eID). Make sure that you spend some time reading and studying Appendix 4A at the end of the chapter. Consider the following the questions.

1. What type of experimental design did Pager use?
2. What were the findings of the chapter?
3. What makes the findings believable?
4. What makes the findings not believable?
5. What would strengthen this study? Are these suggestions realistic?

Note that researchers have to make certain decisions in order to be able to conduct the study. Getting a random sample of a population is not always realistic (or even possible), so we have to do what is possible. Your job, therefore, is to explain your logic to the readers so that they believe your findings.