Quantitative methods are used in the social sciences to be able to generalize an idea on a population. It’s pretty complicated to get data on every single person out there, but there is a whole science of collecting data on a sample of people in order to make generalizations on the population. Before we continue, there are a few points that we need to address.
Thinking to the philosophy of science, employing a statistical analysis makes a (pretty big) assumption that we can generalize the entire population on a sample of people. What are the implications of this?
Does this mean statistics aren’t useful? Of course they are. We use statistics on a daily basis in order to make decisions. And if we do it personally, organizations use them for decision making. However, part of the use of statistics is to be a responsible user of them. You must understand all of the assumptions to employ a specific analysis. When collecting data, you need to make sure that your measures are valid and reliable.
In quantitative research, validity and reliability are arguably two of the most important considerations.
Validity: This refers to the notion that you are testing what you intend to test. For example, if you’re trying to understand how much someone likes ice cream, it would not be a valid measure to ask someone how often they eat ice cream. Maybe they like it very much but are on a diet.
Reliability: This refers to how consistent your measure is. If you were you perform a test several times, we should expect to get the same outcome. However, asking a group of people how much they like ice cream the day after the Blue Bell recall versus two years later when the research question does not refer to listeria would not be a reliable measure.
There are a lot of different types of reliability and validity that a researcher needs to know to ensure his or her measures can be generalizable. For more information, this is a good resource.
Knowing that we need valid and reliable measures, what types of data can we collect?
Generally, there are three types of data that social scientists used to conduct quantitative research: experiments, surveys, and secondary data.
Experiments are just like what you think of in science class. These are most used in psychological-based studies. In the most classic sense, you want to know if an intervention works (for example, a diversity training). You randomly assign people in your study to one of two groups: the intervention (diversity training) or the control (often what had been done before, maybe a handbook on respect). At the end of the experiment, you assess your outcome of interest (level of respect for people of diverse backgrounds), and then use statistics to compare these two groups. If there is significance, then we can say that the intervention works. But again, we have to remember our assumptions that we discussed before.
Surveys are constantly used to assess people’s opinions. How often do you get an email asking to fill out a survey? Survey data are also reported constantly, such as on the news. (Presidential approval ratings, for example). Survey construction is much more challenging than it seems because we have to be particularly sure that our wording of the questions provides us with valid responses. More on this later.
A third type of data that social scientists use in statistical analysis is secondary data. This is exactly what it sounds like: these are data that were collected for another purpose that are then used to answer a research question. This type of research is common in sociological studies. For example, we want to know if there is a difference in high school drop-out rates and how violent a community is. To conduct this analysis, a researcher might collect the high school data from the education agency and the crime data from the police department. These data can then be used to conduct statistical analysis to answer the research question.
To overview quantitative methods, we need to spend some time thinking about the types of data we collect. And, once we have these data, what do we do with them? This is where statistics comes into play.
This section is divided into three models. The first discusses experiments (we can also use experiments to collect quasi-experimental data, which we will discuss there too. The second discusses survey design. Finally, we will discuss basic statistical principles. Please note that all of these topics are meant to provide you with an overview so that you can understand the most important things to consider when designing and conducting quantitative research. You should also be familiar with these so that you can be practical consumers of this research.