What Statistical Analysis Do I Run When Comparing Three Things To Each Other?

A statistical analysis for comparing three or more data sets depends on the type of data collected. Each statistical test has certain assumptions that must be met for the test to work appropriately. Also, what aspects of the data you will compare will affect the test. For example, if each of the three data sets has two or more measurements, you will need a different type of statistical test.

One of the more common statistical tests for three or more data sets is the Analysis of Variance, or ANOVA. To use this test, the data must meet certain criteria. First, the data should be numerical. Ordinal data — such as 5-point scale ratings, called Likert scales — are not numerical data, and the ANOVA will not yield accurate results if used with ordinal data. Second, the data should be normally distributed in a bell curve. If these assumptions are met, the ANOVA test can be used to analyze the variance of a single dependent variable across three or more samples or data sets. Remember, the dependent variable is the factor you are measuring in the study.

In cases where the assumptions for ANOVA are met but you want to measure more than one dependent variable, you will need the Multivariate Analysis of Variance, or MANOVA. The dependent variables are the factors you are measuring and want to examine. The independent variable or variables affect the dependent variable. For example, assume you were measuring the effects of strenuous exercise on blood pressure, weight loss and heart rate. The independent variable is the exercise, and the dependent variables are blood pressure, weight loss and heart rate. In this situation, you would use MANOVA. This statistical test is very complicated to calculate and will require the use of a computer and special software.

There are many different non-parametric tests, but generally non-parametric statistics are used when the data is ordinal and/or not normally distributed. Non-parametric tests include the sign test, chi-square and the median test. These tests are often employed when you are analyzing survey data where the respondents had to rate different statements; for example, a scale of "strongly disagree, disagree, agree, strongly agree" would qualify as ordinal data. These tests are often easy to calculate by hand although a spreadsheet helps.

In addition to inferential tests, you can also use simple descriptive statistics to provide a quick and simple look at the data sets. You can report the average, standard deviations and percentages for each of the three data sets. Descriptive statistics help provide a quick look at the data but cannot be used to draw conclusions. For example, if one of the three data sets has a variable that is 20 percent higher than the other two data sets, you cannot say that the difference is "statistically significant" without using some inferential statistical test, such as ANOVA, MANOVA or a non-parametric test.