How Do I Calculate Repeatability?

Every researcher who conducts an experiment and gets a particular result has to ask the question: "Can I do that again?" Repeatability is a measure of the likelihood that the answer is yes. To calculate repeatability, you conduct the same experiment multiple times and perform a statistical analysis on the results. Repeatability is related to standard deviation, and some statisticians consider the two equivalent. However, you can go one step further and equate repeatability to the standard deviation of the mean, which you obtain by dividing the standard deviation by the square root of the number of samples in a sample set.

To get reliable results for repeatability, you must be able to perform the same procedure multiple times. Ideally, the same researcher conducts the same procedure using the same materials and measuring instruments under the same environmental conditions and does all the trials in a short period of time. Once all the experiments are over, and the results recorded, the researcher calculates the following statistical quantities:

​Mean:​ The mean is basically the arithmetical average. To find it, you sum all the results and divide by the number of results.

​Standard Deviation:​ To find the standard deviation, you subtract each result from the mean and square the difference to ensure you have only positive numbers. Sum up these squared differences and divide by the number of results minus one, then take the square root of that quotient.

​Standard Deviation of the Mean:​ The standard deviation of the mean is the standard deviation divided by the square root of the number of results.

Whether you take repeatability to be the standard deviation or the standard deviation of the mean, it's true that the smaller the number, the higher the repeatability, and the higher the reliability of the results.

A company wants to market a device that launches bowling balls, claiming the device accurately launches the balls the number of feet selected on the dial. Researchers set the dial to 250 feet and conduct repeated tests, retrieving the ball after every trial, and relaunching it to eliminate variability in weight. They also check wind speed before each trial to ensure it's the same for each launch. The results in feet are:

250, 254, 249, 253, 245, 251, 250, 248.

To analyze the results, they decide to use standard deviation of the mean as a measure of repeatability. They use the following procedure to calculate it:

1. Find the Mean

The mean is the sum of all results divided by the number of results = 250 feet.

2. Calculate the Sum of Squares

To calculate the sum of squares, they subtract each result from the mean, square the difference and add the results:

\((0)^2 + (4)^2 + (-1)^2 + (3)^2 + (-5)^2 + (1)^2 + (0)^2 + (-2)^2 = 56\)

3. Find the Standard Deviation (SD)

They find SD by dividing the sum of squares by the number of trials minus one and taking the square root of the result:

\(\text{SD} = \sqrt{\frac{56}{7}} = 2.83\)

4. Calculate Standard Deviation of the Mean (SDM)

They divide the standard deviation by the square root of the number of trials (n) to find the standard deviation of the mean:

\(\text{SDM} = \frac{\text{SD}}{\sqrt{n}} = \frac{2.83}{2.83} = 1\)

An SD or SDM of 0 is ideal. It means that there are no variations among results. In this case, the SDM is greater than 0. Even though the mean of all the trials is the same as the dial reading, there is variance among the results, and it's up to the company to decide whether the variance is low enough to meet its standards.