What is the purpose of the standard deviation?

Question

I don't have any knowledge of statistics beyond high school common sense. Why is the standard deviation usually seen in combinatorics textbooks, and why is the standard deviation defined intentionally? What is its purpose? Thanks in advance. I have checked in Wikipedia and many other websites to see what this is, but they are not very concise and clear.

Answer 1

The variance is a measure of the dispersion (or variability) of the data around the mean. It's worth mentioning that there are other way to quantify the variation in your data, but I'm only going to explain the standard deviation here.

Its purpose is to give you an idea of how 'good' or precise your mean estimates the average of the variable you're working with. It's also needed to calculate confidence intervals, which also yields very useful information about the quality of the mean estimate.

Consider the following simple example:

Suppose you're interested in what type of apple tree gives the best yield, After a little research, you find a dataset that can help you answer that question. Let's say you've got the yield for 10 trees for each of 10 different types of apples, so you calculate the mean of each of the 10 tree types, and get the following means:

100, 50, 89, 67,100, 55, 65, 88, 78, and 68. With just the means, you would likely decide the two types of trees with the best yield are the 2 that gave 100 apples, but what if you also calculated the std. dev. for each tree type and got the following (in the same order as the means): 20,5,5,10,15,17,22,18. Now you might decide to purchase the 3rd variety instead, since it yield 89 (still one of the highest yields) but with a much lower variability (5) vs. the 20 & 15 variation in the tree type that yields 100 apples per tree.

Upshot: Never make a decision based only on the means, you also need to know the quality of the means. Basically, the lower the variability is around the mean, the higher the probability is that you will actually get a yield close to the mean.

Answer 2

Consider the SAT and the ACT test, or any type of standardized testing.

Suppose your score on the ACT is 25.6.

Is that good, is it bad, is it average? Without the mean/average,

you have no clue how good or bad your score really is

Now, let's suppose the mean average is 24.5.

You can say that your score is above average, but that's it.

Is it an A? Probably not.

Is it a B? Is it a C?

What is the minimum grade you must get in order to score a B or a C?

THOSE are the questions that the standard deviation is used to answer.

The standard deviation allows you to DIVIDE the data into classes and

categories like A,B, C, and D.

So if your score is within 1 standard deviation of the mean, you get a C.

If it is between 1 and 2 standard deviations higher than the mean, you get a B.

If it is1 standard deviation below the mean , you get a D.

The EMPIRICAL RULE states the roughly 2/3 of the data lies within 1 standard deviation

from the mean.