Call (+1) 650-212-1212

Blog

Standard Deviation or Standard Error – Just Personal Preference?

Sean Salleh 24 Oct 2013 Risk and uncertainty

Figuring out what people mean is important. Are they talking about standard deviation? Or about standard error? Or do they consider the two things to be equivalent? The only way to find out is to check on how the two definitions are being used. That way, when you display a standard deviation or standard error result from a model with Analytica, you’ll have a better chance of your audience understanding the same thing that you wanted to communicate.

Is this the SD of a sample? Or the standard error? Image source: tqn.com

 

Standard Deviation for Starters

Standard deviation (SD) is a statistical trick for measuring the spread of numbers above and below their average point. Some people use spread and dispersion interchangeably here. However, for others, dispersion has yet another meaning, so we’ll stick to spread here. SD is calculated by first squaring the difference with the mean for each number in a dataset of N numbers, giving a whole parallel set of positive-only numbers: negative or positive differences when squared will always end up positive. Next, add all these positive numbers together and divide by N (this gives the variance). Then take the square root and the result is the standard deviation. See this example of a standard deviation calculation for more information. Note that if the N numbers were in fact a sample drawn from a larger dataset, then it turns out you would divide by N-1, rather than N.

My Standard Deviation is Bigger than Yours

That actually means that you’re doing better, as long as we’re talking about the same average. A bigger standard deviation means the average is doing a ‘lower quality’ job in describing the levels of the numbers in the dataset. A lower (‘tighter’) standard deviation indicates the numbers are all grouped that much closer to the average, making it a better representative of those numbers. This explains why just dealing with averages is such a risky way of analyzing data. As Milton Friedman remarked, it’s unwise to try to wade across a river if the only information you have is that its average depth is four feet.

Please Now Welcome Standard Error (Version 1)

Let’s back up a bit. In our standard deviation calculation above, we found the variance, just before taking its square root to find the standard deviation. Hold it right there. Take the variance and this time, first divide (again) by the number of numbers, N. Then, after that, take the square root. Naturally, the standard error is then smaller than the standard deviation. Which one you use between the two is your choice, as long as you’re consistent.

And Here’s Standard Error Again (Version 2)

A civil engineering definition of standard error also applies a systematic distinction between standard deviation and standard error – but a different one. Now standard error is calculated in the same way as standard deviation, except that instead of dividing by the number of numbers (N), you divide by the number of numbers minus 1 (N-1).  While this is different, it’s still simple enough. Rather more complex is the notion (Version 3) that the standard error of the mean is the standard deviation of a sample mean, which is in turn different than the standard deviation of the data as a whole. Aspirin can be provided to those who require it.

If you’d like to know how Analytica, the modeling software from Lumina, can help you to apply many kinds of statistical analysis, then try a thirty day free evaluation of Analytica to see what it can do for you.

» Back

Sean Salleh

Sean Salleh is a data scientist with experience in guiding marketing strategy from building marketing mix models, forecasting models, scenario planning models, and algorithms. He is passionate about consumer technologies and resource management. He has master's degrees in Operations Research from University of California Irvine and Mathematics from Northeastern University.

Leave a Comment