Alan C. answered • 08/08/19
Math Tutor in Central NJ
The definition of 'error' in a statistical context tends to be the difference between an observation and its expected value. The definition of 'residual' is the difference between an observation and its estimated value.
A key way to understand this to note that statistical errors are largely theoretical since they're calculated using parameters, while residuals can be calculated using the relevant statistics. Remember, a parameter is true value of some number applying to a population, for example, population height has a true mean value parameter, ie, the mean height for all people in the population. However, we can never calculate that, we'd have to survey everyone in the population, so we use a statistic, ie, the mean height of a sample of people.
If we take the height of an individual, the difference in their height and the population mean would be the error. The difference in their height and the sample mean would be the residual.
In the context of regression models, there is a true regression model, that gives an output value based on the input value(s). The difference between an observed value and the expected output value from this true regression would be called the error. However, we cannot calculate the true regression, we calculate the sample regression based on our sample data. So the difference between an observed value and the estimated output value from this sample regression is referred to as the residual instead. Hope this explanation helps!
