11 views

Uploaded by Subash Ghimire

- oilcopbv
- Basic Analysis
- Fluid Flow - Glossary of Terms
- I Loathe Thee: A Study on the Perception of Dislike
- 3 Measure g lab 18.docx
- Assortment to Intention
- Cigarettes Case
- uts.doc
- Exam3_key
- 407
- Economics Seasonal Fluctuation
- Final
- Regression
- P7 Behavioral
- Bilet Engl.
- Session - 6 Statistics and Data Analysis
- Multiple Regression
- Rank Spearman Analisis
- Manual.pdf
- Stats - Assignment 3

You are on page 1of 5

From Wikipedia, the free encyclopedia

In statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of an observed value of an element of a statistical sample from its "theoretical value". The error (or disturbance ) of an observed value is the deviation of the observed value from the (unobservable) true function value, while the residual of an observed value is the difference between the observed value and the estimated function value. The distinction is most important in regression analysis, where it leads to the concept of studentized residuals.

Contents

1 Introduction 1.1 Example 2 Regressions 2.1 Stochastic error 3 Other uses of the word "error" in statistics 4 See also 5 References

Introduction

Suppose there is a series of observations from a univariate distribution and we want to estimate the mean of that distribution (the so-called location model). In this case, the errors are the deviations of the observations from the population mean, while the residuals are the deviations of the observations from the sample mean. A statistical error (or disturbance ) is the amount by which an observation differs from its expected value, the latter being based on the whole population from which the statistical unit was chosen randomly. For example, if the mean height in a population of 21-year-old men is 1.75 meters, and one randomly chosen man is 1.80 meters tall, then the "error" is 0.05 meters; if the randomly chosen man is 1.70 meters tall, then the "error" is 0.05 meters. The expected value, being the mean of the entire population, is typically unobservable, and hence the statistical error cannot be observed either. A residual (or fitting error), on the other hand, is an observable estimate of the unobservable statistical error. Consider the previous example with men's heights and suppose we have a random sample of n people. The sample mean could serve as a good estimator of the population mean. Then we have: The difference between the height of each man in the sample and the unobservable population mean is a statistical error, whereas The difference between the height of each man in the sample and the observable sample mean is a residual.

en.wikipedia.org/wiki/Errors_and_residuals_in_statistics

1/5

1/9/14

Note that the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent . The statistical errors on the other hand are independent, and their sum within the random sample is almost surely not zero. One can standardize statistical errors (especially of a normal distribution) in a z-score (or "standard score"), and standardize residuals in a t-statistic, or more generally studentized residuals.

Example

If we assume a normally distributed population with mean and standard deviation , and choose individuals independently, then we have

(As is often done, the "hat" over the letter indicates an observable estimate of an unobservable quantity called .) The sum of squares of the statistical errors , divided by 2, has a chi-squared distribution with n degrees of freedom:

This quantity, however, is not observable. The sum of squares of the residuals , on the other hand, is observable. The quotient of that sum by 2 has a chi-squared distribution with only n 1 degrees of freedom:

It is remarkable that the sum of squares of the residuals and the sample mean can be shown to be independent of each other. That fact and the normal and chi-squared distributions given above form the basis of calculations involving the quotient

en.wikipedia.org/wiki/Errors_and_residuals_in_statistics 2/5

1/9/14

The probability distributions of the numerator and the denominator separately depend on the value of the unobservable population standard deviation , but appears in both the numerator and the denominator and cancels. That is fortunate because it means that even though we do not know , we know the probability distribution of this quotient: it has a Student's t-distribution with n 1 degrees of freedom. We can therefore use this quotient to find a confidence interval for .

Regressions

In regression analysis, the distinction between errors and residuals is subtle and important, and leads to the concept of studentized residuals. Given an unobservable function that relates the independent variable to the dependent variable say, a line the deviations of the dependent variable observations from this function are the unobservable errors. If one runs a regression on some data, then the deviations of the dependent variable observations from the fitted function are the residuals. However, a terminological difference arises in the expression mean squared error (MSE). The mean squared error of a regression is a number computed from the sum of squares of the computed residuals, and not of the unobservable errors. If that sum of squares is divided by n, the number of observations, the result is the mean of the squared residuals. Since this is a biased estimate of the variance of the unobserved errors, the bias is removed by multiplying the mean of the squared residuals by n / df where df is the number of degrees of freedom (n minus the number of parameters being estimated). This latter formula serves as an unbiased estimate of the variance of the unobserved errors, and is called the mean squared error.[1] However, because of the behavior of the process of regression, the distributions of residuals at different data points (of the input variable) may vary even if the errors themselves are identically distributed. Concretely, in a linear regression where the errors are identically distributed, the variability of residuals of inputs in the middle of the domain will be higher than the variability of residuals at the ends of the domain: linear regressions fit endpoints better than the middle. This is also reflected in the influence functions of various data points on the regression coefficients: endpoints have more influence. Thus to compare residuals at different inputs, one needs to adjust the residuals by the expected variability of residuals, which is called studentizing. This is particularly important in the case of detecting outliers: a large residual may be expected in the middle of the domain, but considered an outlier at the end of the domain.

Stochastic error

The stochastic error in a measurement is the error that is random from one measurement to the next. Stochastic errors tend to be gaussian (normal), in their distribution. That's because the stochastic error is most often the sum of many random errors, and when many random errors are added together, the distribution of their sum looks gaussian, as shown by the Central Limit Theorem. A stochastic error is added to a regression equation to introduce all the variation in Y that cannot be explained by the included Xs. It is, in effect, a symbol of our inability to model all the movements of the dependent variable.

en.wikipedia.org/wiki/Errors_and_residuals_in_statistics 3/5

1/9/14

The use of the term "error" as discussed in the sections above is in the sense of a deviation of a value from a hypothetical unobserved value. At least two other uses also occur in statistics, both referring to observable prediction errors: Mean square error or mean squared error (abbreviated MSE) and root mean square error (RMSE) refer to the amount by which the values predicted by an estimator differ from the quantities being estimated (typically outside the sample from which the model was estimated). Sum of squared errors , typically abbreviated SSE or SSe, refers to the residual sum of squares (the sum of squared residuals) of a regression; this is the sum of the squares of the deviations of the actual values from the predicted values, within the sample used for estimation. Likewise, the sum of absolute errors (SAE) refers to the sum of the absolute values of the residuals, which is minimized in the least absolute deviations approach to regression.

See also

Absolute deviation Consensus forecasts Deviation (statistics) Error detection and correction Explained sum of squares Innovation (signal processing) Innovations vector Lack-of-fit sum of squares Margin of error Mean absolute error Propagation of error Regression dilution Root mean square deviation Sampling error Studentized residual Type I and type II errors

References

1. ^ Steel, Robert G. D.; Torrie, James H. (1960). Principles and Procedures of Statistics, with Special Reference to Biological Sciences. McGraw-Hill. p. 288.

Cook, R. Dennis; Weisberg, Sanford (1982). Residuals and Influence in Regression. (http://www.stat.umn.edu/rir/) (Repr. ed.). New York: Chapman and Hall. ISBN 041224280X. Retrieved 23 February 2013. Weisberg, Sanford (1985). Applied Linear Regression (http://books.google.com/books? id=yRrvAAAAMAAJ&dq=editions:UMM1U2yvYVUC) (2nd ed.). New York: Wiley. ISBN 9780471879572. Retrieved 23 February 2013. Hazewinkel, Michiel, ed. (2001), "Errors, theory of" (http://www.encyclopediaofmath.org/index.php?

en.wikipedia.org/wiki/Errors_and_residuals_in_statistics 4/5

1/9/14

title=p/e036240), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4 Retrieved from "http://en.wikipedia.org/w/index.php?title=Errors_and_residuals_in_statistics&oldid=587543285" Categories: Statistical deviation and dispersion Regression analysis Statistical theory Error Measurement Statistical terminology This page was last modified on 24 December 2013 at 18:34. Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

en.wikipedia.org/wiki/Errors_and_residuals_in_statistics

5/5

- oilcopbvUploaded byminhthuc203
- Basic AnalysisUploaded byHarshita Pathak
- Fluid Flow - Glossary of TermsUploaded byAndreea Florea
- I Loathe Thee: A Study on the Perception of DislikeUploaded bymulanako
- 3 Measure g lab 18.docxUploaded byrobv82
- Assortment to IntentionUploaded bymeilisandini
- Cigarettes CaseUploaded bySahil Nayar
- uts.docUploaded byigres manalu
- Exam3_keyUploaded bycesardako
- 407Uploaded byLuminita Vasile
- Economics Seasonal FluctuationUploaded bykamal4sitm
- FinalUploaded byMichael Li
- RegressionUploaded byPradeep Joshi
- P7 BehavioralUploaded byPaul Saquido Capili
- Bilet Engl.Uploaded byAnca Vochescu
- Session - 6 Statistics and Data AnalysisUploaded byKarthik Abhi
- Multiple RegressionUploaded bymthmstr
- Rank Spearman AnalisisUploaded byTabah48 Albiruni
- Manual.pdfUploaded byXimena Sanchez Duque
- Stats - Assignment 3Uploaded byhaonanzhang
- Consumer_evaluation_of_brand_extension_P (1).pdfUploaded byNaba Zehra
- Zephyr k RatioUploaded bympgent
- 8-A Composite Approach(MS 1992)Uploaded byMautim Tinhyeu
- multivariate-lineare-regression.pdfUploaded byaanismu1893
- Investigating the Statistical Linear Relation between the Model Selection Criterion and the Complexities of Data Mining AlgorithmsUploaded byJournal of Computing
- In FriskUploaded bySumuk_Ramashas_1323
- Econometrics Project Iorganda Beatrice Cristina 133 RevisedUploaded byEmi Baka
- final thesis.docxUploaded byMahrukhMalik
- Mathematics Skills as Predictors of Physics Students’ Performance in Senior Secondary SchoolsUploaded byIjsrnet Editorial
- Assignment1Uploaded bySwapnil Parkhe

- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire
- gone_25Uploaded bySubash Ghimire

- HYPOTH. TESTING.pptUploaded byRobert Manea
- About UnderdeterminationUploaded byAnderson
- LU2_Lecture 3 Mann-Whitney Rank Sum TestUploaded bynurul diyana
- Ernest Bloch Lectures - 1999: Lecture 3Uploaded bymaddog
- ELEMENTS OF STATISTICS / TUTORIALOUTLET DOT COMUploaded byalbert0076
- Chaffee, 3e Chapter 6.pptxUploaded byBradford McCall
- Lecture 11 AnovaUploaded byJane Winner
- strategic thinking.pdfUploaded byLeeanji Galamgam
- Inferential StatisticsUploaded byNurul Intan Fairuz
- SEATWORK 10152014Uploaded byspica25
- To p or Not to p - TeseUploaded byVinicius
- James Franklin-What Science Knows_ And How It Knows It-Encounter Books (2009).pdfUploaded byrobert5918
- Cali - Phenomenology of Perception.pdfUploaded byDario Jimenez
- ToK #29Uploaded byEmīls Dālderis
- TOK Knowledge QuestionsUploaded byGanesh Satyabrat
- Denzau North (1994).pdfUploaded byBruno Santos
- Paired T-testUploaded bySoh Mei Ling
- PanelDataNotes-7Uploaded byAnonymous ed8Y8fCxkS
- Grade 8 February 12 - 16 2018Uploaded byRorenzu Kun
- Statistics NotesUploaded byMei Lin Chen
- Ethos Pathos LogosUploaded byHafizul Idham
- Chapter_01_Regression_5E(1).pdfUploaded bykhoold93
- Interval EstimationUploaded byAntariksha Ganguly
- QM1 L5 IVUploaded bySlim Bctt
- Simple Linear RegressionUploaded byApoorva
- Non Parametric TestsUploaded byElaineAriadneLase
- ANOVA Example Crash Test DummiesUploaded byRhapsody Binasbas
- Lab 1 217 With SolutionUploaded byieltstutor
- A Glossary of Literary TermsUploaded byvanka2121446
- 4.HS6251 - Technical English IIUploaded byNeel Jha