4/6/2016

(5)WhatistheintuitionbehindHoeffdingInequality?Quora

MathematicsandMachineLearning

Proofs(mathematics)

RelatedQuestions

+2

WhatistheintuitionbehindHoeffdingInequality?

HowwouldyouexplainHoeffding'sinequality,andas
anaturalconsequence,theVapnikChervonenkis
dimensiontoatenyearold?

Hoeffding'sinequalitycanbeappliedtotheimportantspecialcaseofidenticallydistributed
Bernoullirandomvariables ,andthisishowtheinequalityisoftenusedin
combinatorics andcomputerscience .Weconsiderabiasedcoin thatshowsheads
withprobability

WhataresomecommonapplicationsoftheAzuma
Hoeffdinginequality?
Whatistheintuitionbehindtheconceptof"the
prosecutor'sfallacy"?

andtailswithprobability
...(more)
Answer

Request

Follow 13

WhatisanintuitiveexplanationofChebyshev's
inequality?

Comment Share Downvote

AskQuestion

AskorSearchQuora

3Answers

Hoeffding'sinequalityisaconcentrationinequalitywhichgivesaboundonthetail
probabilityofthesumofindependentrandomvariables.Thebounddecaysexponentially,
whichistighterthanChebyshev'sinequality.Wecanintuitivelyseetheexponentialdecay
fromthecentrallimittheorem.If X , X , . . . arei.i.d.zeromeanrandomvariables,then
CLTsaysthat

X
1

i=1

Notifications

Arvind

WhatistheintuitionbehindPerceptronweightupdate
formulaw=w+yx?

2.3kViewsIvanhas20+answersinProbability(statistics).

Answer

HowtightofaboundisChebyshev'sinequality?

LinearAlgebra:Whatistheintuitionbehindprinciple
axesofamatrix?

IvanLi,probablyknowsprobability

Read

Whatistheintuitionbehinddualformulation?
Whatistheintuitionbehinddistributionswithout
means?
Whatistheconjugatepriorandwhatistheintuition
behindit?

willtendtoanormaldistribution.Foranormaldistribution,thetaildecayslike e ,
whichisinthesameorderasinHoeffding'sinequality.HencewecanviewHoeffding's
inequalityasabounddescribingthenormalliketailprobabilityofthesumofafinite
numberofi.i.d.randomvariables.ComparedtoCLT,whichisanasymptoticresultand
concernsonlytheprobabilityintheformP {
X > n t} ,Hoeffding'sinequality
andotherconcentrationinequalitiescangiveboundsforfiniten,andcanmeasurethe
probabilitythatthesumoftherandomvariablesdeviatesfromanynumber(donotneed
togrowintheordersquarerootn).However,unlikeCLT,theboundgivenbyHoeffding's
inequalityisnottightingeneral(butisenoughifyoujustneedthetailtodecay
exponentially).

Anothertightresultisthelargedeviationbound(http://en.wikipedia.org/wiki/Lar... )
whichconcernstheprobabilityintheformP {
X > nt} .Youmightalsowantto
lookatotherconcentrationinequalitieslikeChernoffbound(Chernoffbound )and
Bernsteininequality(Bernsteininequalities(probabilitytheory) ).

MoreRelatedQuestions

i=1

QuestionStats
13Followers
2,973Views
LastAskedMar19,2013
Edits

i=1

WrittenMar20,2013ViewUpvotesAnswerrequestedbySathishKumar

JustinRising,PhDinStatistics
1kViewsJustinisaMostViewedWriterinProbability(statistics).

Hoeffding'sinequality isaspecialcaseofMcDiarmid'sinequality ,whichisprobablya

littleeasiertounderstand.Inshort,ifyouhaveafunctiononnrandomvariablesthat
doesn'tchangetoomuchifyouresampleoneofthem,thenthefunctionwillbeclosetoits
meanwithhighprobability.
WrittenMar19,2013ViewUpvotes
Upvote

Downvote Comment

KingshukChatterjee,CTO,DataScientistandaPhotographer
890Views

Simplyput,Hoeffdingprovidesanintuitionof"whetherlearningispossible"andatwhat
cost.Sinceitisnotdependentatallonthesampleandpopulationmeasures,ratheronthe
desirederrorthresholdandsamplesize,onecanbalancethe"accuracy"oflearningwith
thesamplesizeneeded.
Itindicates"whetherlearningispossible"byprovidingaprobabilityupperboundwithin
whichyoucanexpectyoursampletovaryfromthepopulationmeasurebythechosen
errorsize.Thesmallerthebound(RHS),thegreaterthepossibilityoflearning.
WrittenOct12,2013ViewUpvotes

https://www.quora.com/WhatistheintuitionbehindHoeffdingInequality

1/2

4/6/2016
Upvote

(5)WhatistheintuitionbehindHoeffdingInequality?Quora
7

Downvote Comment

TopStoriesfromYourFeed
PopularonQuora

PopularonQuora

AnswerwrittenTue

HowisworklifeinGoogleIndia?

Whatwasthefirstthingyoudid
afterabreakup?

Whyisthesourcecodeon
websitessomessy?

MayankBhura,HiredasSoftware
EngineeratGoogle
275.8kViewsMayankisaMostViewed
WriterinGoogleRecruiting.
Upvote

Downvote Comment

PrinceKumar,Relationshipisallabout
believingineachother
322.8kViewsPrinceisaMostViewed
WriterinDatingandRelationships.

JoshMarsh,B.S.inBiology,Admitted
toMedicalSchool,Workingon
CompletingCompSciMS
9.5kViews

AfterFirstbreakup,Ireadalotandgot
9LPAjobinastartup.AfterSecondbreakup,
Ipracticedcodingalotandgot17LPAjob
inanMNC.AfterThirdbreakup,Ilearned
mobileapplicationdeve...

ReadInFeed

LifeatGoogleisinonewordawesome.

https://www.quora.com/WhatistheintuitionbehindHoeffdingInequality

ReadInFeed

ReadInFeed
Inchrome,ifyourightclickandselect"View
PageSource",youwillgetthis:Ifyouselect

2/2