SearchDataScienceCentral Search

SignUp
SignIn

ANALYTICS BIGDATA HADOOP DATAPLUMBING DATAVIZ JOBS WEBINARS MEMBERSHIP SEARCH CONTACT
SubscribetoDSCNewsletter

AllBlogPosts
MyBlog
Add

10DeepLearningTermsExplainedinSimpleEnglish
PostedbyMikeWaldrononApril11,2016at6:30am
ViewBlog

DeepLearningisanewareaofMachineLearningresearchthathasbeengainingsignificantmediainterestowingtotheroleitisplayinginartificialintelligence
applicationslikeimagerecognition,selfdrivingcarsandmostrecentlytheAlphaGovs.LeeSedolmatches.Recently,DeepLearningtechniqueshavebecomepopularin
solvingtraditionalNaturalLanguageProcessingproblemslikeSentimentAnalysis.

ForthoseofyouthatarenewtothetopicofDeepLearning,wehaveputtogetheralistoftencommontermsandconceptsexplainedinsimpleEnglish,whichwill
hopefullymakethemabiteasiertounderstand.

Perceptron
Inthehumanbrain,aneuronisacellthatprocessesandtransmitsinformation.Aperceptroncanbeconsideredasasupersimplifiedversionofabiologicalneuron.

Aperceptronwilltakeseveralinputsandweighthemuptoproduceasingleoutput.Eachinputisweightedaccordingtoitsimportanceintheoutputdecision.

ArtificialNeuralNetworks
ArtificialNeuralNetworks(ANN)aremodelsinfluencedbybiologicalneuralnetworkssuchasthecentralnervoussystemsoflivingcreaturesandmostdistinctly,the
brain.

ANNsareprocessingdevices,suchasalgorithmsorphysicalhardware,andarelooselymodeledonthecerebralcortexofmammals,albeitonaconsiderablysmaller
scale.

Letscallthemasimplifiedcomputationalmodelofthehumanbrain.

Backpropagation
Aneuralnetworklearnsbytraining,usinganalgorithmcalledbackpropagation.Totrainaneuralnetworkitisfirstgivenaninputwhichproducesanoutput.Thefirststep
istoteachtheneuralnetworkwhatthecorrect,orideal,outputshouldhavebeenforthatinput.TheANNcanthentakethisidealoutputandbeginadaptingtheweights
toyieldanenhanced,morepreciseoutput(basedonhowmuchtheycontributedtotheoverallprediction)thenexttimeitreceivesasimilarinput.

Thisprocessisrepeatedmanymanytimesuntilthemarginoferrorbetweentheinputandtheidealoutputisconsideredacceptable.

ConvolutionalNeuralNetworks
Aconvolutionalneuralnetwork(CNN)canbeconsideredasaneuralnetworkthatutilizesnumerousidenticalreplicasofthesameneuron.Thebenefitofthisisthatit
enablesanetworktolearnaneurononceanduseitinnumerousplaces,simplifyingthemodellearningprocessandthusreducingerror.ThishasmadeCNNs
particularlyusefulintheareaofobjectrecognitionandimagetagging.

CNNslearnmoreandmoreabstractrepresentationsoftheinputwitheachconvolution.Inthecaseofobjectrecognition,aCNNmightstartwithrawpixeldata,then
learnhighlydiscriminativefeaturessuchasedges,followedbybasicshapes,complexshapes,patternsandtextures.
 source:http://stats.stackexchange.com/questions/146413

RecurrentNeuralNetwork
RecurrentNeuralNetworks(RNN)makeuseofsequentialinformation.Unliketraditionalneuralnetworks,whereitisassumedthatallinputsandoutputsareindependent
ofoneanother,RNNsarereliantonprecedingcomputationsandwhathaspreviouslybeencalculated.RNNscanbeconceptualizedasaneuralnetworkunrolledover
time.Whereyouwouldhavedifferentlayersinaregularneuralnetwork,youapplythesamelayertotheinputateachtimestepinanRNN,usingtheoutput,i.e.thestate
oftheprevioustimestepasinput.ConnectionsbetweenentitiesinaRNNformadirectedcycle,creatingasortofinternalmemory,thathelpsthemodelleveragelong
chainsofdependencies.

RecursiveNeuralNetwork
ARecursiveNeuralNetworkisageneralizationofaRecurrentNeuralNetworkandisgeneratedbyapplyingafixedandconsistentsetofweightsrepetitively,or
recursively,overthestructure.RecursiveNeuralNetworkstaketheformofatree,whileRecurrentisachain.RecursiveNeuralNetshavebeenutilizedinNatural
LanguageProcessingfortaskssuchasSentimentAnalysis.

source:http://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf

SupervisedNeuralNetwork
Forasupervisedneuralnetworktoproduceanidealoutput,itmusthavebeenpreviouslygiventhisoutput.Itistrainedonapredefineddatasetandbasedonthis
dataset,canproduceaccurateoutputsdependingontheinputithasreceived.Youcouldthereforesaythatithasbeensupervisedinitslearning,havingforexample
beengivenboththequestionandtheidealanswer.

UnsupervisedNeuralNetwork
Thisinvolvesprovidingaprogrammeormachinewithanunlabeleddatasetthatithasnotbeenpreviouslytrainedfor,withthegoalofautomaticallydiscoveringpatterns
andtrendsthroughclustering.

GradientDescent
GradientDescentisanalgorithmusedtofindthelocalminimumofafunction.Byinitiallyguessingthesolutionandusingthefunctiongradientatthatpoint,weguidethe
solutioninthenegativedirectionofthegradientandrepeatthistechniqueuntilthealgorithmeventuallyconvergesatthepointwherethegradientiszerolocalminimum.
Weessentiallydescendtheerrorsurfaceuntilwearriveatavalley.

WordEmbedding
Similartothewayapaintingmightbearepresentationofaperson,awordembeddingisarepresentationofaword,usingrealvaluednumbers.Wordembeddingscanbe
trainedandusedtoderivesimilaritiesbetweenbothotherwords,andotherrelations.Theyareanarrangementofnumbersrepresentingthesemanticandsyntactic
informationofwordsinaformatthatcomputerscanunderstand.

Wordvectorscreatedthroughthisprocessmanifestinterestingcharacteristicsthatalmostlookandsoundlikemagicatfirst.Forinstance,ifwesubtractthevectorof
ManfromthevectorofKing,theresultwillbealmostequaltothevectorresultingfromsubtractingWomanfromQueen.Evenmoresurprisingly,theresultofsubtracting
RunfromRunningalmostequatestothatofSeeingminusSee.Theseexamplesshowthatthemodelhasnotonlylearntthemeaningandthesemanticsofthesewords,
butalsothesyntaxandthegrammartosomedegree.

Sothereyouhaveitsomeprettytechnicaldeeplearningtermsexplainedinsimpleenglish.Wehopethishelpsyougetyourheadaroundsomeofthetrickytermsyou
mightcomeacrossasyoubegintoexploredeeplearning.

ThisblogwasoriginallypublishedontheAYLIENTextAnalysisblog.

Views:20904

Like
13memberslikethis

Tweet
Share Facebook

Like 325

<PreviousPost
NextPost>

Comment

YouneedtobeamemberofDataScienceCentraltoaddcomments!
JoinDataScienceCentral

Welcometo
DataScienceCentral

SignUp
orSignIn

Orsigninwith:
FOLLOWUS

@DataScienceCtrl|RSSFeeds

TOPCONTENT

1
ThereAreLies,DamnLies,AndJournalism

2
DifferencebetweenMachineLearning,DataScience,AI,DeepLearning,andStatistics

3
25BigDataTermsYouMustKnowToImpressYourDate(Orwhoeveryouwantto)

4
DetectingPlagiarismwithaSimpleTrick

5
140MachineLearningFormulas

6
MachineLearningSummarizedinOnePicture

RSS
ViewAll

ANNOUNCEMENTS

GetaGuaranteedDataScienceJob

10reasonstochooseVillanova'sonlineMSA

ImproveYourRegressionwithCARTandGradientBoosting

Top10BigDataTrendsfor2017

HowCompaniesPutDataScienceinProductionSurveyResults

Whatsyournextbigcareerstep?

AnalyticsisSmartBusinessMSAfromVillanovaUniversity

BigDataTrendsfor2017

NYUStern:DatawithInsight

MarketingAnalyticsandDataScience2017

VIDEOS

ANaturalLanguageProcessing(NLP)ApproachtoDataExploration
AddedbyTimMatteson0Comments0Likes
 OptimalDataAnalyticsArchitecture
AddedbyTimMatteson0Comments2Likes

AddVideos
ViewAll

RESOURCES

MigratinganExcelSpreadsheettoMySQLandtoSpark2.0.1(Part1)
IntroductiontoProgramminginStata
collaborativerecommendationengineimplementationinpython
Benchmarking20MachineLearningModelsAccuracyandSpeed
StataCheatSheet
Selectionofbestarticlesfromourpastweeklydigests
StatisticalAnalysisAdvisorChart
Selectionofbestarticlesfromourpastweeklydigests
FreeOnlineBook:Forecasting,PrinciplesandPractice
38SeminalArticlesEveryDataScientistShouldRead

TOPCATEGORIES

MachineLearning

RProgramming

PythonforDataScience

Visualization,Dashboards

NoSQLandNewSQL

BigData

CheatSheets

InternetofThings

Excel

2017DataScienceCentralPoweredby

Badges|ReportanIssue|PrivacyPolicy|TermsofService