Professional Documents
Culture Documents
Search Latest Articles Latest Tips/Tricks Top Articles Beginner Articles Video Articles Technical Blogs Post an Article or Tip Post your Blog Posting/Update Guidelines Article Competition Quick Answers o Ask a Question about this article o Ask a Question o View Unanswered Questions o View All Questions... o C# questions o ASP.NET questions o VB.NET questions o C++ questions o C#4.0 questions Discussions o All Message Boards... o Application Lifecycle>
o o o o o o o o o o o
o o
Design and Architecture Running a Business Sales / Marketing Collaboration / Beta Testing Work & Training Issues Free Tools C / C++ / MFC>
o o
C# Database
o o o o o o o o o o
System Admin Hosting and Servers Java .NET 4.5 and Visual Studio 11 .NET Framework Mobile Sharepoint Silverlight / WPF Visual Basic Web Development>
ASP.NET CSS JavaScript PHP Site Bugs / Suggestions
o o
Other Languages>
General Indian Topics General Chinese Topics Learning Zones o The Commerce Zone o The Mobile & App Zone o The Cloud Zone o The Hardware Zone o The Parallelism Zone o The WPF / Silverlight Zone o The Flex / Flash Zone o The HTML5 / CSS3 Zone o SharePoint Zone o The SQL Zone o WhitePapers / Webcasts o Solutions Center Features o Who's Who o Most Valuable Professionals o Company Listings o Component & Service Catalog o Competitions o News
o o o o o o o
Help!
o o o o o
What is 'The Code Project'? General FAQ Post a Question Bugs and Suggestions Site Directory
o o o o o
Advertise with us About Us The Lounge The Insider News The Lounge Clever Code Hall of Shame
o o
The Soapbox
149
Languages C# General
This article contains a brief description of BackPropagation Artificial Neural Network and its implementation for Image Recognition See Also
Article
More like this More by this author
Browse Code
Stats
Revisions
Is your email address OK? You are signed up for our newsletters but your email address is either unconfirmed, or has not been reconfirmed in a long time. Please click here to have a confirmation email sent so we can confirm your email address and start sending you newsletters again. Alternatively, you can update your subscriptions. Download source -286.16 KB Download demo project -257.52 KB
Introduction
Artificial Neural Networks are a recent development tool that are modeled from biological neural networks. The powerful side of this new tool is its ability to solve problems that are very hard to be solved by traditional computing methods (e.g. by algorithms). This work briefly explains Artificial Neural Networks and their applications, describing how to implement a simple ANN for image recognition.
Background
I will try to make the idea clear to the reader who is just interested in the topic.
ANNs have the ability to adapt, learn, generalize, cluster or organize data. There are many structures of ANNs including, Percepton, Adaline, Madaline, Kohonen, BackPropagation and many others. Probably, BackPropagation ANN is the most commonly used, as it is very simple to implement and effective. In this work, we will deal with BackPropagation ANNs. BackPropagation ANNs contain one or more layers each of which are linked to the next layer. The first layer is called the "input layer" which meets the initial input (e.g. pixels from a letter) and so does the last one "output layer" which usually holds the input's identifier (e.g. name of the input letter). The layers between input and output layers are called "hidden layer(s)" which only propagate the previous layer's outputs to the next layer and [back] propagates the following layer's error to the previous layer. Actually, these are the main operations of training a BackPropagation ANN which follows a few steps. A typical BackPropagation ANN is as depicted below. The black nodes (on the extreme left) are the initial inputs. Training such a network involves two phases. In the first phase, the inputs are propagated forward to compute the outputs for each output node. Then, each of these outputs are subtracted from its desired output, causing an error [an error for each output node]. In the second phase, each of these output errors is passed backward and the weights are fixed. These two phases is continued until the sum of [square of output errors] reaches an acceptable value.
Implementation
The network layers in the figure above are implemented as arrays of structs. The nodes of the layers are implemented as follows:
Collapse | Copy Code
public double[] Weights; }; [Serializable] struct Input { public double InputSum; public double Output; public double Error; public double[] Weights; }; [Serializable] struct Hidden { public double InputSum; public double Output; public double Error; public double[] Weights; }; [Serializable] struct Output<T> where T : IComparable<T> { public double InputSum; public double output; public double Error; public double Target; public T Value; };
The layers in the figure are implemented as follows (for a three layer network):
Collapse | Copy Code
Training the network can be summarized as follows: Apply input to the network. Calculate the output. Compare the resulting output with the desired output for the given input. This is called the error. Modify the weights for all neurons using the error. Repeat the process until the error reaches an acceptable value (e.g. error < 1%), which means that the NN was trained successfully, or if we reach a maximum count of iterations, which means that the NN training was not successful.
void TrainNetwork(TrainingSet,MaxError) {
public bool Train() { double currentError = 0; int currentIteration = 0; NeuralEventArgs Args = new NeuralEventArgs() ; do { currentError = 0; foreach (KeyValuePair<T, double[]> p in TrainingSet) { NeuralNet.ForwardPropagate(p.Value, p.Key); NeuralNet.BackPropagate(); currentError += NeuralNet.GetError(); } currentIteration++; if (IterationChanged != null && currentIteration % 5 == 0) { Args.CurrentError = currentError; Args.CurrentIteration = currentIteration; IterationChanged(this, Args); } } while (currentError > maximumError && currentIteration < maximumIteration && !Args.Stop); if (IterationChanged != null) { Args.CurrentError = currentError; Args.CurrentIteration = currentIteration; IterationChanged(this, Args); } if (currentIteration >= maximumIteration || Args.Stop) return false;//Training Not Successful return true; }
Where ForwardPropagate(..) and BackPropagate() methods are as shown for a three layer network:
Collapse | Copy Code
private void ForwardPropagate(double[] pattern, T output) { int i, j; double total; //Apply input to the network for (i = 0; i < PreInputNum; i++) { PreInputLayer[i].Value = pattern[i]; } //Calculate The First(Input) Layer's Inputs and Outputs for (i = 0; i < InputNum; i++) { total = 0.0; for (j = 0; j < PreInputNum; j++) { total += PreInputLayer[j].Value * PreInputLayer[j].Weights[i]; } InputLayer[i].InputSum = total; InputLayer[i].Output = F(total); } //Calculate The Second(Hidden) Layer's Inputs and Outputs for (i = 0; i < HiddenNum; i++) { total = 0.0; for (j = 0; j < InputNum; j++) { total += InputLayer[j].Output * InputLayer[j].Weights[i]; } HiddenLayer[i].InputSum = total; HiddenLayer[i].Output = F(total); } //Calculate The Third(Output) Layer's Inputs, Outputs, Targets and Errors for (i = 0; i < OutputNum; i++) { total = 0.0; for (j = 0; j < HiddenNum; j++) { total += HiddenLayer[j].Output * HiddenLayer[j].Weights[i]; } OutputLayer[i].InputSum = total; OutputLayer[i].output = F(total); OutputLayer[i].Target = OutputLayer[i].Value.CompareTo(output) == 0 ? 1.0 : 0.0; OutputLayer[i].Error = (OutputLayer[i].Target - OutputLayer[i].output) * (OutputLayer[i].output) * (1 OutputLayer[i].output); } } private void BackPropagate() { int i, j; double total; //Fix Hidden Layer's Error for (i = 0; i < HiddenNum; i++) { total = 0.0; for (j = 0; j < OutputNum; j++) { total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
} HiddenLayer[i].Error = total; } //Fix Input Layer's Error for (i = 0; i < InputNum; i++) { total = 0.0; for (j = 0; j < HiddenNum; j++) { total += InputLayer[i].Weights[j] * HiddenLayer[j].Error; } InputLayer[i].Error = total; } //Update The First Layer's Weights for (i = 0; i < InputNum; i++) { for(j = 0; j < PreInputNum; j++) { PreInputLayer[j].Weights[i] += LearningRate * InputLayer[i].Error * PreInputLayer[j].Value; } } //Update The Second Layer's Weights for (i = 0; i < HiddenNum; i++) { for (j = 0; j < InputNum; j++) { InputLayer[j].Weights[i] += LearningRate * HiddenLayer[i].Error * InputLayer[j].Output; } } //Update The Third Layer's Weights for (i = 0; i < OutputNum; i++) { for (j = 0; j < HiddenNum; j++) { HiddenLayer[j].Weights[i] += LearningRate * OutputLayer[i].Error * HiddenLayer[j].Output; } } }
As testing the classes requires to train the network first, there must be a folder in this format. "PATTERNS" and "ICONS" folders [depicted below] in the Debug folder fit this format.
History
30th September, 2007: Simplified the app 24th June, 2007: Initial Release
License
This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)
Rate this:
Poor
Excellent
Vote
Profile popups
Noise
Layout
Per page
Go
Refresh My vote of 5 Please help ! Entering and outputting more than one character input value not the same as output value My vote of 3 flow chart Logic behind the programm little confused about PreInput and Input Great Work:-) Increase probability Last Visit: 20:28 15 Feb '12 Last Update: 20:27 15 Feb '12 chikkisherry RonZohan HansiHermann baguswahyu Member 8203782 swarajs praneshkmr hovu ibnkhaldun Member 3913283
Fir 0:45 2
9:12 2
8:38 2
2:35 1 0:05
2:56 2
9:30 2
1234567891
Suggestion
Question
Bug
Answer
Joke
Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.
Permalink | Advertise | Privacy | Mobile Web02 | 2.5.120215.1 | Last Updated 30 Oct 2007 Article Copyright 2007 by Murat Firat Everything else Copyright CodeProject, 1999-2012 Terms of Use Layout: fixed | fluid
Related Articles
Creating animations with Dundas Chart for ASP.NET Smarter Data Labels with Dundas Chart SmartLabels Understanding Chart Areas with Dundas Chart for .NET
Making Sense of Geographic Data with Dundas Map and AJAX DestroyWindow in VBScript SmartLink Create data-driven applications with the Hera Application Framework Towards the self-documenting database: extended properties Digital Signatures and PDF Documents WMP Power Hour APP Using Barcodes in Documents Best Practices How to Retrieve EMC Centera Cluster/Pool Capabilities Using multiple keyboards with different layouts on the same machine "Hey! Is That My Car? How to Sharpen a QuickBird Satellite Image Using DotImage" Integrate your SharePoint environment into the open standards-based WebSphere Portal platform using the Visual Studio IDE VBScript / Excel 2007 - An easy way to access DBF files Retrieving and Storing Call History Knit - A Visual Studio Add-In Drivers, Exceptions and C++ gzTx - A File Transfer Daemon Automating Multipart Zip File Creation