See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/267826732

Bakery products quality control using computer vision: Napolitalàs case

Article

CITATION READS

1 525

5 authors, including:

Luis Manuel Navas Gracia Adriana Correa-Guimaraes

Universidad de Valladolid Universidad de Valladolid
114 PUBLICATIONS   641 CITATIONS    48 PUBLICATIONS   273 CITATIONS   

SEE PROFILE SEE PROFILE

Jaime Gomez-Gil
Universidad de Valladolid
49 PUBLICATIONS   1,548 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Natural products characterization and valorization View project

Computer Vision in Food Industry View project

All content following this page was uploaded by Adriana Correa-Guimaraes on 27 April 2015.

The user has requested enhancement of the downloaded file.

 1

Bakery products quality control using computer vision:

Napolitala`s case

Laura Pariente Bergaz1, Gonzalo Ruiz Ruiz1*, Luis M. Navas Gracia1,

Adriana Correa Guimaraes1, J. Gómez Gil2

1
Department of Agricultural and Forest Engineering, Universidad de Valladolid, Spain
(gruiz@iaf.uva.es) (lmnavas@iaf.uva.es)
2
Department of Communications and Signal Theory and Telematics Engineering
Universidad de Valladolid, Spain (jgomez@tel.uva.es)

* Corresponding author: Gonzalo Ruiz Ruiz - Avenida Madrid, 44,

Campus “La Yutera” – Aulario PALENCIA, 34004, Spain (gruiz@iaf.uva.es)

ABSTRACT

Visual control is used in food and agriculture industry for manufactured goods quality control or
rejecting those products with specifications out of limits in a certain stage of the production
process. In this work some computer vision methods were developed for napolitana (bakery
product) quality control during production time: color and shape defects detection, topping
defects detection and packaging defects detection. All the methods had effectiveness above 95%.

Keywords: Computer vision, food quality control, bakery products, napolitana.

1. INTRODUCTION

Napolitana (Spanish traditional fermented bakery product) is a product classified according the
law into the bakery products (Figure 1). This area of the food and agriculture industry has been
struck for years, mainly by the low added value of these products and their classification as
unhealthful food. In addition, bakery industry had to adapt to the new family models, changing
big packs with high number of units for smaller packs with individually packed products. It made
necessary to invest much money in upgrading and modernizing production lines, which forced
many firms to close or to be sold to large companies.

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.
 2

Figure 1. Napolitana with chocolate topping.

Under these circumstances, it was also essential to improve quality control methods, which is an
added value for bakery products in order to a firm could be selected as a supplier for distribution
chains. To achieve a valuable quality control for industrial bakery products, it should not be
dependent on human errors during inspection, and therefore, quality control systems must be
automated. Computer vision is widely used as a quality control tool in many food industries,
such as sugar, wheat and vegetables industries (Escalera, 2001; Márquez and Añon, 1986;
McDonald and Chen, 1990; Sun, 2000; Sun, 2008)

The napolitana production follows six stages:

 Mixing: the raw material (wheat, sugar, eggs, water, etc.) is weighted, mixed and filtered, if
needed.
 Laminate: the mass is mixed with butter, so the product obtains the typical shape of
fermented products.
 Fermentation: under controlled humidity and temperature, the product grows and acquires
the necessary size.
 Baking: during cooking time the product obtains its final color, texture and humidity.
Furthermore, baking minimizes the microbiological risk.
 Cooling: the temperature of the product is reduced to an acceptable value, which avoids water
condensation during packaging.
 Packaging: napolitanas are packed in plastic trays into plastic bags.

After analyzing the product exhaustively, it is observed that the main quality problems are: color
defects, shape defects, topping defects (syrup and chocolate), trays with lack of units and
napolitanas upside-down into trays. So the objective of this work is the application of computer
vision to detect, measure and control these quality defects, classifying products into and out
specifications.

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.
 3

2. MATERIALS AND METHODS

2.1. Materials

To develop and test computer vision algorithms, it was used an color image bank containing
valid and invalid napolitanas, between 50 and 80 images each class. Images for invalid
napolitanas represent all defects under study: color, shape and topping defects, as well as
uncompleted trays and napolitanas turned upside-down. Images background is white for color,
shape and topping defects; for packaging defects, images were directly taken on the conveyor
belt at the entry of the packaging machine.

Software used to develop algorithms was Vision Builder and LabVIEW, both from National
InstrumentsTM.

2.2. Methods

Different algorithms were developed to detect color, shape, topping and packaging defects. Then
all of them were connected to offer a complete quality control solution.

2.2.1. Color and shape defects detection

This method has two parts, color classification and shape defects detection. To decide if a
napolitana fulfils or not color standards, the algorithm uses the histogram of the red plane and
evaluates the percentage of red component. The second part uses blue plane to segment the
napolitana and then works with brightness plane to extract edges and decide whether it follows a
regular shape or not (Figure 2).

Figure 2. Border analysis for shape defect detection.

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.
 4

2.2.2. Topping defects detection

This algorithm is based on topping detection in subareas of each napolitana. To place these
regions of interest (ROI) it is necessary to place a coordinate system previously. This coordinate
system must be invariant to spatial offset and rotations, so it is placed by detecting two
napolitana’s edges using the blue plane. Then a variable number of rectangular ROIs are used to
cover most of napolitana’s area. In this case red plane is chosen to enhance the contrast between
pastry and chocolate topping (Figure 3).

Figure 3. Division of the napolitana area in six regions

of interest for topping defects detection.

Best results are obtained with six ROIs (2×3) covering the napolitana’s area. By means of an
auto-threshold function, the algorithm measures the percentage of dark objects in each ROI and
then check if this value fulfils specification for all ROIs. A napolitana is rejected if at least one of
ROI has absence of chocolate topping (less than 12% of dark area) or presents topping excess
(more than 45% of dark area).

2.2.3. Packaging defects detection

This algorithm also consists of two parts: detection of empty or partially empty trays and
detection of napolitanas turned upside-down. First of all, it is necessary to place a new
coordinate system which makes the tray position invariant to displacements and rotations. It is
placed by detecting a small metal part which pushes the plastic tray through the packaging
machine (Figure 4).

The coordinate system allows placing three ROIs on the tray, one for each napolitana’s place. To
enhance the product over the tray and the conveyor belt, the blue plane is extracted from color
image (Figure 4). By measuring percentage of dark area within ROI it is possible to detect the
presence (more than 18%) or the absence (less than 18%) of a napolitana in each hole of the tray.

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.
 5

Figure 4. Detection of the metal part which pushes the tray. Blue plane.

The second objective of this stage is to detect napolitanas turned upside-down. To do this, red
plane is used instead of blue due to the enhancement of chocolate topping over the pastry.
Knowing that at this production step all napolitanas have passed topping defects detection, any
one “without” topping must be a napolitana turned upside-down (Figure 5). For each one of the
three ROIs on the tray, percentage of dark area is measured: if it is less than 18% for at least one
ROI, the tray is rejected assuming one or more napolitanas are upside-down.

Figure 5. A plastic tray with tree napolitanas, one of them upside-down.

3. RESULTS

In all the cases, the effectiveness of the computer vision systems is above 95%. Specifically,
95% occurs for detection of color and shape defects, while topping defects and packaging defects
detection achieve 100% of correct classifications.

In the case of color and shape defects (Figure 6) and topping defects (Figure 7), they are
necessary five parallel computer vision systems, because this is the number of production lines in
a specific case. After high efficiency of developed algorithms, there exist two options to reject
invalid products:

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.
 6

 Bypass: this system could switch groups of invalid napolitanas to a rejection belt, but it only
would be profitable for invalid product ratios higher than 0.40%
 Automation: in this case, only invalid products would be rejected, instead of groups
containing valid and invalid units, using one robot in each production line. This options would
be more expensive than bypass and profitable for invalid product ratios higher than 1.15%.

Rejecting not valid packages could be profitable for invalid product ratios higher than 0.52%,
using one vision system in each packaging machine.

Figure 6. A napolitana with shape defects.

Figure 7. A napolitana with topping defects.

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.
 7

4. CONCLUSIONS

After developing inspection algorithms for napolitanas quality control and making the respective
economic study, some conclusions have been reached:

 Computer vision systems for quality control in napolitanas production are viable, given high
rates of well classified products.
 The computer vision system is valid for color, shape, topping and packaging defects, but the
implementation is easier for packaging control.
 Effectiveness percentages are above 95% for color and shape defects detection. For topping
and packaging defects, it reaches 100%.
 The economic viability of the systems could be possible for all cases under study, in addition
to the added value.
 Computer vision systems can improve the quality control in agriculture and food.

5. ACKNOWLEDGEMENTS

The authors are grateful to “GRUPO SIRO” for its support during the carrying out of this work.

6. REFERENCES

Escalera Hueso, A. De la. Visión por computador. Fundamentos y métodos. Prentice Hall. 2001.
Márquez, G.; Añon, M.C.; Influence of reducing sugars and amino acids in the color
development of fried potatoes. Journal of Food Science. 1986.
McDonald, T.; Chen, Y.R.; Application of morphological image processing in agriculture.
Transactions of the ASAE. 1990.
Sun, D.W. Inspecting pizza topping percentage and distribution by a computer vision method.
Journal of Food Engineering. 2000.
Sun, D.W. Computer Vision Technology for food quality evaluation. Elsevier. 2008.

L. Pariente Bergaz, G. Ruiz Ruiz, L.M. Navas Gracia, A. Correa Guimaraes, J. Gómez Gil.
“Bakery products quality control using computer vision: Napolitana`case”. CIGR Workshop on
Image Analysis in Agriculture, 26-27. August 2010, Budapest.

View publication stats