1673

Journal of Food Protection, Vol. 75, No. 9, 2012, Pages 1673–1690

doi:10.4315/0362-028X.JFP-11-545
Copyright G, International Association for Food Protection

Application of Hazard Analysis and Critical Control Point

Methodology and Risk-Based Grading to Consumer Food
Safety Surveys
ELIN HALBACH RØSSVOLL,1 ØYDIS UELAND,1 THERESE HAGTVEDT,1 EIVIND JACOBSEN,2 RANDI LAVIK,2 AND
SOLVEIG LANGSRUD1*

1Nofima, Norwegian Institute of Food, Fisheries and Aquaculture Research, P.O. Box 210, N-1431 Ås, Norway; and 2SIFO, National Institute for Consumer
Research, P.O. Box 4682 Nydalen, N-0405 Oslo, Norway

MS 11-545: Received 13 December 2011/Accepted 19 April 2012

ABSTRACT
Traditionally, consumer food safety survey responses have been classified as either ‘‘right’’ or ‘‘wrong’’ and food handling
practices that are associated with high risk of infection have been treated in the same way as practices with lower risks. In this
study, a risk-based method for consumer food safety surveys has been developed, and HACCP (hazard analysis and critical
control point) methodology was used for selecting relevant questions. We conducted a nationally representative Web-based
survey (n ~ 2,008), and to fit the self-reported answers we adjusted a risk-based grading system originally developed for
observational studies. The results of the survey were analyzed both with the traditional ‘‘right’’ and ‘‘wrong’’ classification and
with the risk-based grading system. The results using the two methods were very different. Only 5 of the 10 most frequent food
handling violations were among the 10 practices associated with the highest risk. These 10 practices dealt with different aspects
of heat treatment (lacking or insufficient), whereas the majority of the most frequent violations involved storing food at room
temperature for too long. Use of the risk-based grading system for survey responses gave a more realistic picture of risks
associated with domestic food handling practices. The method highlighted important violations and minor errors, which are
performed by most people and are not associated with significant risk. Surveys built on a HACCP-based approach with risk-based
grading will contribute to a better understanding of domestic food handling practices and will be of great value for targeted
information and educational activities.

One major goal for the Norwegian Food Safety
Authority is safe food from farm to fork (12). Food safety
managers have traditionally set stringent performance
criteria for food processing steps and for the accepted level
of microorganisms at several points along the food chain to
reduce cases of food poisoning and food contamination. The
hazard analysis and critical control point (HACCP) system
is an approach that can be used to ensure safe food
production, and this system is gradually being implemented
into food safety legislation all world wide. At present,
legislation is aimed at reducing risk and promoting safe
foods and handling practices at every point of the food
production chain, from the primary producer to the
consumer. However, it is both difficult and undesirable to
manage consumer food handling practices through legisla-
tion. Regulations covering shelf life and the accepted level
of microorganisms at the expiration date have been issued to
protect consumers, but product treatment after purchase by
the consumer is unknown. Consequently, food risks
associated with consumer practices are difficult to predict
and prevent.
* Author for correspondence. Tel: z4764970100; Fax: z4764970333;
E-mail: solveig.langsrud@nofima.no.
Knowledge of consumer behavior is important for the
food industry for conducting risk analyses of existing
products. This knowledge would enable production of safer
products through optimization of product packaging or
labeling and could be used by food safety authorities, the
food industry, schools, and others to tailor specific food
safety information to the consumers who need it.
A number of approaches have been used to identify and
determine the prevalence of unsafe food handling practices
among consumers (2, 23, 26, 31–33, 38, 44, 45, 61, 67, 70,
72, 73, 81, 89, 91, 97, 100, 102, 106, 110). These studies
have included either measurement of self-reported behavior
or measurement of observed behavior. Self-reported behav-
ior data are often collected by telephone interviews or
written or Web-based questionnaires. A large number of
consumers are often involved, and the survey may cover
several food preparation processes and food types. The
opportunity to study large groups and collect sufficient data
for statistical analysis is one of the major advantages of this
approach. A limitation of these surveys is that the self-
reported practices do not necessarily reflect actual behavior
(96, 97). Observation studies often take place in the
participant’s home, and observation is conducted by
personnel or through the use of cameras. The advantage
1674 RØSSVOLL ET AL.
of this type of study is the real-life context, which makes it
possible to identify unforeseen risk situations or behaviors.
However, the natural behavior of the participants may be
compromised; they may change their behavior, consciously
or unconsciously, because they are aware of being observed
(96).
One aspect of risk measurement in consumer food
handling studies that has received much less attention is the
effect of the way questions are asked. Published question-
naires should not be copied uncritically and used in other
studies. Although the procedures that control pathogens can
and should be universally applicable, variations may exist
among countries regarding zoonoses, pathogens, and food
cultures. Another factor to be considered is the rapid
development in the global food market. Food is increasingly
being distributed in an international market, and a single
hygienic breakdown during food production could have
great international consequences. New pathogens are
emerging and can be spread globally at great speed (105).
Because of the global food market, consumers are being
presented with new types of foods, which require new
approaches to both hygienic issues and preparation
practices. This globalization creates new challenges for the
food industry, food safety authorities, and the consumer.
Extrapolating surveys across nations can also be a
challenge, due to different food risk situations among
countries. Of equal importance is the consideration of
national differences in eating habits, climate, and culture. To
understanding of consumer behavior, it is crucial that these
issues are addressed during the development of consumer
food safety questionnaires.
The lack of focus on the development of questionnaires
with regard to exposure of risk may be due to the fact that
previous studies have had other objectives. For many
studies, the goal has not been to describe domestic food
handling with regard to the risk of foodborne infection but
rather to examine consumer attitudes (27, 70) and different
aspects of food safety knowledge (2, 61, 67, 73, 81).
A recurring issue in food safety surveys is that food
handling practices associated with high risk of infection are
treated as equal to those practices associated with lower risk.
The survey participants’ responses are classified as either
right or wrong, or acceptable or unacceptable. By using this
methodology, it is not possible to differentiate between
high-risk practices (e.g., tasting raw meat) and low-risk
practices (e.g., thawing food at room temperature). To our
knowledge, only one study has included a system of
measuring the degree of likely risk to the consumer
concerning specific food handling practices. Griffith et al.
(49) and Worsfold and Griffith (109, 110) developed a
standardized risk-based scoring system for observational
studies that included a larger number of consumers and a
range of homemade food products.
A more well-known method used to identify, evaluate,
and control hazards significant for food safety is the
HACCP system (3), which has been recommended by the
World Health Organization (WHO) for more than 30 years
(108). The HACCP methodology has been applied to the
food production chain but has been implemented rarely in
J. Food Prot., Vol. 75, No. 9
the domestic environment. However, the official standards
of the Codex Alimentarius Commission (3) follow the food
chain from primary production to final consumption and
specifically state that everyone, from farmers and growers
to manufacturers and processors to food handlers and
consumers, has a responsibility to assure that the food is safe
and suitable for consumption. Attempts have been made to
implement HACCP systems in the domestic kitchen (21, 50,
62, 86, 87, 93, 99, 101), and a risk-based approach to
consumer safety is used by organizations such as the
International Scientific Forum on Home Hygiene (10) as a
basis for advice to consumers.
The objectives of this study were (i) to develop a more
risk-based method for consumer food safety questionnaires
and (ii) to use a HACCP-based approach for choosing
relevant questions. A HACCP-based approach was used
to ensure that all the practices performed in a domestic
kitchen were adequately addressed in the questionnaire, and
the various food safety violations consumers perform in the
domestic kitchen were graded according to both the
frequency and safety risk of the self-reported actions. This
method therefore more accurately measures the actual safety
risk of consumers’ self-reported food handling practices.
MATERIALS AND METHODS
HACCP plans were made for four food preparation processes
that normally take place in a Norwegian domestic kitchen. The
accuracy of HACCP flow diagrams was confirmed by interviewing
and observing consumers in their own kitchens. A consumer food
safety questionnaire was developed and designed based on the
HACCP results. The food safety survey was then conducted, and
the results were analyzed with and without the risk-based grading
system. Figure 1 shows an overview of the methods used in this
study.
Choice of food preparation processes. Four food prepara-
tion processes were chosen, based on three selection criteria: (i)
meals normally prepared in a Norwegian domestic kitchen (30,
71), (ii) preparation processes that involve the WHO five keys to
safer food (107), and (iii) meals that include processes or food
products in which the most important food pathogens in Norway
have been implicated (17, 52). The following four meals were
selected: hamburger with salad, chicken salad or chicken fillet with
salad, cold cuts of boiled ham, and rice porridge.
Flow diagram construction. The Codex Alimentarius
Commission (3) defined a flow diagram as a systematic repre-
sentation of the sequence of steps or operations used in the
production of a particular food item. Flow diagrams for the four
different food processes were made, starting with purchase of the
raw materials and ending with handling of leftovers. With the help
of a professional market research company (NORSTAT, Oslo,
Norway), a study group of 15 consumers in and around Oslo were
interviewed and filmed as they were preparing dinner for their
families. This group consisted of 5 men and 10 women, 11 of them
with children and 5 of them living outside Oslo. In these interviews
and video sessions (conducted in November 2010), the interview-
ees were given a choice of dishes to prepare, including a chicken
dish and a dish containing minced meat. Interviewees were free to
proceed with their preparations as they wished. The video
recordings were used to validate the flow diagrams, i.e., to
determine whether they accurately reflected the actual preparation
J. Food Prot., Vol. 75, No. 9 HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS
FIGURE 1. Overview of the different methods used in this study.
processes, which they predominantly did. Thus, the flow diagrams
were considered appropriate because no major deviations to the
preparation processes were identified.
None of the participants made rice porridge or cold cuts of
boiled ham. Rice porridge was mentioned as a common Saturday
dish by several of the interviewees. Repeated exposure at room
temperature during meals is the most important step when handling
cold cuts of boiled ham. The most interesting operation associated
with rice porridge is the treatment of leftovers. Both of these food
handling operations last for several days, which would not have
been possible to study in video sessions.
HACCP. All microbiological hazards associated with each
step in the flow diagrams were listed. Hazard analyses for all four
processes were conducted, based on the four flow diagrams. The
hazard analyses were limited to microbiological hazards; chemical
or physical hazards were not included. Critical control points
(CCPs) were identified by using the decision tree recommended in
the Codex (3). Important steps that did not qualify as CCPs but
were still important for food safety were classified as operational
prerequisite programs (OPRPs) (5, 6, 104).
Selection of questions. A total of 220 questions were
collected from nine food safety surveys conducted in other
countries (1, 27, 44, 45, 59, 61, 68, 81, 89). All questions were
sorted into categories based on the HACCP results from the four
food preparation processes and the WHO five keys to safer food
(107).
The HACCP methodology was used to determine whether
certain categories were incompletely covered or missing. For
instance, being ill while preparing food for others and tasting raw
1675
or partially raw minced meat while cooking were important
practices according to the HACCP method. These issues were not
completely covered in previous studies, so questions about these
subjects were constructed and included in the questionnaire.
Questions that were overlapping or difficult to answer or that had
limited relevance to Norwegian conditions were removed.
Questions that were vague or that could be misinterpreted were
rewritten.
The questions were evaluated by the project group, i.e., six
researchers whose expertise included food microbiology, food
safety, HACCP systems, veterinary medicine, and consumer
research. An advisory board consisting of representatives from
the Norwegian Food Safety Authority, food industry, retailers, and
other stakeholders also contributed to design of the questionnaire.
In its revised form, the questionnaire contained 33 questions
about food safety knowledge and risk-based food handling
practices. Most answers were based on a five-point ordinal scale,
with choices of ‘‘always,’’ ‘‘often,’’ ‘‘sometimes,’’ ‘‘seldom,’’ and
‘‘never.’’ An alternative of ‘‘don’t know’’ or ‘‘not applicable’’
also was available (74).
The questionnaire was part of a broader interdisciplinary
study with a total of 54 questions. Eleven questions on demo-
graphic conditions were added for the subsequent analysis. The
questionnaire took approximately 20 min to complete.
Quality assurance. The questionnaire was revised by 14
consumers, five food scientists who did not participate in the
project, and NORSTAT. These advisors were asked to comment on
the wording and format, state whether they understood the
questions, indicate any ambiguities, and identify potential errors
or missing parts. Adjustments were made based on this feedback.
Pilot survey. A pilot study of the revised questionnaire was
conducted in 50 households in March 2009. The study results
indicated no need for further revisions; the questions and the
questionnaire seemed to function according to expectations.
Study population and sampling plan. The study participants
were selected from a telephone recruited panel consisting of 72,000
people living in Norway, which was developed and maintained by
NORSTAT. The requirement for composition of the study group
was that it should be nationally representative for Norway, i.e.,
composition should be proportional to the population in different
parts of the country, and quotas were allocated based on gender
and county. All respondents were 18 years of age or older. All
respondents who completed the Web survey had the opportunity to
win a gift voucher of NOK 5000, in addition to NORSTAT’s
incentive program. The survey was conducted during March and
April 2009, with a total of 2,008 respondents (58).
Risk-based grading with demerit points. The self-reported
answers in the survey were given a score based on demerit points
awarded for a food safety and/or hygiene violation or for absence
or lack of knowledge of recommended controls or preventative
measures. This score was based on the Worsfold and Griffith food
operation risk (FOR) score, a logarithmic scale ranging from 0 to
90 demerit points (49, 96, 109, 110). This method originally
included direct observation and temperature measurement in
combination with a standardized risk-based scoring system derived
from epidemiological data (49, 109, 110). A high risk score
indicates that more food safety violations occurred or that fewer
control measures were applied (49, 96). Modifications to the
original FOR scoring scheme were made for a better fit to the
survey format and present Norwegian conditions. The risk for
1676 RØSSVOLL ET AL.
Norwegian consumers making food in other countries or importing
their own food from other countries with different food safety
situations was not taken into account.
Frequency coefficient. In addition to the demerit points, the
self-reported frequency of a particular violation was taken into
account. A frequency coefficient between 0 and 1 was given
according to how often respondents reported performing the
specific violation (always or often ~ 1, sometimes or seldom ~
0.5, and never ~ 0). The frequency coefficient was multiplied by
the violation’s demerit point score to get a total risk score:
frequency coefficient | demerit points ~ total risk score. Thus,
respondents who reported performing a violation ‘‘always’’ and
‘‘often’’ received a higher total risk score than did respondents
who reported performing a violation ‘‘sometimes’’ or ‘‘seldom.’’
Respondents who claimed a violation ‘‘never’’ happened or
answered ‘‘not applicable’’ were not awarded any demerit points.
This approach also was applied to questions concerning
degree of agreement with a specific statement about food safety
(totally or partly agree, neither agree nor disagree or partly
disagree, and totally disagree), questions about knowledge of
certain food safety practices (‘‘do you know the temperature in
your refrigerator?’’), questions about how often respondents
consumed high-risk foods, and questions about how often a
particular food safety practice was performed (e.g., hand washing
before food preparation).
Calculation with and without risk-based demerit points.
Total risk scores were calculated by multiplying a violation’s
demerit points, ranging from 0 to 90, by its frequency coefficient.
The same calculation also was made without the risk-based demerit
points; demerit points were replaced with the classification
‘‘correct’’ or ‘‘wrong,’’ which gave all questions a new demerit
value of 0 or 1, respectively. The frequency coefficient was
unchanged and used to multiply the question’s demerit points as
previously described.
Data analysis. The questionnaire responses were analyzed
using SPSS Statistics version 19 software (IBM, Armonk, NY) and
Office Excel 2003 (Microsoft, Redmond, WA).
RESULTS
HACCP. Tables 1 through 4 show the HACCP results
for the four food preparation processes with identified CCPs
and OPRPs.
The questionnaire. The 33 questions about food safety
knowledge and handling, their demerit points, and the
frequency coefficients are shown in Table 5. Worsfold and
Griffith’s (109, 110) FOR score system was adjusted and
adapted to present Norwegian conditions. The concept was
first introduced in 1982 by Bryan (29), who originally
developed a risk index system for the catering industry that
was the basis for the development of a customized scoring
system for use in the domestic environment (49, 109, 110).
Worsfold and Griffith adjusted the FOR score so that the
risk was related to the food itself. The resultant food safety
risk score was the sum of the demerit points a person
received multiplied by the potential risk of the food
involved (49, 109). The risk related to the food was
expressed as the food risk coefficient, ranging from 1 to 5,
based on the food’s association with foodborne outbreaks
J. Food Prot., Vol. 75, No. 9
(109). The food risk coefficients considered eggs, chicken,
ham, and beef (49, 109). The original food risk coefficients
as defined by Worsfold and Griffith were not applicable
to Norwegian conditions. Norwegian eggs and meat are
associated with an insignificant risk of salmonellosis
because Norwegian livestock populations are virtually free
from Salmonella (53). Norwegian livestock populations and
the food industry have been subject to controls for decades,
and until 1999 there was a general ban on import of live
animals and animal products to Norway (7). Imports of live
animals after the general ban was lifted remain limited
(7, 85) because of protectionism with high tariffs. The
relatively cold climate, the geographical outpost position in
Europe, and the prevalence of small, dispersed livestock
farms are factors that also contribute to the lack of a national
Salmonella problem (7, 14, 64). The Norwegian Salmonella
surveillance and control programs (63) are approved by the
European Union Commission (43), allowing Norway to
require additional guarantees regarding Salmonella when
importing live animals, feed, and food products of animal
origin from the European Union (63). The estimated
prevalence of Salmonella is less than 0.3% in the examined
populations of Norwegian cattle, swine, and poultry for all
years since 1995, when the surveillance programs were
established (7, 63). From 1982 to the present, no reported
salmonellosis outbreaks have been associated with Norwe-
gian meat products (17). Risk of exposure to Salmonella is
mainly associated with international trade in food (7). In
2010, the import of beef to Norway constituted 6% of the
total beef production, and the import of poultry meat
constituted 0.2% of the total poultry production (15, 78).
Approximately 1,500 human cases of salmonellosis are
reported each year in Norway, of which 75 to 80% are
acquired outside the country (7, 18).
Because the food risk coefficients were not used in
this study, the potential risk of the food involved was
incorporated into the specific food handling demerit
points. The demerit points were intended to take into
account the risk and severity of each process hazard and
the desirability of applying control measures to reduce or
eliminate the hazard at each stage of the food preparation
process (109, 110). Behaviors associated with risk of
direct transmission of pathogens or risk of infection with
highly virulent pathogens were awarded 50 demerit points,
e.g., undercooking meat, consumption of raw meat, and
improper cooling of large portions (19, 29, 49, 77, 95, 109,
110). Respondents who stated that they did not avoid
preparing food for others while being ill themselves with
vomiting and/or diarrhea were awarded 50 demerit points
because this behavior is a well-known way of spreading
contagious diseases, with the food acting as a vehicle (17,
22, 48).
Agricultural produce was not considered in the original
food risk coefficients. According to Worsfold and Griffith’s
FOR score, unwashed parsley, leeks, and lettuce were given
2 demerit points (49, 109, 110). Because of recent national
and international outbreaks associated with imported lettuce
(41, 65, 80, 92), sprouts (9, 39, 90), and sugar peas (51),
these items were considered high-risk foods. Rangel et al.
TABLE 1. HACCP analysis for hamburger and salad
Demerit Survey CCP or
Process step Hazardsa Possible source or causeb pointsb question OPRPc

Procuring minced meat and salad Presence of PB (e.g., EHEC, sporogenous Damaged packaging
bacteria, and Staphylococcus aureus) and Older than ‘‘use by’’ date 5 1
viruses Contaminated raw material
J. Food Prot., Vol. 75, No. 9

Transporting minced meat and salad Growth of PB, toxin production Temperature abuse during transport
Cooling and/or storage of minced meat and salad Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
Opening minced meat package Meat packaging contaminates work surface and/or
equipment
Preparing food for others during own illnessd 50 33
Handler does not wash hands after handling raw meate 15 30, 32
Mixing ingredients and shaping hamburgers Presence of PB, viruses, and/or toxins Process contaminates work surface and/or equipment
Handler does not wash hands after handling raw meat 15 30, 32
Tasting raw minced meatd 50 8
Cooking hamburgers Survival of PB and viruses and heat stable Not cooked to internal temperature of at least 74uC 50 9 CCP
toxins Tasting raw minced meat 50 8
Presence of PB, viruses, and/or toxins
Washing salad Persistence of PB and/or viruses Lettuce and ingredients not washedf 50 7, 17, 18 OPRP
Cutting salad Contamination with PB and/or viruses Dirty cutting board and/or knife 10 21, 20, 19
Serving, cooling, and storing hamburgers and salad Growth of PB, toxin production Food kept at room temperature for .2 hg 20 11 OPRP
at room temperature
Cooling hamburgers and salad (immediately after Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
cooking or leftovers)
Reheating hamburgers Survival of PB and viruses, growth of PB, Not cooked to internal temperature of at least 74uC 50 14 CCP
production of heat stable toxins
New serving, cooling, and storing hamburgers and Growth of PB (sporogenous bacteria), toxin Food kept at room temperature for .2 h 20 11 OPRP
salad at room temperature production
Cooling and storing hamburgers and salad Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
Reheating hamburgers a third time Survival of PB (sporogenous bacteria), Not cooked to internal temperature of at least 74uC 50 14 CCP
production of heat stable toxins Food reheated more than once, with intervening holding 40 12
periods at room temperature
a
PB, pathogenic bacteria; EHEC, enterohemorrhagic Escherichia coli.
b
From Worsfold and Griffith (109, 110).
c
CCP, critical control point; OPRP, operational prerequisite program.
d
Not included in the original demerit points of Worsfold and Griffith (109, 110) but an important point according to the HACCP analyses because of the potential for direct infection.
e
Omitting hand washing after handling raw meat is an important step according to the HACCP analyses, so it was given a penalty of 15 demerit points, instead of the 10 points allotted by Worsfold
HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS

and Griffith (109, 110). (Fried chicken cut on a dirty board was given 15 demerit points by Worsfold and Griffith (110).)
f
According to Worsfold and Griffith (109, 110), 2 demerit points were given when parsley, leeks, and lettuce were not washed. Because of recent national and international outbreaks associated with
imported lettuce, sprouts, and sugar peas, not washing lettuce or parboiling sprouts or sugar peas was given 50 demerit points owing to the potential for direct infection.
g
According to Worsfold and Griffith (109, 110), 30 demerit points were given when foods were kept at room temperature for longer than 3 h but less than 6 h. Foods kept at room temperature for
longer than 6 h but less than 12 h were awarded 60 demerit points, and foods kept at room temperature for longer than 12 h were awarded 90 demerit points.
1677
TABLE 2. HACCP analysis for chicken and salad
1678

Demerit Survey CCP or

Process step Hazardsa Possible source or causeb pointsb question OPRPc

Procuring chicken and salad Presence of PB (e.g., Campylobacter, Damaged packaging

Transporting chicken and salad
Cooling and storage of chicken and salad
Opening package of chicken
Salmonella, sporogenous bacteria,
Staphylococcus aureus) and viruses
Growth of PB, toxin production
Survival and growth of PB, toxin production
Older than ‘‘use by’’ date 5 1
Contaminated raw material
Temperature abuse during transport
Refrigerated at too high temperature (above 4uC) 10 16
Chicken packaging contaminates work surface and/or equipment
RØSSVOLL ET AL.

Preparing food for others during own illnessd 50 33

Handler does not wash hands after handling raw meate 15 30, 32
Washes chicken
Handling and preparing raw chicken Presence of PB and viruses Contaminating work surface and/or equipment
Handler does not wash hands after handling raw meat 15 30, 32
Tasting raw chickend 50 8
Cooking chicken Survival of PB and viruses, production of Not cooked to internal temperature of at least 74uC 50 10 CCP
heat stable toxins Tasting raw chicken 50 8, 10
Presence of PB and viruses
Washing salad Persistence of PB and/or viruses Lettuce and ingredients not washedf 50 7, 17, 18 OPRP
Cutting salad or possibly mixing cooked chicken Contamination with PB and/or viruses Dirty cutting board and/or knife 10 21, 20, 19
into salad
Serving, cooling, and storing chicken and salad at Growth of PB, toxin production Food kept at room temperature for .2 hg 20 11 OPRP
room temperature
Cooling and storing chicken and salad (immediately Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
after cooking or leftover)
Reheating chicken Survival of PB and viruses, growth of PB, Not cooked to internal temperature of at least 74uC 50 14 CCP
production of heat stable toxins
New serving, cooling, and storing chicken and salad Growth of PB (sporogenous bacteria), toxin Food kept at room temperature for .2 h 20 11 OPRP
at room temperature production
Cooling and storing chicken and salad Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
Reheating chicken for third time Survival of PB (sporogenous bacteria), Not cooked to internal temperature of at least 74uC 50 14 CCP
production of heat stable toxins Food reheated more than once, with intervening holding 40 12
periods at room temperature
a
PB, pathogenic bacteria.
b
From Worsfold and Griffith (109, 110).
c
CCP, critical control point; OPRP, operational prerequisite program.
d
Not included in the original demerit points of Worsfold and Griffith (109, 110) but an important point according to the HACCP analyses because of the potential for direct infection.
e
Omitting hand washing after handling raw meat is an important step according to the HACCP analyses, so it was given a penalty of 15 demerit points, instead of the 10 points allotted by Worsfold
and Griffith (109, 110). (Fried chicken cut on a dirty board was given 15 demerit points by Worsfold and Griffith (110).)
f
According to Worsfold and Griffith (109, 110), 2 demerit points were given when parsley, leeks, and lettuce were not washed. Because of recent national and international outbreaks associated with
imported lettuce, sprouts, and sugar peas, not washing lettuce or parboiling sprouts or sugar peas was given 50 demerit points owing to the potential for direct infection.
g
According to Worsfold and Griffith (109, 110), 30 demerit points were given when foods were kept at room temperature for longer than 3 h but less than 6 h. Foods kept at room temperature for
longer than 6 h but less than 12 h were awarded 60 demerit points, and foods kept at room temperature for longer than 12 h were awarded 90 demerit points.
J. Food Prot., Vol. 75, No. 9
 J. Food Prot., Vol. 75, No. 9

TABLE 3. HACCP analysis for cold cuts of boiled ham

Process step Hazardsa Possible source or causeb Demerit pointsb Survey question CCP or OPRPc

Procuring cold cuts Presence of PB (e.g., Listeria, Yersinia, sporogenous Damaged packaging
bacteria, Staphylococcus aureus) and viruses Older than ‘‘use by’’ date 5 1, 2
Contaminated raw material
Transporting cold cuts Growth of PB, toxin production Temperature abuse during transport
Cooling and storing cold cuts Survival and growth of PB (Listeria, Yersinia), toxin Refrigerated at too high temperature (above 4uC) 10 16 OPRP
production
Opening package of cold cuts Packaging contaminates work surface and/or
equipment
Preparing food for others during own illnessd 50 33
Serving Growth of PB, toxin production Food kept at room temperature for .2 he 20 11 OPRP
Cooling and storing cold cuts Survival and growth of PB (Listeria, Yersinia), toxin Refrigerated at too high temperature (above 4uC) 10 16 OPRP
production
New serving Growth of PB, toxin production Food kept at room temperature for .2 h 20 11 OPRP
Cooling and storing cold cuts Survival and growth of PB (Listeria, Yersinia), toxin Refrigerated at too high temperature (above 4uC) 10 16 OPRP
production
a
PB, pathogenic bacteria.
b
From Worsfold and Griffith (109, 110).
c
CCP, critical control point; OPRP, operational prerequisite program.
d
Not included in the original demerit points of Worsfold and Griffith (109, 110) but an important point according to the HACCP analyses because of the potential for direct infection.
e
According to Worsfold and Griffith (109, 110), 30 demerit points were given when foods were kept at room temperature for longer than 3 h but less than 6 h. Foods kept at room temperature for
longer than 6 h but less than 12 h were awarded 60 demerit points, and foods kept at room temperature for longer than 12 h were awarded 90 demerit points.
HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS
1679
 1680

TABLE 4. HACCP analysis for rice porridge

Process step Hazardsa Possible source or causeb Demerit pointsb Survey question CCP or OPRPc
RØSSVOLL ET AL.

Procuring rice and milk Presence of PB (e.g., Bacillus and Clostridium) Damaged packaging
Older than ‘‘use by’’ date 5 1
Contaminated raw materials
Transporting rice and milk Growth of PB, toxin production Temperature abuse during transport
Cooling and storing milk Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
Mixing rice and milk Preparing food for others during own illnessd 50 33
Cooking Survival of PB, production of heat stable toxins Not cooked to internal temperature of at least 50 CCP
74uC
Serving and storing at room Growth of PB, toxin production Food kept at room temperature for .2 he 20 11 OPRP
temperature
Cooling Survival and growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
Reheating Survival of PB Not cooked to internal temperature of at least 50 14 CCP
74uC
New serving and storing at room Growth of PB (sporogenous bacteria), toxin production Food kept at room temperature for .2 h 20 11 OPRP
temperature
Cooling and storing Growth of PB, toxin production Refrigerated at too high temperature (above 4uC) 10 16
Reheating for third time Survival of PB (sporogenous bacteria), production of Not cooked to internal temperature of at least 50 14 CCP
heat stable toxns 74uC
Food reheated more than once, with intervening 40 12
holding periods at room temperature
a
PB, pathogenic bacteria.
b
From Worsfold and Griffith (109, 110).
c
CCP, critical control point; OPRP, operational prerequisite program.
d
Not included in the original demerit points of Worsfold and Griffith (109, 110) but an important point according to the HACCP analyses because of the potential for direct infection.
e
According to Worsfold and Griffith (109, 110), 30 demerit points were given when foods were kept at room temperature for longer than 3 h but less than 6 h. Foods kept at room temperature for
longer than 6 h but less than 12 h were awarded 60 demerit points, and foods kept at room temperature for longer than 12 h were awarded 90 demerit points.
J. Food Prot., Vol. 75, No. 9
J. Food Prot., Vol. 75, No. 9 HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS
(94) reviewed Escherichia coli O157:H7 outbreaks reported
in the United States from 1982 to 2002. The food vehicle in
41% of the foodborne outbreaks was ground beef, and
produce was associated with 21% of the outbreaks. Greig
and Ravel (47) analyzed 4,093 foodborne outbreaks
reported internationally between 1988 and 2007 for which
the agent and the food source were known; 12% of these
outbreaks were associated with produce, and 12% were
associated with beef. Globalization of the food supply has
made fruits and vegetables available regardless of season
(103). Of the total lettuce and other fresh vegetables in
Norway in 2010, around 65% was imported (13). Produce
can be contaminated at any point during growth, harvesting,
processing, distribution, and preparation (24, 34, 37, 79).
Farming and processing practices in many countries could
therefore play an important role in produce-associated
outbreaks (103). The leading Norwegian company that
provides fresh and freshly processed fruits and vegetables
purchases produce from 150 suppliers in 80 countries
worldwide (8), including countries where Salmonella,
Shigella, and enterohemorrhagic E. coli are more commonly
associated with produce.
Since 1994, nine outbreaks of infectious intestinal
diseases linked to lettuce, sprouts, sugar peas, and basil have
been reported in Norway, with more than 260 confirmed
cases. Eight of the nine national outbreaks were due to
imported produce (11, 36, 39, 41, 51, 65, 80, 92). Since
1989, only eight outbreaks in Norway have been associated
with meat and poultry, with more than 218 confirmed cases,
and three of these eight outbreaks were associated with
imported meat and chicken (17, 28, 57). Neglecting to wash
lettuce and parboil raw vegetables such as sugar peas and
sprouts were therefore awarded 50 demerit points, because
eating such agricultural produce without any preventive
measures such as washing or parboiling is a potential source
of direct infection.
Consumption of unpasteurized cheese, raw meat as
steak tartare or carpaccio, and traditional Scandinavian
foods such as half-fermented fish (rakfisk) and cured
salmon (gravlaks) was awarded 50 demerit points. The lack
of heat treatment for these products and the long
fermentation process of rakfisk and gravlaks makes these
foods high risk, and they have been involved in many cases
of foodborne infection and intoxication (17, 20, 40, 42, 46,
60, 88, 98).
Forty demerit points were assigned for repeatedly
reheating cooked foods after they were held at room
temperature (109, 110). Twenty demerit points were
awarded for storing foods for more than 2 h at room
temperature. Originally, Worsfold and Griffith awarded 30
demerit points for storing food at room temperature for
longer than 3 h. Foods kept at room temperature for longer
than 6 h were given 60 demerit points, and a maximum of
90 demerit points were awarded when foods were kept at
room temperature for longer than 12 h (109, 110). The
importance of storing foods at room temperature was
adjusted downward compared with undercooking of meat
and eating raw meat, for instance, because enterohemor-
1681
rhagic E. coli in meat constitutes a greater risk of severe
disease than do sporogenous bacteria (83).
Processes that are known risk factors for cross-
contamination, such as not washing hands immediately
after handling raw meat and or after toilet visits (25, 69),
were given 15 demerit points. In Worsfold and Griffith’s
original FOR score system, 10 demerit points were awarded
for not washing hands after handling raw chicken (49, 109,
110), although a penalty of 15 demerit points was added
for cutting fried chicken on dirty boards (110). Fewer
microorganisms are transferred through cross-contamination
from raw meat than by eating the raw meat directly, thus
these cross-contamination practices were assigned fewer
demerit points. Other cross-contamination practices such as
insufficient washing of knives and cutting boards, improper
use of kitchen cloths, and insufficient hand washing were
awarded 10 demerit points.
Lack of control of the temperature in the home
refrigerator was assigned 10 demerit points (109, 110).
Processes such as thawing of raw meat and meat products
contribute infrequently to foodborne infection and intoxi-
cation and were awarded 5 demerit points. This point value
is lower than that originally used by Worsfold and Griffith
(109, 110), who assigned 10 demerit points for the thawing
of raw foods and storage of frozen foods. The demerit score
for thawing raw meat and meat products was lowered based
on the findings of Ingham et al. (56) and Lianou and
Koutsoumanis (76), who found that thawing whole chicken
and retail ground beef at room temperature was not
particularly hazardous. Lack of control or lack of knowledge
of ‘‘use-by’’ dates was assigned 5 demerit points, as
originally proposed by Worsfold and Griffith (109, 110).
The incidence of listeriosis in the United Kingdom in 1995,
when the original demerit points were assigned (4, 82), was
considered similar to the present incidence of listeriosis in
Norway (16); therefore, the demerit points associated with
use-by dates were not changed.
Risk-based grading. Table 6 shows the results of the
two demerit points systems, with and without risk-based
grading, for each survey question. The question on cooling
of large portions generated most demerit points when
analyzed with risk-based grading. Without risk-based
grading, the question on thawing meats at room temperature
was given the highest score. The question about hand
washing after toilet visits was given the lowest score with
both methods.
DISCUSSION
In the present study, we developed a new approach to
risk-based food safety questionnaires. A number of
methodologies were combined to provide a tool for better
grading of various food handling practices to measure the
risk of acquiring foodborne disease. The HACCP method
was used both for selecting the questions to the question-
naire and for subsequent risk-based grading of the
questions. The CCPs and the OPRPs in the HACCP plan
coincided to a great extent with the risk-based demerit
points (Tables 1 through 4). The frequencies of food
TABLE 5. Questions about food safety knowledge and handling, their demerit points, and the grading coefficients
1682

Question Demerit points Coefficients

‘‘Use-by’’ date
Always or often Sometimes or seldom Never Not applicable
1 How often do you look at the ‘‘use-by’’ date when buying foods such as 5 0 0.5 1 0
minced meat?
2 What do you do with cold cuts of boiled ham in which the ‘‘use-by’’ date
has expired?
Continue eating it 5 1 0.5 0 0
RØSSVOLL ET AL.

‘‘No difference, the different producers say things

Yes Noa in different ways’’
3 Do you know the difference between the ‘‘use-by’’ and ‘‘best-before’’ labels? 5 0 1

Consumption of high-risk foods

1 or 2 times/wk 3–11 times/yr
or more or more More seldom Never
How often do you do the following:
4 Half-fermented fish (rakfisk) or cured salmon (gravlaks) 50b 1 0.5 0 0
5 Unpasteurized cheese 50b 1 0.5 0 0
6 Raw meet as steak tartare or carpaccio 50b 1 0.5 0 0
7 Sprouts, such as bean sprouts or alfalfa sprouts 50b 1 0.5 0 0

Consumption of undercooked meat

Always or often Sometimes or seldom Never Not applicable
8 Do you taste raw or partially raw minced meat while cooking? 50 1 0.5 0 0
Totally or partly Neither or partly
agree disagree Totally disagree Not applicable
To what extent do you agree with the following:
9 I prefer hamburger to be pink in the middle (not completely done). 50 1 0.5 0 0
10 I prefer chicken fillet to be pink in the middle (not completely done). 50 1 0.5 0 0

Food storing practices

Always or often Sometimes or seldom Never Not applicable
11 Dinner stands for more than 2 h on the kitchen table (at room temperature). 20 1 0.5 0 0
12 Leftovers gets reheated for the second time (e.g., 2 days in a row), for a 40 1 0.5 0 0
total of three heating periods.
13 When I make large portions of food to be served later, I make sure the food 50 0 0.5 1 0
is cooled rapidly (e.g., by placing the food in cold water or outside in the
winter).
14 When reheating leftovers from a food such as a casserole, I make sure that 50 0 0.5 1 0
it is heated to the boiling point.
15 How often do you thaw frozen meat or meat products at room temperature? 5 1 0.5 0 0
J. Food Prot., Vol. 75, No. 9
TABLE 5. Continued
Question Demerit points Coefficients

Refrigerator temperatures
Yesc ,4uC .4uC No
J. Food Prot., Vol. 75, No. 9

16 Do you know the temperature in your refrigerator? 10 0 1 1

Salad practices
Totally or partly Neither or partly
agree disagree Totally disagree Not applicable
To what extent do you agree with the following:
17 I wash lettuce before I eat or serve it. 50d 0 0.5 1 0
18 I parboil (blanch) raw vegetables such as sugar peas and sprouts before 50d 0 0.5 1 0
I eat or serve them.

Cross-contamination practices
Using hot water,
Using cold water or soap, dish brush,
kitchen cloth or dishwasher None of these
19 How do you usually clean your cutting board after cutting raw meat or chicken?e 10 1 0 0
20 How do you usually clean the knife?e 10 1 0 0
Always or often Sometimes or seldom Never Not applicable
After cutting raw meat, which of the following do you do with the knife?
21 Continuing using the knife as it is 10 1 0.5 0 0

What do you use your kitchen cloth for?f

22 Wiping up thawing water from meat products 10 1 0.5 0 0
23 Cleaning hands 10 1 0.5 0 0
24 Cleaning children’s face and hands 10 1 0.5 0 0
25 Wiping up earth spilled from potted plants 10 1 0.5 0 0

Hygiene practices
Always or often Sometimes or seldom Never Not applicable
How often do you do the following:
26 Wash hands before preparing food 10 0 0.5 1 0
27 Use soap when washing hands during food preparation 10 0 0.5 1 0
HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS

28 Wash hands after blowing the nose 10 0 0.5 1 0

29 Wash hands after playing with a pet 10 0 0.5 1 0
30 Wash hands immediately after handling raw meat 15g 0 0.5 1 0
31 Wash hands after toilet visits 15h 0 0.5 1 0
1683
 1684

TABLE 5. Continued
RØSSVOLL ET AL.

Question Demerit points Coefficients

Did not clean them,

used a towel or
kitchen cloth, or Used a paper towel or Used soap and hot Do not remember or
rinsed in cold water rinsed in hot water water none of these
32 The last time you handled raw meat, how did you clean your hands? 15 1 0.5 0 0
Yes Sometimes Never Not applicable
33 If you are vomiting and/or have diarrhea, do you avoid preparing food 50h 0 0.5 1 0
for others?
a
‘‘Use-by’’ date was considered the same as the ‘‘best-before’’ date. Ten percent of the respondents answered ‘‘no’’ to Q3, but because of the possibility to misinterpret, this alternative was excluded
from the analysis and not given any demerit points.
b
Not included in the original demerit points of Worsfold and Griffith. Consumption of these high-risk foods describes a potential direct way of infection and was therefore given a high score.
c
Respondents that answered ‘‘yes’’ to Q16 and specified a temperature in the recommended interval of 0 to 4uC were not awarded any demerit points. Respondents who specified a temperature
higher than recommended were awarded 10 demerit points.
d
According to Worsfold and Griffith (109, 110), 2 demerit points were given when parsley, leeks, and lettuce were not washed. Because of recent national and international outbreaks associated with
imported lettuce, sprouts, and sugar peas, not washing lettuce or parboiling sprouts or sugar peas was given 50 demerit points owing to the potential for direct infection.
e
Multiple responses for this question were permitted, but only the responses ‘‘using cold water’’ and ‘‘using the kitchen cloth’’ were awarded demerit points.
f
Respondents who stated they changed their kitchen cloth every day were not awarded any demerit points.
g
Washing hands after handling raw meat is an important step according to the HACCP analyses, so omission of this practice was given a penalty of 15 demerit points, instead of the original 10 points
used by Worsfold and Griffith. (Fried chicken cut on a dirty board was given 15 points by Worsfold and Griffith (110).) Omission of hand washing after toilet visits also was given 15 demerit points
because of the great risk of cross-contamination.
h
Not included in the original demerit points of Worsfold and Griffith but an important point according to the HACCP analyses. Preparing foods for others during your own illness represents a risk of
contaminating the food and subsequent spread of the disease and was therefore given a high number of demerit points.
J. Food Prot., Vol. 75, No. 9
J. Food Prot., Vol. 75, No. 9 HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS
handling violations reported by the survey participants also
could be regarded as parallel to the HACCP system.
According to the Codex Alimentarius (3), likely occurrences
of hazards and the severity of their adverse health effects
should be included wherever possible when conducting a
hazard analysis with the HACCP system. The frequencies
constitute a measure equivalent to the likelihood of
occurrence of hazards, and the demerit points, as used in
this study, are equivalent in the same manner to the severity
of resulting health effects.
The system of awarding risk-based demerit points for
incorrect food handling practices resulted in a ranking of the
most important practices; this ranking differed completely
from the traditional ‘‘right’’ or ‘‘wrong’’ ranking (Table 6).
The traditional approach used in consumer food safety
surveys with ‘‘right’’ and ‘‘wrong’’ food handling actions
describes violations that are frequently performed by many
consumers. However, these common practices are not
necessarily the most crucial practices with regard to the
risk of acquiring foodborne disease. Only 5 of the 10 food
handling violations most frequently performed were among
the 10 practices associated with most risk. Of the 10 most
common food handling violations, 6 were associated with
time and temperature, i.e., storing food at too high a
temperature for too long. Two questions dealt with eating
high-risk foods, one with cross-contamination, and one with
knowledge about shelf-life labeling. When analyzing the
results with risk-based grading, the 10 questions with the
highest scores and therefore were associated with the
highest risk dealt with different aspects of lack of heat
treatment. Four questions covered the handling of leftovers,
and six were about eating high-risk products (Table 6). The
products in the those six questions were defined as high risk
because of their missing or inadequate heat treatment.
Consumption of raw meat dishes or incompletely cooked
chicken fillet and preparation of food for others during own
illness, which are high-risk practices, were not among the 10
most important high-risk practices in this analysis because
so few people reported performing them.
Use of the risk-based grading system for survey
responses provides a much more realistic picture of risks
associated with domestic food handling practices. This
approach can highlight important violations, such failing to
rapidly cool down large portions of food, and identify minor
errors that are performed by most people but do not
constitute significant risk, e.g., thawing meat or meat
products at room temperature (56, 76, 109). Domestic food
handling and food safety practices are complex actions that
cannot be simplified into the two categories ‘‘right’’ or
‘‘wrong.’’ The minimum and maximum demerit point score
for our survey participants were 2.5 and 530, respectively,
which illustrates the large variation in knowledge of food
safety and safe food handling practices among consumers.
The total risk score calculated for all the survey participants
followed an approximately normal distribution. The average
score was 165 demerit points. The 5th percentile of the
distribution had a very low total risk score of less than 50
demerit points, and the 95th percentile had a total risk score
of more than 310 demerit points.
1685
The questions we created overlapped to a large extent
with the 28 valid and reliable questions identified by
Kendall et al. (66). These researchers used five pathogen
control factors to select food handling questions. The
questions were tested for reliability and validity by
comparing questionnaire responses with observed behaviors
or behaviors reported during an interview (66). The five
pathogen control factors (practicing personal hygiene,
cooking foods adequately, avoiding cross-contamination,
keeping foods at safe temperatures, and avoiding foods from
unsafe sources) (66, 84) coincide with the WHO recom-
mended five keys to safer food (107), which was one of the
methods used to create questions in the present study. The
four HACCP plans in the present study were designed to
encompass normally prepared meals and the most important
food pathogens on a specific national level.
Four of the 28 questions included in the study by Kendall
et al. (66) were related to high-risk foods for pregnant women,
who were not part of the participant group in the present study.
Three questions referred to raw eggs, which do not constitute
any significant risk in Norway with regard to Salmonella (52).
Thus, 21 of the questions from Kendall et al. (66) were
included in or group of 33 food safety questions, although
they were modified slightly and written in Norwegian. Use of
the HACCP methodology when designing the food safety
survey resulted in reliable and relevant questions that covered
the various steps in domestic food production. The HACCP-
based approach also provided a useful overview and
documentation of all microbiological hazards, some of which
could be covered by a survey questionnaire and others that
could be explored with other approaches, such as observation,
interviews, or temperature logs.
One limitation of the present study was the use of self-
reported behavior, which must be taken into consideration
when interpreting the data. Overreporting of ‘‘good’’ and
socially desirable behavior (the social desirability bias) is a
well-known phenomenon in food safety surveys (35, 38, 75,
96). Nevertheless, Kendall et al. (66) found a close
correlation between self-reported behavior from a question-
naire and behavior observed in the kitchen or reported during
an interview. The 21 food safety questions also used in the
present survey met the validity criterion at $70% agreement
between self-reported and observed behavior (66). Tests were
conducted to determine the stability of these questions over
time, and these 21 questions met the reliability criteria of
$75% agreement (66). The use of a predesigned food
handling violation score produces a highly reliable outcome
because of its replicability (96). However, the results of a self-
reported survey consist of personal accounts of actions,
which may or may not reflect actual behaviors (96) and will
most likely be a description of a ‘‘best case’’ result. Another
limitation of this study is that qualitative methods were used.
Quantitative risk assessment could have been used instead of
the HACCP approach in this study or as an extension or
validation of that approach (54, 55). Because of large
uncertainties in both the dose-response and exposure for
many of the microbiological hazards in question and the
difficulties of encompassing all of the practices that occur in a
domestic kitchen, the outcome would be primarily a quali-
1686 RØSSVOLL ET AL. J. Food Prot., Vol. 75, No. 9

TABLE 6. Questions ranked with and without risk-based grading

Rank with Mean score with Rank without
Short version of question Full question risk-based grading risk-based grading risk-based grading

Cooling of large portions When I make large portions of food to 1 17.70 6

be served later, I make sure that it cools
down rapidly.
Reheat leftovers Leftovers get reheated for the second time 2 14.38 5
(e.g., 2 days in a row).
Sprouts How often do you eat sprouts such as bean 3 12.95 10
sprouts or alfalfa sprouts?
Pink hamburgers To what extent do you agree: I prefer 4 11.58 11
hamburger pink in the middle (not
completely done).
Unpasteurized cheese How often do you eat unpasteurized 5 11.25 12
cheese?
Parboil raw vegetables To what extent do you agree: I parboil raw 6 10.41 4
vegetables such as garden peas and
sprouts before I eat or serve them.
Taste raw meat Do you taste raw or partially raw minced 7 9.33 14
meat while cooking?
Reheat leftovers When reheating leftovers from food such 8 8.73 15
as a casserole, I make sure that it is
heated to the boiling point.
Raw fish How often do you eat half-fermented fish 9 7.86 19
or cured salmon?
Food kept at room To what extent do you agree: Dinner 10 6.88 7
temperature stands more than 2 h on the kitchen
table (at room temperature).
Temperature in fridge Do you know the temperature in your 11 6.03 2
refrigerator?
Pink chicken fillet To what extent do you agree: I prefer 12 5.39 23
chicken fillet pink in the middle
(not completely done).
Raw meat How often do you eat raw meet as steak 13 4.21 26
tartare or carpaccio?
Cloth cross-contamination Do you use your kitchen cloth to wipe up 14 3.39 8
thawing water from meat products?
Own illness If you are vomiting and/or have diarrhea, 15 3.35 28
do you avoid preparing food for others?
Thaw at room temperature How often do you thaw frozen meat or 16 3.10 1
meat products at room temperature?
Wash lettuce To what extent do you agree: I wash 17 2.88 30
lettuce before I eat or serve it.
Expired ‘‘use-by’’ date Do you continue eating cold cuts of boiled 18 2.42 3
ham for which the ‘‘use-by’’ date has
expired?
Cloth cross-contamination Do you use your kitchen cloth to clean 19 2.30 13
your hands?
Hand washing after raw The last time you handled raw meat, 20 1.76 21
meat how did you clean your hands?
Soap when washing hands How often do you use soap when washing 21 1.66 16
hands during food preparation?
Hand washing after nose How often do you wash hands after 22 1.64 17
blowing the nose?
Knife use after raw meat After cutting raw meat, do you 23 1.59 18
continue using the knife as it is?
Hand washing after raw How often do you wash hands 24 1.44 24
meat immediately after handling raw meat?
Knowledge of shelf-life Do you know the difference between the 25 1.40 9
labels ‘‘use-by’’ and ‘‘best-before’’ labels?
Hand washing after pet How often do you wash hands after 26 1.28 20
playing with a pet?
J. Food Prot., Vol. 75, No. 9 HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS 1687

TABLE 6. Continued
Rank with Mean score with Rank without
Short version of question Full question risk-based grading risk-based grading risk-based grading

Cleaning of cutting board How do you usually clean your cutting 27 1.17 22
board after cutting raw meat or
chicken?
Cloth cross-contamination Do you use your kitchen cloth to wipe 28 1.10 25
up earth spilled from potted plants?
Cleaning of knife How do you usually clean the knife? 29 0.68 27
Cloth cross-contamination Do you use your kitchen cloth to clean 30 0.46 32
children’s face and hands?
Hand washing before food How often do you wash hands before 31 0.36 31
preparing food?
‘‘Use-by’’ date How often do you look at the ‘‘use-by’’ 32 0.30 29
date when buying foods such as minced
meat?
Hand washing after toilet How often do you wash hands after 33 0.21 33
toilet visits?

tative assessment. A better approach was to use qualitative
terms such as low, medium, and high risk, and the HACCP
approach was preferred. Although some regard qualitative
terms as inexact and less scientific, they may be more
intelligible to consumers, who are at the center of the issue
(49).
The HACCP methodology was used in the present
study to develop relevant and adequate questions concern-
ing consumer food handling in the domestic environment.
The HACCP system is a useful tool in domestic food
preparation because it highlights the most important steps
(CCPs and OPRPs) that consumers should be aware of in
everyday home cooking. A HACCP plan is designed
around prerequisite programs (3), which constitute an
essential foundation for and provides continuous support to
the HACCP-based systems (5, 104). Prerequisite programs
often are general descriptions that only indirectly deal with
food safety issues. Such programs may include issues other
than food safety and describe, for instance, production
facilities and equipment, control of raw materials, sanita-
tion, pest control, storage and distribution, traceability, and
recall (5, 6, 104). OPRPs are identified in the HACCP plan
as vital for controlling the likelihood of introducing food
safety hazards to the product (5). Good hygiene practices
with standard operation procedures for sanitation are
examples of prerequisite programs (3, 5, 6). These
principles cannot easily be extrapolated to the domestic
environment, but some equivalents can be assigned to
domestic food handling practices. General good hygiene
practices based on knowledge of food safety and hygiene,
use of procedures in the domestic kitchen that are
equivalent to standard operation procedures for sanitation,
and appropriate facilities and equipment in a domestic
kitchen are the foundations for the domestic HACCP-based
approach. Application of all 12 steps of the HACCP system
as described in the Codex Alimentarius (3) is not practical
or possible to achieve in the domestic kitchen. However,
informing consumers about the CCPs is both feasible and
of the utmost importance if we are to ensure safe food from
farm to fork.
Adapting the method of risk-based grading to food safety
surveys makes it possible to grade the various responses for
low-risk to high-risk practices with a standardized and
validated risk-based scoring system derived from epidemio-
logical data (29, 49, 109, 110). The risk-based grading system
can be used with many consumer groups, but the demerit
points must be adjusted to the group in question. The demerit
points used in the present study are meant for the general
population. When using this system in surveys of vulnerable
groups such as pregnant women, the demerit points should be
adjusted to encompass their enhanced susceptibility and
subsequent higher risk of acquiring foodborne diseases. If the
national situation regarding zoonoses and other pathogens
were to change in the future, e.g., if Salmonella were to
become established in the Norwegian domestic animal
population, the demerit points would need to be changed
accordingly to reflect the increased risk. Because this study
was conducted in Norway and adapted to Norwegian
conditions, adaptations must be made to questions and
measurements for relevant use in other countries. However,
the principle of using risk-based grading to determine actual
risk from consumers’ behaviors is valid.
Use of this survey method will help define which food
handling practices represent a risk of contracting foodborne
disease and which are of minor importance. These results
can thus contribute to a broader overview and a better
understanding of domestic food handling practices. The
outcome of surveys founded on an HACCP approach with
risk-based grading is of great value for the design of targeted
information and educational activities by food authorities, the
food industry, and educational establishments.
ACKNOWLEDGMENTS
This work was supported by the Norwegian Research Council, the
Fund for Research levy on agricultural products, and research funds from
the Norwegian Agricultural Authority.
1688 RØSSVOLL ET AL.
REFERENCES
1. Altekruse, S. F., D. A. Street, S. B. Fein, and A. S. Levy. 1996.
Consumer knowledge of foodborne microbial hazards and food-
handling practices. J. Food Prot. 59:287–294.
2. Angelillo, I. F., M. R. Foresta, C. Scozzafava, and M. Pavia. 2001.
Consumers and foodborne diseases: knowledge, attitudes and
reported behavior in one region of Italy. Int. J. Food Microbiol.
64:161–166.
3. Anonymous. 2003. Recommended international code of practice—
general principles of food hygiene including annex on hazard
analysis and critical control point (HACCP) system and guidelines
for its application. CAC/RCP 1-1969, Rev. 4-2003. Codex
Alimentarius Commission. Available at: http://www.codexalimentarius.
net/download/standards/23/CXP_001e.pdf. Accessed 20 October 2011.
4. Anonymous. 2003. Recent trends in listeriosis in the UK. Advisory
Committee on the Microbiological Safety of Food. Available
at: www.food.gov.uk/multimedia/pdfs/acm667.pdf. Accessed 28
March 2012.
5. Anonymous. 2005. Food safety management systems—require-
ments for any organization in the food chain. ISO 22000:2005.
International Organization for Standardization, Geneva.
6. Anonymous. 2008. Prerequisite programmes on food safety for food
manufacturing. PAS 220:2008. British Standards Institution,
London.
7. Anonymous. 2010. Norway—trends and sources of zoonoses and
zoonotic agents in humans, foodstuffs, animals and feedingstuffs
including information on foodborne outbreaks, antimicrobial
resistance in zoonotic agents and some pathogenic microbiological
agents in 2009. Zoonoses monitoring. European Food Safety
Authority, Parma, Italy.
8. Anonymous. 2011. BAMA annual report, 2010. BAMA, Oslo.
9. Anonymous. 2011. Gemeinsame Pressemitteilung von Bundesin-
stitut für Risikobewertung, Bundesamt für Verbraucherschutz und
Lebensmittelsicherheit und Robert Koch-Institut, EHEC O104:
H4-Ausbruchgeschehen in Deutschland aufgeklärt: Auflöser
waren Sprossen von aus Ägypten importierten Bockshornkleesa-
men. Available at: http://www.bfr.bund.de/de/presseinformation/
2011/21/ehec_o104_h4_ausbruchsgeschehen_in_deutschland_
aufgeklaert__ausloeser_waren_sprossen_von_aus_aegypten_importierten_
bockshornkleesamen-82843.html. Accessed 30 August 2011.
10. Anonymous. 2011. Home hygiene & health. The portal dedicated
exclusively to the science and practice of home hygiene.
International Scientific Forum on Home Hygiene. Available at:
http://www.ifh-homehygiene.org/IntegratedCRD.nsf/IFH_Home?
OpenForm. Accessed 21 November 2011.
11. Anonymous. 2011. National outbreak of shigellosis. Norwegian
Institute of Public Health. Available at: http://www.fhi.no/eway/
default.aspx?pid~233&trg~MainLeft_5565&MainArea_5661~
5565:0:15,4640:1:0:0:::0:0&MainLeft_5565~5544:92576::1:
5569:1:::0:0. Accessed 23 March 2012.
12. Anonymous. 2011. About the Norwegian Food Safety Authority.
Available at: http://www.mattilsynet.no/english/about. Accessed 2
November 2011.
13. Anonymous. 2011. Overview. Fruits and vegetables 2000–2010. In
G. Rebnes (ed.), Summary of trends in consumption of fruits and
vegetables in Norway. Norwegian Fruit and Vegetables Marketing
Board, Oslo.
14. Anonymous. 2011. Smittevernboka—Salmonellose. Norwegian
Institute of Public Health. Available at: http://www.fhi.no/eway/
default.aspx?pid~233&trg~MainLeft_6039&MainArea_5661~
6039:0:15,5078:1:0:0:::0:0&MainLeft_6039~6041:82847::1:6043:
106:::0:0. Accessed 22 December 2011.
15. Anonymous. 2011. Status in Norwegian meat and egg production
2010. Animalia. Norwegian Meat and Poultry Research Centre, Oslo.
16. Anonymous. 2012. European Union summary report on trends and
sources of zoonoses, zoonotic agents and food-borne outbreaks in
2010. EFSA J. 10(3):2597.
17. Anonymous. 2012. Information about outbreaks of contagious
diseases in Norway. Norwegian Institute of Public Health. Available
at: www.utbrudd.no. Accessed 28 March 2012.
J. Food Prot., Vol. 75, No. 9
18. Anonymous. 2012. MSIS. Norwegian Surveillance System for
Communicable Diseases. Norwegian Institute of Public Health.
Available at: http://www.msis.no/. Accessed 2 February 2012.
19. Arnesen, L. P. S., A. Fagerlund, and P. E. Granum. 2008. From soil
to gut: Bacillus cereus and its food poisoning toxins. FEMS
Microbiol. Rev. 32:579–606.
20. Ballangrud, P., and A. C. Sellgren. 1983. Human botulisme type E
etter konsum av rakfisk. Tidsskr. Nor. Lægeforen. 103:1525–
1527.
21. Beard, T. D. 1991. HACCP and the home—the need for consumer
education. Food Technol. 45:123–124.
22. Beatty, M. E., G. Shevick, K. Shupe-Ricksecker, E. Bannister, A.
Tulu, K. Lancaster, N. Alexander, D. E. Zellner, E. Lyszkowicz, and
C. R. Braden. 2009. Large Salmonella Enteritidis outbreak with
prolonged transmission attributed to an infected food handler,
Texas, 2002. Epidemiol. Infect. 137:417–427.
23. Bergsma, N. J., A. R. H. Fischer, E. D. Van Asselt, M. H.
Zwietering, and A. E. I. De Jong. 2007. Consumer food preparation
and its implication for survival of Campylobacter jejuni on chicken.
Br. Food J. 109:548–561.
24. Beuchat, L., F. F. Busta, N. J. Farber, E. H. Garrett, L. J. Harris,
M. E. Parish, and T. V. Suslow. 2003. Analysis and evaluation of
preventive control measures for the control and reduction/elimina-
tion of microbial hazards on fresh and fresh-cut produce. Available
at: http://www.fda.gov/Food/ScienceResearch/ResearchAreas/
SafePracticesforFoodProcesses/ucm091016.htm#current. Accessed
27 February 2012.
25. Bloomfield, S. F., and E. A. Scott. 2003. Developing an effective
policy for home hygiene: a risk-based approach. Int. J. Environ.
Health Res. 13:S57–S66.
26. Brennan, M., M. McCarthy, and C. Ritson. 2007. Why do
consumers deviate from best microbiological food safety advice?
An examination of ‘‘high-risk’’ consumers on the island of Ireland.
Appetite 49:405–418.
27. Brewer, M. S., and M. Rojas. 2008. Consumer attitudes toward
issues in food safety. J. Food Saf. 28:1–22.
28. Bruun, T., G. Sorensen, L. P. Forshell, T. Jensen, K. Nygard, G.
Kapperud, B. A. Lindstedt, T. Berglund, A. Wingstrand, R. F.
Petersen, L. Muller, C. Kjelso, S. Ivarsson, M. Hjertqvist, S.
Lofdahl, and S. Ethelberg. 2009. An outbreak of Salmonella
Typhimurium infections in Denmark, Norway and Sweden, 2008.
EuroSurveillance 14:6.
29. Bryan, F. L. 1982. Foodborne disease risk assessment of foodservice
establishments in a community. J. Food Prot. 45:93–100.
30. Bugge, A. B. 2005. Dinner—a sociological analysis of Norwegian
dinner practice. Ph.D. thesis. Norwegian University of Science and
Technology, Trondheim.
31. Byrd-Bredbenner, C., J. Maurer, V. Wheatley, D. Schaffner, C.
Bruhn, and L. Blalock. 2007. Food safety self-reported behaviors
and cognitions of young adults: results of a national study. J. Food
Prot. 70:1917–1926.
32. Cates, S. C., R. A. Morales, S. A. Karns, L. A. Jaykus, K. M. Kosa,
T. Teneyck, C. M. Moore, and P. Cowen. 2006. Consumer
knowledge, storage, and handling practices regarding Listeria in
frankfurters and deli meats: results of a Web-based survey. J. Food
Prot. 69:1630–1639.
33. Cody, M. M., and M. Hogue. 2003. Results of the Home Food
Safety—It’s in Your Hands 2002 survey: comparisons to the 1999
benchmark survey and Health People 2010 food safety behaviors
objective. J. Am. Diet. Assoc. 103:1115–1125.
34. Cooley, M., D. Carychao, L. Crawford-Miksza, M. T. Jay, C.
Myers, C. Rose, C. Keys, J. Farrar, and R. E. Mandrell. 2007.
Incidence and tracking of Escherichia coli O157:H7 in a major
produce production region in California. PLoS One 2(11):e1159.
35. Curtis, V., S. Cousens, T. Mertens, E. Traore, B. Kanki, and I. Diallo.
1993. Structured observations of hygiene behaviors in Burkina-
Faso—validity, variability, and utility. Bull. W.H.O. 71:23–32.
36. Denny, J., J. Threlfall, J. Takkinen, S. Löfdahl, T. Westrell, C.
Varela, B. Adak, N. Boxall, S. Ethelberg, M. Torpdahl, M.
Straetemans, and W. van Pelt. 2007. Multinational Salmonella
J. Food Prot., Vol. 75, No. 9 HACCP AND RISK-BASED GRADING IN CONSUMER FOOD SAFETY SURVEYS
Paratyphi B variant Java (Salmonella Java) outbreak, August–
December 2007. EuroSurveillance 12:3332.
37. De Roever, C. 1998. Microbiological safety evaluations and
recommendations on fresh produce. Food Control 9:321–347.
38. Dharod, J. M., R. Perez-Escamilla, S. Paciello, A. Bermudez-Millan,
K. Venkitanarayanan, and G. Damio. 2007. Comparison between
self-reported and observed food handling behaviors among Latinas.
J. Food Prot. 70:1927–1932.
39. Emberland, K., S. Ethelberg, M. Kuusi, L. Vold, L. Jensvoll,
B. Lindstedt, K. Nygard, C. Kjelsø, M. Torpdahl, G. Sørensen,
T. Jensen, S. Lukinmaa, T. Niskanen, and G. Kapperud. 2007.
Outbreak of Salmonella Weltevreden infections in Norway, Den-
mark and Finland associated with alfalfa sprouts, July–October
2007. EuroSurveillance 12:3321.
40. Ericsson, H., A. Eklow, M. L. Danielsson Tham, S. Loncarevic,
L. O. Mentzing, I. Persson, H. Unnerstad, and W. Tham. 1997. An
outbreak of listeriosis suspected to have been caused by rainbow
trout. J. Clin. Microbiol. 35:2904–2907.
41. Ethelberg, S., M. Lisby, B. Bottiger, A. C. Schultz, A. Villif,
T. Jensen, K. E. Olsen, F. Scheutz, C. Kjelso, and L. Muller. 2010.
Outbreaks of gastroenteritis linked to lettuce, Denmark, January
2010. EuroSurveillance 15:2–4.
42. Ethelberg, S., G. Sorensen, B. Kristensen, K. Christensen,
L. Krusell, A. Hempel-Jorgensen, A. Perge, and E. M. Nielsen.
2007. Outbreak with multi-resistant Salmonella Typhimurium
DT104 linked to carpaccio, Denmark, 2005. Epidemiol. Infect.
135:900–907.
43. European Free Trade Association. 1995. EFTA Surveillance
Authority decision of 19 June 1995 approving the plan presented
by Norway to monitor and control Salmonella in poultry. Decision
68/95/COL. European Free Trade Association, Geneva.
44. Fischer, A. R. H., L. J. Frewer, and M. J. Nauta. 2006. Toward
improving food safety in the domestic environment: a multi-item
Rasch scale for the measurement of the safety efficacy of domestic
food-handling practices. Risk Anal. 26:1323–1338.
45. Gilbert, S. E., R. Whyte, S. M. Paulin, R. J. Lake, and P. van der
Logt. 2007. Survey of domestic food handling practices in New
Zealand. Int. J. Food Microbiol. 117:306–311.
46. Greenland, K., C. de Jager, A. Heuvelink, K. van der Zwaluw,
M. Heck, D. Notermans, W. van Pelt, and I. Friesema. 2009.
Nationwide outbreak of STEC O157 infection in the Netherlands,
December 2008–January 2009: continuous risk of consuming raw
beef products. EuroSurveillance 14:2–5.
47. Greig, J. D., and A. Ravel. 2009. Analysis of foodborne outbreak
data reported internationally for source attribution. Int. J. Food
Microbiol. 130:77–87.
48. Greig, J. D., E. C. D. Todd, C. A. Bartleson, and B. S. Michaels.
2007. Outbreaks where food workers have been implicated in the
spread of foodborne disease. Part 1. Description of the problem,
methods, and agents involved. J. Food Prot. 70:1752–1761.
49. Griffith, C., D. Worsfold, and R. Mitchell. 1998. Food preparation,
risk communication and the consumer. Food Control 9:225–232.
50. Griffith, C. J., and D. Worsfold. 1994. Application of HACCP to
food preparation practices in domestic kitchens. Food Control 5:
200–204.
51. Heier, B. T., K. Nygard, G. Kapperud, B. A. Lindstedt, G. S.
Johannessen, and H. Blekkan. 2009. Shigella sonnei infections in
Norway associated with sugar peas, May–June 2009. EuroSurveil-
lance 14(24):pii~19243.
52. Hofshagen, M., B. T. Heier, and K. Hauge. 2008. Norway 2008.
Trends and sources of zoonoses and zoonotic agents in humans,
foodstuffs, animals and feedingstuffs. Report on trends and sources
of zoonoses. European Food Safety Authority, Parma, Italy.
53. Hofshagen, M., B. T. Heier, and K. Hauge. 2010. Norway 2010.
Trends and sources of zoonoses and zoonotic agents in humans,
foodstuffs, animals and feedingstuffs. Report on trends and sources
of zoonoses. European Food Safety Authority, Parma, Italy.
54. Hoornstra, E., M. D. Northolt, S. Notermans, and A. W. Barendsz.
2001. The use of quantitative risk assessment in HACCP. Food
Control 12:229–234.
1689
55. Hoornstra, E., and S. Notermans. 2001. Quantitative microbiological
risk assessment. Int. J. Food Microbiol. 66:21–29.
56. Ingham, S. C., R. K. Wadhera, M. A. Fanslau, and D. R. Buege.
2005. Growth of Salmonella serovars, Escherichia coli O157:H7,
and Staphylococcus aureus during thawing of whole chicken and
retail ground beef portions at 22 and 30uC. J. Food Prot. 68:1457–
1461.
57. Isakbaeva, E., B. A. Lindstedt, B. Schimmer, T. Vardund, T. L.
Stavnes, K. Hauge, B. Gondrosen, H. Blystad, H. Kløvstad, P.
Aavitsland, K. Nygard, and G. Kapperud. 2005. Salmonella
Typhimurium DT104 outbreak linked to imported minced beef,
Norway, October–November 2005. EuroSurveillance 10:E051110.1.
58. Jacobsen, E., and R. Lavik. 2011. Domestic kitchen habits and food
hygiene. Fagrapport 3, 2011. National Institute for Consumer
Research, Oslo.
59. Jay, L. S., D. Comar, and L. D. Govenlock. 1999. A national
Australian food safety telephone survey. J. Food Prot. 62:921–928.
60. Jensen, T., D. Jacobsen, E. von der Lippe, and M. Yndestad. 1998.
Botulism after intake of half-fermented fish. Tidsskr. Nor.
Lægeforen. 118:4366–4367.
61. Jevsnik, M., V. Hlebec, and P. Raspor. 2008. Consumers’ awareness
of food safety from shopping to eating. Food Control 19:737–745.
62. Jones, M. V. 1998. Application of HACCP to identify hygiene risks
in the home. Int. Biodeterior. Biodegrad. 41:191–199.
63. Jore, S., P. Hopp, M. Hofshagen, B. Bergsjø, T. Bruheim, M. Falk,
O. Eikenes, and H. Glosvik. 2011. The surveillance and control
programme for Salmonella in live animals, eggs and meat in
Norway. In S. Sviland and H. Hellberg (ed.), Surveillance and
control programmes for terrestrial and aquatic animals in Norway.
Annual report 2010. National Veterinary Institute, Oslo.
64. Kapperud, G. 2008. Næringsmiddelbårne infeksjoner og intoksi-
kasjoner: forekomst og betydning, p. 27–47. In P. E. Granum (ed.),
Matforgiftning, Nærinsmiddelbårne infeksjoner og intoksikasjoner.
Norwegian Academic Press, Kristiansand.
65. Kapperud, G., L. Rorvik, V. Hasseltvedt, E. Hoiby, B. Iversen, K.
Staveland, G. Johnsen, J. Leitao, H. Herikstad, and Y. Andersson.
1995. Outbreak of Shigella sonnei infection traced to imported
iceberg lettuce. J. Clin. Microbiol. 33:609–614.
66. Kendall, P. A., A. Elsbernd, K. Sinclair, M. Schroeder, G. Chen, V.
Bergmann, V. N. Hillers, and L. C. Medeiros. 2004. Observation
versus self-report: validation of a consumer food behavior
questionnaire. J. Food Prot. 67:2578–2586.
67. Kennedy, J., V. Jackson, I. S. Blair, D. A. McDowell, C. Cowan,
and D. J. Bolton. 2005. Food safety knowledge of consumers and
the microbiological and temperature status of their refrigerators. J.
Food Prot. 68:1421–1430.
68. Kennedy, J., V. Jackson, C. Cowan, I. Blair, D. McDowell, and D.
Bolton. 2005. Consumer food safety knowledge—segmentation of
Irish home food preparers based on food safety knowledge and
practice. Br. Food J. 107:441–452.
69. Kennedy, J., A. Nolan, S. Gibney, S. O’Brien, M. A. S. McMahon,
K. McKenzie, B. Healy, D. McDowell, S. Fanning, and P. G. Wall.
2011. Determinants of cross-contamination during home food
preparation. Br. Food J. 113:280–297.
70. Kennedy, J., M. Worosz, E. C. Todd, and M. K. Lapinski. 2008.
Segmentation of US consumers based on food safety attitudes. Br.
Food J. 110:691–705.
71. Kjærnes, U. (ed.). 2001. Eating patterns—a day in the lives of
Nordic peoples. National Institute for Consumer Research, Oslo.
72. Kosa, K. M., S. C. Cates, S. Karns, S. L. Godwin, and D. Chambers.
2007. Consumer home refrigeration practices: results of a Web-
based survey. J. Food Prot. 70:1640–1649.
73. Kosa, K. M., S. C. Cates, S. Karns, S. L. Godwin, and D. Chambers.
2007. Consumer knowledge and use of open dates: results of a Web-
based survey. J. Food Prot. 70:1213–1219.
74. Lawless, H. T., and H. Heymann. 1999. Sensory evaluation of food.
Kluwer Academic Publishers, New York.
75. Levy, A. S., C. J. Choiniere, and S. B. Fein. 2008. Practice-specific
risk perceptions and self-reported food safety practices. Risk Anal.
28:749–761.
1690 RØSSVOLL ET AL.
76. Lianou, A., and K. P. Koutsoumanis. 2009. Evaluation of the effect of
defrosting practices of ground beef on the heat tolerance of Listeria
monocytogenes and Salmonella Enteritidis. Meat Sci. 82:461–468.
77. Lindstrom, M., A. Heikinheimo, P. Lahti, and H. Korkeala. 2011.
Novel insights into the epidemiology of Clostridium perfringens
type A food poisoning. Food Microbiol. 28:192–198.
78. Lundstein, A. T. 2011. Market report 2010. Price and market
assessments of Norwegian agricultural products. Norwegian Agri-
cultural Authority, Oslo.
79. Lynch, M. F., R. V. Tauxe, and C. W. Hedberg. 2009. The growing
burden of foodborne outbreaks due to contaminated fresh produce:
risks and opportunities. Epidemiol. Infect. 137:307–315.
80. MacDonald, E., B. T. Heier, T. Stalheim, K. S. Cudjoe, T. Skjerdal,
A. Wester, B. A. Lindstedt, and L. Vold. 2011. Yersinia
enterocolitica O:9 infections associated with bagged salad mix in
Norway, February to April 2011. EuroSurveillance 16:10–12.
81. McCarthy, M., M. Brennan, A. L. Kelly, C. Ritson, M. de Boer, and
N. Thompson. 2007. Who is at risk and what do they know?
Segmenting a population on their food safety knowledge. Food
Qual. Pref. 18:205–217.
82. McLauchlin, J. 1996. The role of the Public Health Laboratory
Service in England and Wales in the investigation of human
listeriosis during the 1980s and 1990s. Food Control 7:235–239.
83. Mead, P. S., L. Slutsker, V. Dietz, L. F. McCaig, J. S. Bresee, C.
Shapiro, P. M. Griffin, and R. V. Tauxe. 1999. Food-related illness
and death in the United States. Emerg. Infect. Dis. 5:607–625.
84. Medeiros, L. C., V. N. Hillers, P. A. Kendall, and A. Mason. 2001.
Food safety education: what should we be teaching to consumers? J.
Nutr. Educ. 33:108–113.
85. Melkild, I., and T. Tollersrud. 2007. Import av levende dyr—har det
hatt konsekvenser for norsk husdyrhelse? Animalia. Meat and
Poultry Research Centre. Available at: http://www.animalia.no/
Artikler/2007/Import-av-levende-dyr-har-det-hatt-konsekvenser-for-
norsk-husdyrhelse/. Accessed 22 December 2011.
86. Michanie, S., F. L. Bryan, P. Alvarez, and A. B. Olivo. 1987.
Critical control points for foods prepared in households in which
babies had salmonellosis. Int. J. Food Microbiol. 5:337–354.
87. Michanie, S., F. L. Bryan, P. Alvarez, A. B. Olivo, and A. Paniagua.
1988. Critical control points for foods prepared in households whose
members had either alleged typhoid fever or diarrhea. Int. J. Food
Microbiol. 7:123–134.
88. Miettinen, M. K., A. Siitonen, P. Heiskanen, H. Haajanen, K. J.
Bjorkroth, and H. J. Korkeala. 1999. Molecular epidemiology of an
outbreak of febrile gastroenteritis caused by Listeria monocytogenes
in cold-smoked rainbow trout. J. Clin. Microbiol. 37:2358–2360.
89. Mitakakis, T. Z., M. I. Sinclair, C. K. Fairley, P. K. Lightbody, K.
Leder, and M. E. Hellard. 2004. Food safety in family homes in
Melbourne, Australia. J. Food Prot. 67:818–822.
90. Mohle-Boetani, J. C., J. Farrar, P. Bradley, J. D. Barak, M. Miller,
R. Mandrell, P. Mead, W. E. Keene, K. Cummings, S. Abbott, S. B.
Werner, and T. Invest. 2009. Salmonella infections associated with
mung bean sprouts: epidemiological and environmental investiga-
tions. Epidemiol. Infect. 137:357–366.
91. Nesbitt, A., S. Majowicz, R. Finley, B. Marshall, F. Pollari, J.
Sargeant, C. Ribble, J. Wilson, and N. Sittler. 2009. High-risk food
consumption and food safety practices in a Canadian community. J.
Food Prot. 72:2575–2586.
92. Nygård, K., J. Lassen, L. Vold, Y. Andersson, I. Fisher, S. Loefdahl,
J. Threlfall, I. Luzzi, T. Peters, M. Hampton, M. Torpdahl, G.
Kapperud, and P. Aavitsland. 2008. Outbreak of Salmonella
J. Food Prot., Vol. 75, No. 9
Thompson infections linked to imported rucola lettuce. Foodborne
Pathog. Dis. 5:165–173.
93. Pavic, S., M. Smoljanovic, D. Lastre, S. Bukovski, A. Arseze,
M. Hadziosmanovic, B. Miokovic, and L. Kozacinski. 2001. How to
apply some basic HACCP principles for domestic preparation of
stuffed peppers after family outbreak due to Clostridium perfrin-
gens. Arch. Lebensmittelhyg. 52:55–59.
94. Rangel, J. M., P. H. Sparling, C. Crowe, P. M. Griffin, and D. L.
Swerdlow. 2005. Epidemiology of Escherichia coli O157:H7
outbreaks, United States, 1982–2002. Emerg. Infect. Dis. 11:603–
609.
95. Ravel, A., J. Greig, C. Tinga, E. Todd, G. Campbell, M. Cassidy,
B. Marshall, and F. Pollari. 2009. Exploring historical Canadian
foodborne outbreak data sets for human illness attribution. J. Food
Prot. 72:1963–1976.
96. Redmond, E. C., and C. J. Griffith. 2003. A comparison and
evaluation of research methods used in consumer food safety
studies. Int. J. Consum. Stud. 27:17–33.
97. Redmond, E. C., and C. J. Griffith. 2003. Consumer food handling in
the home: a review of food safety studies. J. Food Prot. 66:130–161.
98. Rocourt, J., C. Jacquet, and A. Reilly. 2000. Epidemiology of
human listeriosis and seafoods. Int. J. Food Microbiol. 62:197–
209.
99. Ropkins, K., and A. J. Beck. 2000. HACCP in the home: a
framework for improving awareness of hygiene and safe food
handling with respect to chemical risk. Trends Food Sci. Technol.
11:105–114.
100. Sanlier, N. 2009. The knowledge and practice of food safety by
young and adult consumers. Food Control 20:538–542.
101. Schmitt, R., F. L. Bryan, M. Jermini, E. N. Chilufya, A. T.
Hakalima, M. Zyuulu, E. Mfume, C. Mwandwe, E. Mullungushi,
and D. Lubasi. 1997. Hazards and critical control points of food
preparation in homes in which persons had diarrhea in Zambia. J.
Food Prot. 60:161–171.
102. Scott, E., and N. Herbold. 2010. An in-home video study and
questionnaire survey of food preparation, kitchen sanitation, and
hand washing practices. J. Environ. Health 72:8–13.
103. Sivapalasingam, S., C. R. Friedman, L. Cohen, and R. V. Tauxe.
2004. Fresh produce: a growing cause of outbreaks of foodborne
illness in the United States, 1973 through 1997. J. Food Prot. 67:
2342–2353.
104. Stevenson, K. E., and D. T. Bernard (ed.). 1999. HACCP: a
systematic approach to food safety. Food Processors Institute,
Washington, DC.
105. Tauxe, R. V. 2002. Emerging foodborne pathogens. Int. J. Food
Microbiol. 78:31–41.
106. van Asselt, E., A. Fischer, A. E. I. de Jong, M. J. Nauta, and R. de
Jonge. 2009. Cooking practices in the kitchen—observed versus
predicted behavior. Risk Anal. 29:533–540.
107. World Health Organization. 2001. Five keys to safer food. Available
at: http://www.who.int/foodsafety/consumer/5keys/en/. Accessed 30
August 2011.
108. World Health Organization. 2007. Food safety, hazard analysis
critical control point system (HACCP). Available at: http://www.
who.int/foodsafety/fs_management/haccp/en/. Accessed 30 August
2011.
109. Worsfold, D., and C. Griffith. 1995. A generic model for evaluating
consumer food safety behaviour. Food Control 6:357–363.
110. Worsfold, D., and C. J. Griffith. 1997. Assessment of the standard of
consumer food safety behavior. J. Food Prot. 60:399–406.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.