Using a learning from practice style

to help practitioners improve

health impact assessment

Introduction
This bulletin summarises the learning from
practice work programme carried out at
the Health Development Agency (HDA). It
provides information on how the learning
from practice process was undertaken for
specific health impact assessment (HIA) topics
and how effective it has been, and offers
recommendations on how the HIA learning
from practice approach may be improved.
The learning from practice work
programme sought to help practitioners
achieve improvements in the practice and
understanding of HIA through a series of
workshops and summary bulletins. This
bulletin identifies learning points drawn from
this process, based on the individual projects
that made up the work programme and their
evaluation. The programme consisted of:
• Four learning from practice workshops on
four different HIA topics
• Four bulletins derived from the findings of
the workshops, one on each HIA topic
• A report describing how the workshops
were undertaken
• A report describing the evaluation of the
work programme.
The work programme centred around four
pilot learning from practice workshops.
Practitioners from around the country were
invited to these to hear about, and share,
examples of promising HIA practice. Each
workshop had a different theme, and
each was evaluated on the day using an
evaluation form. A series of learning from
practice bulletins was produced, based on
the material provided by the case studies
presented and discussions held at the
workshops. These bulletins covered the four
workshop topics:
• Influencing the decision-making process
through health impact assessment (Taylor
et al., 2003a)
• Evaluating health impact assessment
(Taylor et al., 2003b)
• Deciding if a health impact assessment is
required (screening for HIA) (Taylor et al.,
2003c)
• Addressing inequalities through health
impact assessment (Taylor et al., 2003d).
A report describing how the workshops
were run has also been produced (Gowman
et al., 2003). The work programme ended
with an independent evaluation of the
series of HIA workshops and the resources
produced, and their impact on practitioners
(Crozier, 2004).
Further information on any of these elements
can be found at www.hiagateway.org.uk

1
 Learning points from running the workshops
Identifying participants
Participants were identified and registered
for the workshops by email, via email groups
known to the HDA, up to three months in
advance. The only requirement was that the
participants should have some experience in
undertaking HIA, and preferably (although
not essential) some experience in the subject
area being discussed. Participants were asked
to volunteer case study material suitable for
presentation at the workshops.
Numbers and timing
Thirty-one people attended at least one
workshop, with a range of 9–14 participants
at each workshop (several people attended
more than one workshop). For various
reasons a small number were not able to
attend each workshop, despite having
booked. The workshops ran from 9.30am to
3.30pm, with the first half hour for arrivals
and coffee.
Use of a facilitator
A central component of the learning from
practice method is the role of a facilitator
who can draw out responses from the whole
group and encourage debate and discussion.
At all four workshops an external facilitator
was contracted to undertake this role, and
to provide advice on how to structure the
workshops based on the evidence about how
people learn.
Structure of the day
The aims of the workshops were to:
• Identify examples of projects that
demonstrate aspects of promising practice
• Identify particular elements and processes
that need to be in place to make such
activities successful
• Actively disseminate and share this learning
with those who are in the process of
planning and providing similar provision
• Test the learning from practice workshop
model and assess its benefits for future
application.
Each workshop had a similar structure:
2
• Welcome and introductions
• Rationale for and outline of the day,
including background information on
the theory of the learning from practice
approach, and topic-specific background
• Aims of the day, both HDA’s and
participants’ aims
• Learning from practice case study
presentations, each 10–15 minutes
long, detailing a practitioner’s personal
experience of working on a specific topic
• Group clarification of the definitions/
understanding of the topic being discussed
• Facilitated small-group work to discuss and
draw out learning from the case studies
presented, and from participants’ own
experiences
• Group discussion on the benefits to
practitioners of the topic under discussion
• Group discussion where lessons learned
from the morning were distilled into ‘what
promising practice looks like’
• Action planning by participants about
changes they may make to their practice
• On-the-day evaluation forms completed.
A full explanation of how the workshops
were run, including background and rationale
information, is given by Gowman et al.
(2003).
Evaluation findings (Crozier, 2004) and
our reflections
Identifying participants
The evaluation reported that identifying
participants for the workshop using regional
contacts and email contact lists worked well.
Nearly all participants valued the networking
from these meetings (Crozier, 2004).
Numbers
The evaluation reported that between 12
and 14 participants at each workshop was an
appropriate number to undertake the main
tasks required, particularly when it came
to splitting into small groups. The authors
recalled that lower numbers than this (eg one
workshop with nine people) interrupted the
planned structure of the workshops as it was
not easy to split into two separate groups to
discuss the two case studies presented.
Timing
The authors noted that the timing of the
workshops required consideration. People
could not start too early because of travel
time from distant destinations, and people
tended to ‘drift away’ after 3pm to travel
home. Depending on where participants
come from, it is important to consider
shorter days to allow people travelling time
from and to their home. If the workshops
move beyond the pilot stage, less time will
be needed for evaluation and this may help
shorten the day.
Use of a facilitator
The evaluation results showed that the
majority of participants appreciated the use
of the facilitator, but a small minority did not.
Facilitating the workshops was difficult as
there were a number of strong personalities
present, who sometimes clashed. The topics
covered by the workshops included areas
where there is not full agreement among the
HIA community about how things should be
done, raising the potential for conflict. Having
a professional facilitator who knew how to
deal with such situations was critical. But this
view was not shared by all participants, and
organisers must be aware that a minority
of participants do not want to be led by
a ‘non-expert’ and see this as insulting to
acknowledged leaders in the field (Crozier,
2004).
The authors, however, believe the learning
from practice style is not an expert-driven
process, and is based on practitioners
discussing and sharing their leaning and
experience – so we believe facilitation by non-
experts in the particular subject area is a valid
approach. The use of a facilitator allowed the
organisers to participate more fully and derive
benefit from the day.
Structure of the day
The evaluation reports that the majority of
respondents were satisfied with the structure
and implementation of the workshops, and
felt that the workshop structure and features
within it had enabled their personal objectives
for attending to be realised. The format is
flexible enough to allow small additions or
subtractions to the workshop format to suit
the topic/participants/timing (Crozier, 2004).
Recommendations for running
workshops
• As networking was so valued by
participants, the networking opportunity
should be highlighted in any initial email
advertising.
• We recommend that a suitable number is
12–14 participants, and that a small charge
for attendance at the workshop would
reduce the risk of participants pulling out
at the last minute. Overbooking participant
numbers by 25% ensures that if people do
pull out, adequate numbers remain to run
the workshop successfully.
• We recommend that an individual with
excellent facilitation skills is required for the
smooth running of this type of workshop.
The facilitator can deal with conflict,
produce ideas on workshop format and
structure, and allow the organisers to
participate more fully in the proceedings
– while adhering to the learning from
practice style.
• We recommend that start and end times
should be chosen to suit the travelling
distances of participants, and workshops
should err on the side of being shorter
rather than longer.
• We believe the structure of the workshops
(as described above) is appropriate and
suitable for future workshops.
3
 Learning points about learning from practice
The learning from practice approach used in
the workshops is underpinned by a theoretical
model, the certainty and agreement matrix
(Stacey, 1999). The workshop structure was
developed on the basis of evidence reviews
about changing people’s practice (NHS
Centre for Reviews and Dissemination, 1999;
Ollerearnshaw et al., 2000). Both sources are
discussed below.
Theoretical model
This work is underpinned by a business model
called the certainty and agreement matrix
(Figure 1). The matrix can be used to choose
an appropriate action within a complex
system. It allows managers to choose from
an array of approaches to deal with a specific
issue, or helps them to make a decision
(Stacey, 1999).
The user answers two simple questions and
notes where the answers to those questions
intersect on a matrix. The matrix then
suggests a suitable approach to deal with the
issue, eg guiding a change process. The two
questions are:
Figure 1 What change process is appropriate? (Adapted from Stacey, 1996)
Low Innovation
Creativity
Political
Agreement compromise
Standards
guidance
High
High
4
• Are issues ‘close to agreement’ – what level
of agreement is there in the group/team/
organisation about the desirable outcomes?
For example do we all agree that this is the
outcome we want from dealing with this
issue or from making this decision?
• Are issues ‘close to certainty’ – can cause
and effect linkages be shown? For example
if we do this, then that will follow. This is
usually the case when a similar decision
has been made in the past, allowing
extrapolation from past experience to
predict future outcome with a good level
of certainty.
Previous work at the HDA (Taylor and
Quigley, 2001, 2002) highlighted that
there was ‘moderate certainty’ about how
to do HIA, because of the many different
methods available to undertake an HIA, each
with different stages and using different
information within each stage. Similarly, there
was only moderate agreement about the
desired outcomes for HIA – commentators
have described many different reasons for
doing an HIA, with each HIA attempting to
achieve something unique.
‘Zone of
complexity’
Trial and error
Experiment
Certainty Low
Receiving a moderate answer for both
certainty and agreement placed HIA
within the ‘zone of complexity’ (Figure 1).
Standards and guidance, tools often used
by government to attempt change, are not
suitable in the case of HIA. This allowed
us to consider other ways of working
to change practice within the zone of
complexity that use creativity, innovation,
and trial-and-error to develop practice
within the field of HIA.
A learning from practice approach that
had previously been used in isolated topic
areas at the HDA appeared suitable as
an innovative method for use within HIA
(McCormick, 2002). The basic premise of
this approach is that people with academic/
theoretical experience and those with
practical experience are brought together in
an informal learning environment, such as a
workshop, to share the learning they have
gained through study and/or experience.
It was anticipated that individuals working
towards a common goal would be able to
ask questions, be challenged, support one
another, and so develop better practice and
knowledge.
Experience of how people learn
The learning from practice approach was
also informed by relevant documents relating
to the evidence about how people learn
and make changes to their practice (NHS
Centre for Reviews and Dissemination,
1999; Ollerearnshaw et al., 2000). Principles
outlined in these documents about how
people learn were used in designing the HIA
workshops:
• The more complex the learning (eg HIA),
the more benefit is gained from using
interactive approaches. The learning from
practice workshop approach is interactive
• To change practice you need to avoid
‘instructions from the centre’ and instead
involve the practitioners to
- describe their own experiences
- identify what will work best in their
own circumstances
- discuss and learn from external
guidance and models rather than
importing them wholesale.
Again, the learning from practice approach
allowed these features to be practised
through case studies presented by
practitioners, small-group discussions, and the
presentation of research and theory where
available.
Evaluation findings (Crozier, 2004) and
our reflections
Learning style
The results of the external evaluation clearly
showed that the majority of participants
valued, and responded well to, this type of
learning. They found the workshops useful,
stimulating and engaging, and valued the
facilitated discussion. They appreciated
learning from peers and experts in the field,
and the good use of case studies. There
was a small minority for whom participatory
learning with facilitated discussion was not
their preferred method of learning – they
would have preferred a more ‘teacher–
student, lecture-type’ approach (Crozier,
2004). However, as noted in the reviews of
evidence about how people learn, practice
development work is less likely to be effective
using a didactic teacher–student approach.
Background material and case studies
A small minority of participants reported
in the evaluation that they disliked the
initial background work that informed the
workshops because of its ‘lack of rigour’
(Crozier, 2004). In the pilots, case studies
were invited from participants and delivered
in the workshops. These were the hooks
on which discussion on ‘how to do projects
well’ was hung, and were generally limited to
‘what was done’. A minority of participants
wanted these examples to be ‘systematically
identified best practice’. Some participants did
not believe they had adequate time before
the workshops to read over the background
material and case studies (Crozier, 2004).
We believe that a mix of good and not-so-
good (real-life) projects stimulates discussion,
is more in keeping with the learning from
practice style (rather than being expert-
driven), and allows practitioners to become
more involved in the process by letting them
present their own work – as recommended by
the experience of how people learn.
5
 Participation of experts
The evaluation reported that participants
valued the opportunity to hear and reflect on
other people’s experiences – both from and
between experienced practitioners and other
experts. However, organisers also need to be
aware that a small number of participants
may give more information than they receive.
This is, of course, the purpose of the day – to
share learning – and some people will have
more experience than others. This was not an
issue for all people in this situation, but some
felt they had ‘had all of their learning sucked
out of them’. Others acknowledged that, as
an expert, they would expect this to happen
in most situations, and were happy to share
their knowledge (Crozier, 2004).
We suggest that in some situations it may
be appropriate to consider a master-class
approach, where top experts only are invited
to some learning from practice events, levelling
out knowledge and skills differences. This
would cater for experts who would like to
develop their skills and knowledge further
within this style of learning approach. However,
this suggestion is opposed to the learning
from practice style, where learning is shared
among participants of different knowledge
and skill levels. The best solution may be that
any promotional material should make it clear
that experts’ experience and knowledge will
be drawn on heavily – this would manage
participants’ expectations more carefully, and
those who do not wish to participate in such a
process can choose not to.
Impact on practitioners’ work
The external evaluation also investigated
whether, and in what ways, the workshops
had an impact on practitioners’ work. In
general, respondents struggled to identify
any specific impacts from the workshops.
When pressed, those whose work involved
teaching about HIA mentioned that they had
used the learning to inform the content of
their teaching, or had passed the bulletins
on to students. Some were able to identify
a moment within a workshop that had
helped them come to some new realisation
about the subject; others could not (Crozier,
6
2004). The action planning component of
the workshop elicited good responses from
participants about how they may change their
work in the future (Gowman et al., 2003),
but it is difficult to determine if these changes
were carried forward. Because of the small
amount of impact data collected, we believe
that further evaluation would be useful to
determine if practice has improved, and that
in the meantime these workshops offer a
‘promising approach’ to achieve improvement
in HIA practice. The ability of participants to
recall specific changes to their practice was
limited because the workshops occurred up
to ten months earlier. This time lag was a flaw
in this component of the evaluation.
Recommendations about the learning
from practice approach
• Stacey’s (1999) certainty and agreement
matrix is a suitable model for making
decisions on how to deal with an issue
– in this case, how to improve practice
within HIA.
• Using evidence on how people learn is
highly recommended for designing the
workshop structure.
• Case studies should be sought from
practitioners attending the workshops,
rather than best practice case studies
being provided from the centre. The
case studies are the hooks on which
discussions on how to do projects well are
hung, and allow practitioners to present
their own work.
• Providing participants with background
information and case studies well
before the workshops is essential – we
recommend at least one week before.
• Promotional material must state clearly that
experts’ and/or experienced practitioners’
experience and knowledge will be drawn
on heavily.
• As a small minority do not enjoy the
learning from practice approach,
consideration should be given to providing
other methods of learning (other than the
learning from practice workshop approach)
to cater for such people’s learning
preferences.
Producing the learning from practice bulletins
Bulletin production practitioners. Also, the evidence as to how
people learn recommends against such an
The production of the bulletins was a
approach, pointing instead to practitioners
central component of this work, and they
being heavily involved in developing the
were a planned outcome of the workshops,
material.
although this was not listed as an aim during
the workshops as we wanted participants The production of bulletins is not a required
to focus on the learning from practice style output from such learning from practice
(Gowman et al., 2003). workshops, and it would be up to future
organisers to decide if bulletins were needed
The bulletins drew on the information
for the topic they are covering. We believe
provided during the workshops, particularly
that the bulletins are useful in providing a
from the case studies and group discussions.
lasting reminder for those attending the
They were designed to provide practical
workshops, as well as for the many people
recommendations for improving the
who did not attend. Also, some people
application of HIA within the four topic
prefer a written style of learning, and the
areas covered; to provide practical case study
publications provide for their needs. Funding
examples; and to outline the importance
bodies or managers often like to see a hard
of achieving such practice. Information
copy report, and the bulletins also fulfil
exchanged and presented during the course
this function. The most important function
of the workshops, including the case studies,
of a bulletin, though, is that it provides a
informed the content of the bulletins. All the
background document for future workshops
workshop participants were provided with
on the same topic – potentially allowing this
an opportunity to comment on the content
learning style for a particular topic to be rolled
before publication.
out across the country.
Evaluation findings (Crozier, 2004) and
our reflections Recommendations on producing
bulletins
The evaluation reported that the majority
• Production of a bulletin is recommended
of participants believed the bulletins were
where a workshop has covered a topic
clear, well presented and succinct. However,
for the first time using the learning from
a small number did not believe that the
practice style.
workshops were the appropriate way to
• Bulletins should be produced after the
develop such a publication, and that the HDA
workshop – the case studies, literature and
would have been better going through a
theoretical bases can all be merged with
more formal, expert-driven process. However,
other learning identified at the workshops
we believe that the material produced from
to produce a balanced publication. The
the learning from practice workshops was
involvement of practitioners in this process
also of considerable value, and an expert-
is critical, if you are interested in changing
driven process can discount the views of
practice.

7
 Learning points about evaluation
How we evaluated the workshops and
bulletins
The evaluation took place in three stages, the
first two undertaken by the HDA, and the
third by an independent researcher:
• On-the-day feedback forms were filled in
• Participants were asked to comment on
the draft bulletins
• Participants were followed up, up to 10
months later, and asked to take part in an
interview.
The HDA’s rationale for undertaking the
evaluation was:
• The HDA’s role is to determine what
interventions work. Testing interventions
– including workshops – is crucial for
developing the evidence base in public
health
• Use of the learning from practice approach
as a method for improving practice is
increasing, and it is important that any
learning from pilot work be transferred to
future work.
On-the-day feedback form
The on-the-day feedback form was
distributed to workshop participants at
the end of each of the four workshops.
Participants completed the three-sided
form and handed it back to the facilitator.
The form did not collect the name or any
other identifying details of respondents. The
questionnaire included items on:
• How well participants felt the workshop
had achieved its stated objectives
• Views on the content and presentation of
the workshops
• Whether the learning gained would be
useful in participants’ own work
• Administration and organisation of the
workshop
• Any other comments.
Commenting on the draft bulletins
The bulletins were circulated to all participants
while they were in a draft stage, allowing
time for comments to be incorporated into
the structure or content.
8
Follow-up interviews
Follow-up interviews of participants were
developed and carried out by an independent
contracted researcher in consultation with
the HDA, and a full report is available
(Crozier, 2004). All the workshop participants
were invited to participate in the follow-
up interviews, and those who agreed were
interviewed in November 2003. The stated
aims of the research were to:
• Assess whether the workshop aims and
objectives were achieved
• Assess the acceptability and
appropriateness of the approach used
• Evaluate the intended and unintended
impacts of the workshops
• Identify how the workshop process,
including its evaluation, could be improved
• Determine the usefulness of the resources
created.
The results were presented anonymously
to reduce any potential influence on
participants’ responses. The research was
made up of:
• Analysis of on-the-day evaluation sheets
and rough notes from the workshop
• Semi-structured interviews with workshop
participants and organisers conducted by
telephone or face-to-face
• A structured questionnaire sent by email
to participants overseas or who were
otherwise not contactable by telephone.
Areas covered by the research included:
• Marketing and customer care before,
during and after the workshops
• Motivations and expectations for
attending
• Views on the workshops – structure,
content, level, time for each item
covered
• Impacts of attending the workshop
(intended and unintended)
• Learning from practice bulletins
• Overall assessment of the workshops and
their contribution to HIA.
Evaluation findings (Crozier, 2004) and
our reflections
The on-the-day feedback form drew an
overwhelmingly positive response about the
workshops, whereas the follow-up interviews
identified a small minority of participants who
were unhappy with the learning from practice
approach. There was little sign in any of the
on-the-day feedback forms of the concerns
that some participants (a small minority) later
expressed in the in-depth interviews.
We believe that the on-the-day feedback
forms were inadequate to gain a full
understanding of all participants’ views,
nor did the responses provide a realistic
impression of the value of the workshop.
However, they did accurately gauge the
general thrust of the follow-up interviews,
that the majority of participants valued the
learning from practice style and found the
workshops useful.
The evaluation reported that feedback was
also positive when workshop participants
were given the opportunity to comment on
the learning from practice bulletins, with only
minor changes to the bulletins suggested by
participants.
The follow-up interviews provided much more
detailed information, and drew out the full
spectrum of experiences of the workshop
participants. Overall, the majority of
participants valued the learning from practice
style of teaching, and found the workshops
useful. The interviews provided useful data
for planning future workshops, for gaining
insights into how to improve the learning
from practice approach, and for highlighting
issues that a minority of participants may
experience. However, the evaluation
described the delay between carrying out
the workshops and the eventual follow-
up interviews (up to 10 months) as a
major flaw. This is likely to have inhibited
participants’ ability to recall and respond in
detail to the questions put to them by the
interviewer, and means that the findings of
the follow-up interviews must be treated
with caution.
Recommendations for evaluation
• On-the-day feedback forms should be
short, provide basic feedback and cover
short-term and administrative matters.
• Future workshops should be evaluated
externally, especially as additional impact
information is required for this style of
learning.
• Individual responses/data from the action
planning session (eg a practitioner saying
they will write up their case studies) should
be recorded so that they can be followed
up later in an external evaluation.
• External evaluation should occur relatively
soon after the event – no longer than
two to three months after – to allow
participants enough time to change or
implement new practice, but not allow
time for memory recall to fade.
• If bulletins are produced from the
workshops, they may need to be evaluated
separately to avoid delaying the workshop
evaluation.
• External evaluation is appropriate for
gathering the information required,
provided it is carried out in a timely
fashion. Future ways to improve an
evaluation could include covering these
additional questions:
- did participants contact anyone from the
day again?
- did they make new contacts?
- did they tell anyone else about what
they had done/learnt?
- did they do anything differently as a
result of attending?
- did they use the materials, examples,
points made on the day?
- did they find the day more or less useful
than traditional dissemination of case
studies, and if so why, how, etc?
9
 Further information
The HIA Gateway website
(www.hiagateway.org.uk) provides access
to HIA-related resources, networks and
information to assist those participating in
the HIA process. The site is designed for both
beginners and seasoned HIA practitioners.
The four summary bulletins, the workshop
report and the evaluation of the learning from
practice work programme (detailed in the
References) are all available on the website
(à Resources à Other materials à General
guidance).
References
Crozier, A. (2004). Evaluation of health impact
assessment learning from practice workshops.
London: Health Development Agency.
Gowman, N., Taylor, L. and Quigley, R. (2003).
Learning from practice: report of a series
of workshops for health impact assessment
practitioners. London: Health Development
Agency.
McCormick, G. (2002). Report to the Teenage
Pregnancy Unit on ‘promising practice’ project.
London: Health Development Agency.
NHS Centre for Reviews and Dissemination
(1999). Effective health care: getting evidence
into practice. Effective Health Care Bulletin 5 (1):
1-16.
Ollerearnshaw, S., King, E. and Wright, S. (2000).
The effectiveness of different mechanisms for
spreading best practice. London: Office of Public
Management.
10
Stacey, R.D. (1999). Strategic management
and organisational dynamics: the challenge of
complexity. New York: Financial Times/Prentice
Hall.
Taylor, L. and Quigley, R. (2001). Looking back to
move forward: a retrospective process evaluation
of five HIA case studies. Presentation at the
4th Annual HIA Conference, Liverpool, 16–17
October 2001. London: Health Development
Agency (unpublished).
Taylor, L. and Quigley, R. (2002). Health impact
assessment: a review of reviews. London: Health
Development Agency.
Taylor, L., Gowman, N. and Quigley, R. (2003a).
Learning from practice bulletin: Influencing the
decision-making process through health impact
assessment. London: Health Development
Agency.
Taylor, L., Gowman, N. and Quigley, R. (2003b).
Learning from practice bulletin: Evaluating
health impact assessment. London: Health
Development Agency.
Taylor, L., Gowman, N., Lethbridge, J. and
Quigley, R. (2003c). Learning from practice
bulletin: Deciding if a health impact assessment
is required (screening for HIA). London: Health
Development Agency.
Taylor, L., Gowman, N. and Quigley, R. (2003d).
Learning from practice bulletin: addressing
inequalities through health impact assessment.
London: Health Development Agency.
Authors:
Robert Quigley, Q-Research & Assessment Ltd and Lorraine Taylor, Health Development Agency

Copies of this publication are available to download from the HDA website (www.hda.nhs.uk).

Health Development Agency

Holborn Gate
330 High Holborn
London
WC1V 7BA

Email: communications@hda-online.org.uk
URL: www.hda.nhs.uk

© Health Development Agency 2004

ISBN 1-84279-259-8

About the Health Development Agency

The Health Development Agency (www.hda.nhs.uk) is the national authority and information
resource on what works to improve people’s health and reduce health inequalities in England.
It gathers evidence and produces advice for policy makers, professionals and practitioners,
working alongside them to get evidence into practice.

11