Standing on

the
shoulders
of giants
How the evidence base in
career development can
inform your practice

Tristram Hooley
(Professor of Career Education)

www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Evidence based practice and policy

Understanding what is
known about the
efficacy of career
development
Monitoring
implementation and
checking efficacy

Developing new
policies and services

Implementing new
policies and sevices

www.derby.ac.uk/icegs
www.derby.ac.uk

Take a long hard look in the mirror

What in your practice is
evidence based?
What in your practice is
not evidence based?
What evidence would you
like?
What do you think works
and why?
Whose job is to get you
the evidence you need?

www.derby.ac.uk/icegs
www.derby.ac.uk

By the end of this session we will

have
1. Reviewed the existing literature on the evidence base on
career development.
2. Considered the implications of existing research for your
practice.
3. Mapped gaps in the literature and identified how filling
some of those gaps could enhance your practice.
4. Discussed strategies to gather evidence for your
practice, in particular by thinking about how to build
alliances with academics and established researchers.

www.derby.ac.uk/icegs
www.derby.ac.uk

www.derby.ac.uk/icegs
www.derby.ac.uk

Overview

Does career guidance work?

What works best?
What do we mean by evidence?
Where do we go from here?

www.derby.ac.uk/icegs
www.derby.ac.uk

Overview

Does career guidance work?

What works best?
What do we mean by evidence?
Where do we go from here?

www.derby.ac.uk/icegs
www.derby.ac.uk

Well

How do you know?

www.derby.ac.uk/icegs
www.derby.ac.uk

Knowledge

Knowledge
Evidence
www.derby.ac.uk/icegs
www.derby.ac.uk

Does it work?

Research is
specific not
general.
However there is
clear evidence
that careers work
has a range of
impacts.

www.derby.ac.uk/icegs
www.derby.ac.uk

Key papers

Whiston et al.s (1998) meta-analysis of 47 studies identified impacts across

all types of career interventions.
Bimrose et als (2008) five-year longitudinal tracking study of 50 career
guidance clients found that one-to-one guidance interventions were regarded
as useful by clients, and that guidance services can support adults to make
successful transitions in a turbulent labour market.
Vuori et al.s (2012) paper used a randomised control trial to demonstrate the
impact of a group intervention on career management skills.
Carey & Dimmitt (2012) found that there was consistent evidence of a
positive relationship between well-organised school counselling programmes
and the educational outcomes of students.
And lots more see the reference list at the end.

www.derby.ac.uk/icegs
www.derby.ac.uk

Overview

www.derby.ac.uk/icegs
www.derby.ac.uk

What do you do that you think works

really well?

www.derby.ac.uk/icegs
www.derby.ac.uk

Feedback
Best technique/approach from each group.
Did anything divide you?

www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Summarising the evidence base: focus

on the individual

www.derby.ac.uk/icegs
www.derby.ac.uk

How are you focusing on the individual?

What could you be doing

new/additionally/differently?

www.derby.ac.uk/icegs
www.derby.ac.uk

Summarising the evidence base:

support learning and progression

www.derby.ac.uk/icegs
www.derby.ac.uk

How are you supporting learning and

progression?

What could you be doing

new/additionally/differently?

www.derby.ac.uk/icegs
www.derby.ac.uk

Summarising the evidence base:

ensuring quality

www.derby.ac.uk/icegs
www.derby.ac.uk

How are you supporting learning and

progression?

What could you be doing

new/additionally/differently?

www.derby.ac.uk/icegs
www.derby.ac.uk

10 evidence-based principles for the design

of lifelong guidance services

www.derby.ac.uk/icegs
www.derby.ac.uk

Popular careers ideas with ????

around the evidence
The idea that the labour market and
organisations were once stable and are
now boundaryless
Learning styles
MBTI
NLP
The importance of matching personality
type to job roles
Career plans
LinkedIn profiles
Generational differences
Etc, etc., etc.

www.derby.ac.uk/icegs
www.derby.ac.uk

Overview

www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

What is evidence?
1. Information that informs practice. This
might include LMI, research about career
decision making and employer attitudes
etc.
2. The available and publicly accessible
body of facts that can support
propositions about careers work.

www.derby.ac.uk/icegs
www.derby.ac.uk

Example 1: Australian LMP

Australian
Labour
Market Portal
http://
lmip.gov.au/d
efault.aspx?L
MIP

www.derby.ac.uk/icegs
www.derby.ac.uk

Example 2: Hyperbolic discounting

Given two similar rewards, humans show a preference for
one that arrives sooner rather than later (present biased).
Would you prefer a dollar today or three dollars
tomorrow?
Would you prefer a dollar in one year or three dollars in
one year and one day?
This seems to hold true for career decision making.
See
Schoenfelder, & Hantula (2003); and Saunders & Fogarty (2001).

www.derby.ac.uk/icegs
www.derby.ac.uk

Example 3: Employers attitudes to

disability
Australian employers hold relatively positive attitudes
regarding individuals with disabilities.
However, employers feel less positive about disabled
people in the workplace.
This impacts negatively on hiring decisions, provision of
accommodations and work performance appraisals.
See Burke et al. (2013).

www.derby.ac.uk/icegs
www.derby.ac.uk

Implications of these examples

None of these examples say anything about career
development practice (whether it works or what works)
But all are suggestive of implications for careers work.
How can we engage with such a broad range of
literature?

www.derby.ac.uk/icegs
www.derby.ac.uk

Strategies for engagement

Seek out syntheses and knowledge translation e.g.
Running in a Forest, Jockey and the Architect.
Commit to learning one thing a week e.g. watch a
Ted Talk.
Go to more conferences
Get on Twitter and follow some interesting people.
Write a blog

www.derby.ac.uk/icegs
www.derby.ac.uk

Levels of impact (See Kirkpatrick)

www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Reaction
How do users of services react to them?
Use of feedback forms and happy sheets.

Voices of Users (Vilhjlmsdttir et al., 2011) is a summary of

the experience of career guidance clients in Nordic
countries. It found that the majority of participants were
satisfied with the service that they had received and felt that
the counsellor with whom they had worked had been
supportive and understanding.

www.derby.ac.uk/icegs
www.derby.ac.uk

Learning
Did the individual learn something as a result of the
intervention.
If you tested them before and after would something have
changed?
Learning for now or later (Kuijpers & Meijers, 2012) found
that students career competencies (reflection, exploration,
proactivity and networking) were correlated with the
presence of a practice based and inquiry based curriculum
which allowed them to engage in career conversations.

www.derby.ac.uk/icegs
www.derby.ac.uk

Behaviour
How do individuals behaviour change following an
intervention.
Can you measure what they do differently?
A career workshop was developed in Switzerland to
promote the career choice readiness of young adolescents.
In an evaluation of the workshop with 334 Swiss students in
the 7th grade, Hirschi & Lge found that three months after
the workshop, participants had significantly increased their
career decidedness, career planning and career exploration.

www.derby.ac.uk/icegs
www.derby.ac.uk

Results
What actually happens as a result of your intervention?
Do people get jobs, better qualifications etc.
A report in Northern Ireland (Regional Forecasts, 2008)
examined the impact of the Educational Guidance Service
for Adults on the Northern Ireland economy. The study used
a mix of quantitative and qualitative methods, including
detailed analysis of the services client data, to estimate the
economic value of the service. This was estimated to be
9.02 net additional tax revenue for every 1 of public
money invested.
www.derby.ac.uk/icegs
www.derby.ac.uk

Your practice
What level of evidence is it based on?
Could you evaluate it at the next level up?

www.derby.ac.uk/icegs
www.derby.ac.uk

Overview

www.derby.ac.uk/icegs
www.derby.ac.uk

What evidence would you like to exist?

www.derby.ac.uk/icegs
www.derby.ac.uk

Finding evidence
1.
2.
3.
4.
5.

Search
Synthesise
Develop new research questions
Conduct research or commission studies
Publish new evidence

www.derby.ac.uk/icegs
www.derby.ac.uk

Why evaluate?
Evaluation enables us to:
examine what we do
think about how we can improve it
decide on whether it was worth doing
provide others with a summary to help them to understand
what was done.

www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

What is evaluation?
When people seek to evaluate what they are doing as part of
an attempt to learn and improve, they are usually undertaking
a formative evaluation, so called because it is undertaken to
inform what is done while the activity is still in progress.
We would like to find out how to do these things better
When people evaluate to make a judgement on the value of a
particular activity and to draw out what has been learnt, it is
usually a summative evaluation; so called because it
attempts to create a summary of what has been achieved and
what the impacts have been.
We would like to find out how well these things work
www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Discussion: Evaluation data

What data do you have which could allow you to make a
judgement about the impact of your practice?
What level of impact would it describe?
What data could you collect which could allow you to
make a judgement about the impact of your practice?
What level of impact would it describe?

www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Publish or perish
Too little evaluation and impact work on careers work is
published.
Writing up your evaluation for broader circulation is an
important way to support the development of the sector.
Self publication
Journal publication
Partnership with academics
Using external consultants
www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Write it down
My main reason for writing is simple: I do not know
what I think until I have written it. In conversation
one can get away with loose, exploratory thinking,
but in writing it down one has to weigh up the
arguments and the evidence, and decide what it all
means and where one stands. It is hard work, but
important; and if published, it adds to the body of
knowledge on which others can draw.
Tony Watts
www.derby.ac.uk/icegs
www.derby.ac.uk

On working with academics and

researchers
What matters to them

Time
Money
Authorship
Impact

What you can give them

Your time
Funding
Authorship or co-authorship
Data
Access to interventions and research populations
The change to impact on practice

www.derby.ac.uk/icegs
www.derby.ac.uk

Final reflection

www.derby.ac.uk/icegs
www.derby.ac.uk

References and resources

Australian Labour Market Portal http://lmip.gov.au/default.aspx?LMIP
Bimrose, J., Barnes, S-A. & Hughes, D. (2008). Adult Career Progression and Advancement: a FiveYear Study of the Effectiveness of Guidance, Coventry/London: Warwick Institute for Employment
Research/Department for Innovation, Universities and Skills.
Burke, J., Bezyak, J., Fraser, R. T., Pete, J., Ditchman, N., & Chan, F. (2013). Employers' Attitudes
Towards Hiring and Retaining People with Disabilities: A Review of the Literature. The Australian
Journal of Rehabilitation Counselling, 19(01), 21-38.
Carey, J. & Dimmitt, C. (2012). School counseling and student outcomes: summary of six statewide
studies. Professional School Counseling, 16(2): 146-153.
Hirschi, A. & Lge, D. (2008). Increasing the career choice readiness of young adolescents: an
evaluation study. International Journal for Educational and Vocational Guidance, 8(2): 95-110.
Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU.
Kirkpatrick, D.L. (1994). Evaluating Training Programs: The Four Levels. San Francisco: BerrettKoehler.
Kuijpers, M., & Meijers, F. (2012). Learning for now or later? Career competencies among students in
higher vocational education in the Netherlands. Studies in Higher Education, 37(4), 449-467.

www.derby.ac.uk/icegs
www.derby.ac.uk

References and resources II

Regional Forecasts (2008). Examining the Impact and Value of EGSA to the NI Economy. Belfast:
Regional Forecasts.
Saunders, R., & Fogarty, G. J. (2001). Time discounting in relation to career preferences. Journal of
Vocational Behavior, 58(1), 118-126.
Schoenfelder, T. E., & Hantula, D. A. (2003). A job with a future? Delay discounting, magnitude effects,
and domain independence of utility for career decisions. Journal of Vocational Behavior, 62(1), 43-55.
Vilhjlmsdttir, G., Dofradttir, A.G. & Kjartansdttir, G.B. (2011). Voice of Users Promoting Quality
of Guidance for Adults in the Nordic Countries. Oriveden Kirjapaino, Finland: Nordic Network of Adult
Learning.
Vuori, J., Toppinen-Tanner, S. & Mutanen, P. (2012). Effects of resource-building group intervention on
career management and mental health in work organizations: randomized controlled field trial. Journal
of Applied Psychology. 97(2): 273-286.
Whiston, S.C., Sexton, T.L. & Lasoff, D.L. (1998). Career intervention outcome: a replication and
extension of Oliver and Spokane (1988). Journal of Counseling Psychology, 45(2): 150-165.

www.derby.ac.uk/icegs
www.derby.ac.uk

Stuff Ive written

Dent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013).
Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition.
Bristol: HEFCE.
Hooley, T. (2014). The Evidence Base on Lifelong Guidance. Jyvskyl, Finland: European Lifelong
Guidance Policy Network
Hooley, T., Hutchinson, J. and Neary S. (2014) Evaluating Brightside's Approach to Online Mentoring.
Derby: International Centre for Guidance Studies. University of Derby.
Hooley, T., Marriott, J. and Wellens, J. (2012).
What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury
Academic.
Hooley, T., Matheson, J. & Watts, A.G. (2014). Advancing Ambitions: The Role of Career Guidance in
Supporting Social Mobility. London: Sutton Trust.
Hooley, T., Mellors-Bourne, R. and Sutton, M. (2013). Early Evaluation of Unistats: User Experiences.
Bristol: HEFCE.
Marriott, J. and Hooley, T. (2014). Evaluating the Legacy Careers Project. Derby: International Centre
for Guidance Studies, University of Derby.
Taylor, A.R. & Hooley, T. (2014).
Evaluating the impact of career management skills module and internship programme within a universit
y business school.
British Journal of Guidance & Counselling, 42(5): 487-499.
www.derby.ac.uk/icegs
www.derby.ac.uk

Tristram Hooley
Professor of Career Education
International Centre for Guidance Studies
University of Derby
http://www.derby.ac.uk/icegs
t.hooley@derby.ac.uk
@pigironjoe
Blog at
http://adventuresincareerdevelopment.wordpress.com
www.derby.ac.uk/ic
egs www.derby.ac.uk/icegs
www.derby.ac.uk

Remember, remember
Your practice is only as good as the evidence that
underpins it.
There is no point in being really good at doing something
that has no impact.
Your job and your profession are vulnerable where there
is no evidence.
There arent many professional career education and
guidance researchers.
Research takes some skill, but it isnt rocket surgery.

www.derby.ac.uk/icegs
www.derby.ac.uk