Available online at www.sciencedirect.

com

International Journal of Project Management 26 (2008) 223237

www.elsevier.com/locate/ijproman

The development and delivery of an industry led project

management professional development programme: A case study
in project management education and success management
Mehmood Alam a,*, Andrew Gale a, Mike Brown b, Callum Kidd a
a

Engineering Project Management Research Group, School of Mechanical, Aerospace and Civil Engineering,
The University of Manchester, P.O. Box 88, Sackville Street, Manchester M60 1QD, UK
b
Programme Management Centre, Rolls-Royce plc, Derby, UK
Received 13 December 2007; accepted 18 December 2007

Abstract
Global changes inuence the project environment, client relationships and the behaviour of suppliers. The people managing projects
(the project management community of practice) are increasingly important, requiring professional development and training. Project
management education is time and resource intensive.
Historically conventional return on investment criteria have not been applied to investment in the areas of education and training. This paper reviews a case study modular distance learning programme: The Project Management Professional Development
Programme; providing education in generic project management for a consortium of four international companies across aerospace,
infrastructure, oil and gas, nuclear design, construction and information technology sectors. The programme started in May 2000
and has currently circa 200 delegates having graduated 100. It is an academic-industrial collaboration between The University of
Manchester (UoM) and Rolls-Royce, AMEC, Goodrich and EDS with some guest organisations participating also. The literature
on educational issues, professional development, competence and Benet Metrics (return on training investment) are reviewed. Particular attention is paid to the management and development of the programme and the project management of project management
education. The drivers, development and implementation of a managed learning environment, and blended learning are discussed.
This includes issues related to expectation management and the interesting benets of educating dierent members of the supply
chain represented by the industrial partners.
The linkages between Benet Metrics, project management competencies and learning outcomes in the context of an industrial-academic partnership are specically explored. Discussions and conclusions focus on lessons learnt and suggestions on the development and
delivery of the programme and its eectiveness.
2007 Elsevier Ltd and IPMA. All rights reserved.
Keywords: Competence; Project management; Professional development; Benet metrics; Management development

1. Introduction

*
Corresponding author. Tel.: +44 (0) 161 306 2604; fax: +44 (0) 161 306
4252; mobile: +44 789 272 2122.
E-mail
address:
muhammad.alam@postgrad.manchester.ac.uk
(M. Alam).

0263-7863/$30.00 2007 Elsevier Ltd and IPMA. All rights reserved.

doi:10.1016/j.ijproman.2007.12.005

This paper concerns a modular distance-delivered generic project management professional development programme for cross-sector industrial partners. It is run as a
project and the ongoing research reported here arises from
a maturing academic-industrial educational partnership
which has been running for nearly eight years. Over that

224

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

time 100 students (delegates) have graduated with a masters degree in Project Management as well as others with
postgraduate modules, certicates and diplomas.
This paper begins with a brief introduction to the discussion on the drivers and criteria for project success. The
authors then move on to explain the concepts of Benet
Metrics and Return on Investment in relation to professional development education and training. Anecdotal evidence from industrialists suggests that companies fail to
take project management education seriously because of
the diculties in demonstrating its eects on protability
and competitiveness. Companies are unaware of the variables that inuence such programmes and as a result are
unable to measure a return on their investment.
The paper briey describes the case study course: The
Project Management Professional Development Programme (a modular distance learning course) and then
reports on the interim ndings from three surveys, forming
the initial phase of an industry led action research initiative
to investigate Benets Metrics. Conclusions are drawn
relating to programme eectiveness and lessons learned
to date from the industry-academic partnership.
2. Using project management for achieving project success
In a changing and uncertain world project management
is becoming increasingly important for the delivery of successful projects and is acknowledged as more eective than
traditional functional management [1,2] in doing this. The
Project Management Institute Project Management Body
of Knowledge (PMBoK) Guide [3] denes a project as a
temporary endeavour undertaken to create a unique product or service, and states that although projects vary considerably in type and scale they are a tool employed by the
organisation to achieve the strategic plan. Project Management on the other hand is the application of knowledge,
skills, tools and techniques to project activities to meet project requirements [3]. Processes used in Project Management include initiating, planning, executing, controlling
and closing. The Association for Project Management
Body of Knowledge [4] takes a slightly dierent view, stating that Project Management is the most eective way of
introducing unique change and constitutes the successful
management of projects.
These denitions are both heavily oriented to achieve
project success [2]. There is a considerable volume of literature in the eld of project management dealing with project
success, and this tends to fall into three major categories:
dealing with project success criteria; project success factors
and those that confuse the two [5]. The Iron Triangle [6]
comprises three well recognised criteria (cost, time and
quality) against which project success is measured (Fig. 1).
In addition, many studies have expanded project success
criteria to include such things as organisational objectives,
stakeholder satisfaction, customer benets and future
potential for the organisation. Researchers do not reach a
consensus on project success criteria. Morris and Hough

Cost

Quality

Time
Fig. 1. The Iron Triangle [6].

[7] use project function, project management and the contractors business performance to measure project success.
Lim and Mohamed [8] dene micro and macro criteria to
measure project success. Their micro criteria comprise
time, cost, quality, performance and safety, whilst macro
criteria include the micro criteria plus the project outcome
benet. On the other hand some consider project eciency,
customer benets, organisational success and the future
potential to an organisation as critical when measuring
project success [9]. However, the majority of research practitioners [1013] consider project success as an important
project management issue [5]. For instance, the PMBoK
guide published by the PMI suggests that project success
criteria should include the Iron Triangle and key project
stakeholder satisfaction [14].
Considering the emerging body of research on project
success, Crawford [5] identies twenty four success factors
as primary for successful projects. The majority of these
factors are directly related to project management competence and demonstrates that the competence, knowledge,
skills and attributes of project managers, are critical to project success [5]. The competence of project managers is in
itself a factor in the successful delivery of projects. Project
managers need to have competence in those areas that have
the most impact on successful outcomes.
From an industry perspective, it would be useful to be
able to say that if an outstanding project manager is
responsible for a project, it guarantees that the project will
be a success, but this is not always true and whilst a poor
project manager may doom the project to failure an outstanding project manager may not necessarily guarantee
success. Thus, it appears that the direct causal relationship
between an outstanding individual and project success is, at
best, tenuous. This may be due, for instance, to the complexity of the project environment. Contributing factors
include:
(a) the quality of the project team over which, in a matrix
organisation, the project manager has little selection
control;
(b) the project management maturity of the organisation;
(c) the level of stakeholder convergence;

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

(d) context stability, including technology maturity and

experience of similar projects.
It would be reasonable to suggest that knowledgeable,
experienced project managers with good leadership skills
may make a dierence. Therefore, providing project management education and professional development for project managers demonstrating appropriate behaviours
should result in improved competence. However, corporate
sponsorship for project management education programmes has to date been a leap of faith.
3. Project management education: The concept of return on
investment
Organisations are spending signicant amounts annually on professional development for their employees with
a view to reaping the reward in the form of competitive
advantage. According to a training survey [15], employers
spend over $50 billion per year on formal employee training and education and approximately $180 billion per year
is spent on informal, on-the-job training. Despite this
investment in educational programmes in the eld of project management education, it is dicult to be sure that
the knowledge gained is linked to the context of professional practice which will benet the employing organisation [16].
Expenditure on training and education represent an
investment for which there would be the expectation of a
return deemed benecial to the organisation. It would be
expected that Return on Investment (ROI) would be a fundamental consideration.
The value of professional development and training programmes delivered by universities and other providers continues to attract much debate. According to Jim Hinds
(Marakon Associates) who advises FTSE 100 companies
on the drivers of performance [17]:

225

Companies have failed to take training (and education)

as seriously as they should because it is hard to demonstrate
exactly how much impact it has on the bottom line and you
cannot measure it, it is hard to communicate the need for it
to the board or CEO.
There are several issues driving increased interest in,
and application of, ROI including pressure from clients
and senior managers. Competitive economic pressures
can cause intense scrutiny of all capital expenditure,
including training and development costs. Engineering
processes such as Total Quality Management (TQM),
Business Process Re-engineering and Continuous Process
Improvement have created a renewed interest in measurement and evaluation, including the measurement of training eectiveness. Also, the general trend towards
accountability in organisations on functions playing a signicant role in delivering projects has led to project management and human resources to measure their
contribution [18].
Higher education institutions (HEIs) like The University
of Manchester, as providers, are also interested in evaluating the extent to which their programmes are successful in
meeting organisational needs. These needs include: raising
the prole of project management, career development,
promoting a cadre of project management, specic interest
groups and increasing the project management knowledge
of employees. These (or similar) factors have created an
increase in the application of ROI in project organisations.
Departing from the Kirkpatrick 4 level evaluation model
[19], Phillips [18] proposes a ROI model. This model
(Fig. 2) comprises ve levels; each investigating dierent
elements of a professional development/training
programme.
Level 1, Reactions: measures programme participant satisfaction along with their plans to apply what they have
learned. Whilst this level also investigates customer

Data Analysis
Planning
Tabulate
Programme
costs

Reporting
Develop
evaluation
plans and
Baseline data

Reactions
Level I

Learning
Level II

Application
Level III

Business Impact
Level IV

Convert Data to
Monetary value

ROI level
Level V
Reach
conclusions/
generate report
Identify
Intangible
benefits

Fig. 2. The ROI Model [18].

226

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

satisfaction, a favourable reaction does not ensure that

participants have learned new skills or knowledge.
Level 2, Learning: focuses on what participants have
learned during the programme, using tests, skill practices, role plays, simulations, group evaluations, and
other assessment tools. However, a positive measure at
this level is no guarantee that what is learned will be
applied on the job.
Level 3, Application and Implementation: determines
whether participants applied what they learned on the
job. Appraisal by line managers and 360appraisals are
well recognised tools for tracking the application of
knowledge. Though level 3 is important to gauge the
success of the application of a programme, it still does
not guarantee that there will be a positive business
impact on the organisation.
Level 4, Business Impact: focuses on the actual results
achieved by the programme participants as they successfully apply what they have learned. Typical level 4 measures include output, quality, cost, time, and customer
satisfaction. Although the programme may produce a
measurable business impact, there is still a concern that
the programme may cost too much.
Level 5, Return on Investment: compares the monitory
benets from the programme with the programme costs.
Although ROI can be expressed in several ways, it is
usually presented as percentage or cost/ benet ratio.
Measures pertaining to levels 1 and 2 are mostly quantitative and can be assessed easily at individual level. Levels 3
and 4 involve task and organisation oriented measures
which are mostly investigated qualitatively [20]. It is clear
that it is necessary to measure programme eectiveness
beyond individual performance demands, involving the setting of clear qualitative measures, which is not easy to
achieve.
Phillips [18] writes convincingly on the relationship
between competence and ROI. He is a devotee of developing algorithms for the calculation of real nancial benet in
order to compare benets with training, education, and
development investment costs [21]. He argues that organisations have moved from training for activity to training
with focus on bottom-line results. However, it is very
dicult to see how project management development

Organizations

programmes with complex inputs and slow-to-develop

intangible benets can be tted into this quantitative
approach [21].
Industrial organisations prefer the concept of Benet
Metrics. Companies invest in training relating to management skills without a clear understanding of how knowledge learned on a particular educational programme
becomes meaningful in practice, and benets the organisation as a whole. Most recently, researchers [2224] have
also begun to question this relationship. ROI can be
dened as a subset of Benet Metrics. The ROI approach
is limited to the quantication of benets from training
and education, whereas Benet Metrics attempts to investigate programme eectiveness by accounting for both tangible and intangible educational benets.
Benet Metrics can best be explained by taking a three
dimensional perspective. Organisations in partnerships
with educational institutions invest in educating and training their employees in an attempt to seek improvements at
individual, project, business unit and corporate levels. The
companies expect improved results in terms of intangible
benets and the organisations bottom line, such as return
on capital employed (ROCE), protability and cost savings. Higher Educational Institutions are equally interested
in assessing programme eectiveness and eciency in meeting corporate objectives [25]. The construct is illustrated in
Fig. 3.
Some would argue that there is no valid and reliable
relationship between indirect measures of intangible
benets like increased job satisfaction and increased
organisational commitment and quantitative nancial
bottom-line indicators. There may be arguments based
on anecdotal evidence, but these are not statistically valid
associations [21]. Rowe [26] argues that it is not possible
to make measurements that enable ROI to be evaluated
with respect to intangible benets. On the other hand,
direct measures evaluating the success of an organisation
are quantitative and internationally understood (i.e. return
on capital employed (ROCE), protability, turnover, and
market share).
Measuring the eectiveness of a professional development programme requires the separation of training and
education from a multitude of other variables (such as marketing, management attention, changes in procedures,

BM

Educational Institu
tes/
training

Benefits---Increased Profits,
ROCE, intangible benefits

Employees

Fig. 3. BM Construct [25].

Learning -----Increase in
competencies, skills development,
knowledge gain

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

adjustment in standards, interest rates, economic and political stability) that can inuence long-term organisational
performance. The immediate eects of training and education on KPIs are usually invisible and they only become
visible when monitored over long periods [27]. Fig. 4 presents a framework for the investigation of Benet Metrics
involving linkages and associations between investment,
education and training and ROCE in relation to competence, change, and their measurement.
4. The case of an industry-led modular masters project
management professional development programme
(PMPDP)
The method and speed of development of the syllabus,
content, learning outcomes, form, delivery, management
and internationalisation of the programme give an indication of how industry and academia can work swiftly and
eectively to meet a growing need for modular masters
level exible blended learning. In September 1999 the lead
industrial consortium training partner (Rolls-Royce) concluded tendering for a training partner and in May 2000
delegates (students) were studying a new modular masters
programme (PMPDP).
In 1999 a group of senior project management practitioners from across Rolls-Royce were meeting regularly with
the following objectives:






to create a community of project management;

to share good practice;
to encourage professionalism;
to develop project management sta at all levels;
to raise the prole of project management and its importance to business performance across the company.

The programme objectives developed collaboratively by

The University of Manchester (UoM) and the industry

227

consortium were to support the above corporate objectives

by:
 providing project management education at a postgraduate level;
 providing a basis for a ladder of opportunity for those
seeking a postgraduate qualication;
 providing through the Foundation Module the preparation for sitting Association for Project Management
examinations;
 providing single modules in specic project management
elds;
 providing exible learning to enable busy engineers,
managers and other employees to study for a postgraduate qualication;
An 8 module  12 topic matrix was developed to
describe the purpose and content of the taught modules.
Currently this matrix (now Revision 19) is still used to
review the programme and forms an appendix to the contract between the consortium and the university. During
the run up to delivering the programme several major
review meetings, attended by industry specialists and academics, were held to progressively freeze the learning objectives, content and delivery. It is clear, looking back, that
these meetings practitioner/academic balance to content
and helped to support a team spirit and common language
between academia and industry. The learning outcomes
owed from the interpretation of the discussions to develop
the syllabus or Matrix.
Fig. 5 illustrates the relationship between curriculum
development, delivery and feedback. This is a seamless process involving delegates and their employers. Delegate feedback occurs at every stage of delivery and assessment and
from employers through the formal Steering Group and
informally via the project management single point contact
between the consortium representative and the university
Programme Director. The eectiveness of this project management style is very important in terms of managing
expectations for all the stakeholders. All feedback is
reported in some shape or form at the Steering Group
which meets twice per annum.
Central to the cycle illustrated in Fig. 5 are the learning
outcomes for the individual modules and the programme
as a whole. These learning outcomes are the basis of content, delivery and assessment.
5. Programme content

Fig. 4. Benet Metrics framework (ROI cycle) [21].

For each module, delegates receive a workbook in

hardcopy, CD or CD-Rom format, replicated on the university website. At each entry point (twice per annum)
there are introductions explaining the objectives and
learning points for the modules. The modules include
self-assessment exercises and exercises designed to develop
certain intellectual skills and reinforce theoretical concepts
and practical applications. Each module is accompanied

228

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

INDUSTRY
DRIVERS
Behaviours
Competencies
KPIs
Return on
Investment

DEMAND
Industry

DISCUSSION
University and
Industry

STEERING
GROUP
University
Industry

Objectives
Fields of study
Process/Delivery
Content
Assessment
Planned Learning Outcomes

OUTCOME
Syllabus in
form of Matrix
of Modules

OUTCOMES
Action list
Changes:
to process
to modules

WRITING
Module
Co-ordinators

EDITING
University and
Industry editing

OUTCOMES
Module Workbooks
Specifications: Programme and
Modules
Delegate and Tutor Guide
Virtual Learning Environment
FEEDBACK
From
Delegates
Tutors
Industrialists

Student Learning Process

To achieve Learning
Outcomes
DELIVERY
Residential events
Personal learning
Web Enabled
Learning
ASSESSMENT
Coursework
Examinations

Fig. 5. Programme development and review cycle [28].

by a set text and directed reading. These are designed to

form a personal project management library for delegates.
Further reading and useful internet resources are listed
too.

gramme is moving towards the increasing use of continuous assessment and the incorporation of reective practice.

6. Assessment

The programme is run as a web-enabled distance learning programme. Delegates attend three day plenary sessions in April and October and one day mid-session
events in December, January, July and September. Plenary
events cover orientation, lectures, keynote lectures, introductions, learning skills, clinics and examinations. Mid-ses-

All modules are assessed over the six months study period and during the following plenary event with a two hour
written examination where appropriate. Also, an industry
aligned coursework assignment is submitted. The pro-

7. Method of delivery

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

sion events cover tutorial support, seminars, clinics and

revision classes. All events involve supply chain integration
networking and informal events at which consortium
seniors are present. The companies play an important role
in approving the structure and content of the events and
actively participate in their delivery.
8. Managed learning environment
UoM uses software called WebCT to facilitate a managed learning environment within which a virtual learning
environment is sustained. The Joint Information Systems
Committee (JISC) has produced a very useful Brieng
Paper on Virtual and Managed Learning Environments
(Everett, 2001).
The three functions of a managed learning environment
are:
 content
 communication
 administration
The use of the software to web-enable the programme is
gradually evolving and the feedback from delegates is crucial to the success of the web-enabled aspect of the programme. Delegates also have access to the UoM library,
enabling access to thousands of on-line journals. Delegates
can access their local academic libraries, as reference only
users.

229

randum of Understanding and Intellectual Property

Agreement.
11. Benet metrics, project management competencies and
learning outcomes
Considering the emerging body of research and debate
on competence in project management, the strongest argument for an association between measurable bottom-line
indicators and intangible benets from the activities of education, training programmes may be to do with competence [21].
12. Competence
The argument for improving the competence of employees is strong. A competence based approach enables the
identication of appropriate people to train and educate
[28,29]. Competence can be dened as [30]:
an underlying characteristic of an individual that is
causally related to criterion-referenced and/ or superior
performance in a job or situation.
It is a deeply rooted and enduring part of personality,
and can predict behaviour and performance in a variety
of situations. Moreover, it encompasses the ability to predict individual performance against a specic standard.
There are argued to be ve characteristics of competence
[30]:

9. Management of the programme

The programme is managed on a day-to-day basis by
UoM academic and academic related-sta. A Programme
Manager is supported by a Distance Learning Administrator. The eight modules are tutored and examined by
Module Co-ordinators. Overseeing and managing the
team is the c Programme Director. The lead industrial
partner (Rolls-Royce plc) has a single point of contact
manager (Industrial Partners Manager) representing all
industrial partners in the consortium. He is in very close
regular contact with the Programme Director. The programme is subject to the normal academic scrutiny and
quality assurance procedures including the involvement
of an External Examiner. Research and Reective Practice industry led dissertation projects are supervised by
university sta in collaboration with industrial partner
managers.
10. International developments
In November 2001, just after 9/11, the masters programme was launched in the United States for North
American employees of the industrial partners. Pennsylvania State University is running a common sister masters
programme with the same learning outcomes. This involves
close collaboration between both universities via a Memo-

(a) Motives: The things a person consistently thinks

about or wants that cause action. Motives drive,
direct, and select behaviour towards certain actions
or goals.
(b) Traits: Physical characteristics and consistent
responses to situations or information.
(c) Self concept: A persons attitudes, values, or selfimage.
(d) Knowledge: Information a person has in specic
elds.
(e) Skills: The ability to perform a certain physical or
mental task.
13. Project management competence
The International Project Management Association
(IPMA) [31,21, p.1095] undertook a benchmarking study
across all members of the IPMA and developed the International Competency Baseline (ICB) which denes competence as:
Knowledge Experience PersonalAttitudes
Here knowledge and experience relate to function and attitudes to behaviours. The IPMA denition of competence is
very much aligned with the views held by Crawford [32]
that competence is not a single construct.

230

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

The type and level of competence has practical implications for a project managers development. As shown in
Fig. 6, knowledge and skills competence tend to be visible
and relatively, surface characteristics of people. They are
relatively easy to develop - professional development programmes being the most obvious way. Self concept, trait
and motive competence are more hidden, deeper and central to personality. These types of competence are at the
bottom of the iceberg and are more dicult to assess
and develop. It is more cost-eective to select for these
competencies, rather than to train. Self-concept competencies, comprising attitudes and values, lie somewhere in
between and can be changed through personality development experiences, albeit with more time and diculty [30].
The literature identies three dierent approaches to
dening project management competence [29].
(1) The input approach, common in the USA, assumes that
individuals require knowledge, skills and behaviours
to be competent at work. This approach forms the
basis of the BoK developed by the PMI in their
PMBoK [29].
(2) The UK based process approach accounts for processes and functions needed by project managers to
deliver projects successfully. The IPMAs International Competence Baseline is derived from this
approach [29].
(3) The output approach popular in Australia focuses on
the actions of project managers to deliver projects.
This approach forms the baseline for the project management competence standard developed by AIPM
[29].
Concern for the competence of project managers has
fuelled interest in the development of standards and the
certication of project managers, used for assessment, recognition and as a guide for the development of project
management competence. Standards are derived from
typical activities, responsibilities and requirements from

practice. For example the IPMA [21, p.1095] established

a four level structure to assess PM competence.
IPMA Level A: the certied project director shall be able
to direct all projects of the company.
IPMA Level B: the certied project manager shall be able
to manage complex projects him/ herself.
IPMA Level C: the certied PM professional shall be able
to manage non-complex projects him/herself and assist the manager of a complex
project in all aspects of project
management.
IPMA Level D: the certied PM practitioner shall have the
knowledge in all project management
areas.
Professional project management competencies are
achieved by the combination of education and knowledge
acquired during training and skills developed through
experience and application of such knowledge and experience through eective behaviours [28].
14. Measuring competence
There are several indicators commonly used in the measurement of competence [21, p.1099]: continuous education,
examinations,
portfolios,
selfassessments,
interviews, outcomes and peer review. Three techniques
can be used to measure competence: checklists, observational methods and the framework approach. The rst
two approaches focus on performance measurement
whereas the framework approach relates to an integrative
perspective on competence.
A 360-degree feedback approach is widely used for measuring project management competence [21, p.1100]. Data
are collected from an individuals peers, direct reports,
internal customers, and line managers. The technique
involves questionnaires (sometimes software based) to
obtain an evaluation of individuals from multiple sources.

Skill
Self concept
Visible
Skill
Knowledge

Trait,
Motive
Attitudes/ Values

Hidden

Self-Concept
Trait Motive
Knowledge
Surface competencies:
Mosteasy to develop
Fig. 6. Competence Structure [30, p.11].

Core Personality:
i
Dfficult
to develop

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

231

Companies like Rolls-Royce, Goodrich, AMEC and

EDS have an interest in measuring the competence of their
project management communities. Rolls-Royce, for example, has developed a web-enabled competence measurement system [33].
Whatever the method of education or training adopted
it is important to have clarity on the objectives. Assessment
needs to be aligned with learning outcomes.

cies may lead to the specication of certain LOs. A number

of groups (universities, professional bodies, employers)
have vested interests in promoting dierent positions on
competence, BoKs and LOs. In practical terms, in purchasing education and training, industrial organisations expect
the eect on students to lead to improvements in individual
and team performance; contributing in turn to the protability of the organisation.

15. Learning outcomes (LOs)

16. A brief discussion on research being undertaken into

benet metrics

British universities are required by the Quality Assurance Agency for Higher Education (QAAHE) to state the
Learning Outcomes (LOs) for all programmes and modules
of programmes, at both undergraduate and postgraduate
level.Reduced to its simplest form, an outcomes-based
approach to learning has three components:
 an explicit statement of learning intent expressed as outcomes that reect aims and values;
 the process to enable the outcomes to be achieved and
demonstrated (curriculum, teaching, learning, assessment and support methods);
 the criteria for judging achievement of the intended outcomes. [34, Appendix D]
These learning outcomes are subdivided into four
elements:





Academic Knowledge
Intellectual Skills
Subject Practical Skills
Transferable Skills

The APM, PMI and other organisations concerned with

project management publish BoKs. These were considered
when developing the case study programme presented
above, but the BoKs do not easily relate to either learning
outcomes or competencies. Further, there are some obvious gaps in all the BoKs; particularly in the area of people
and culture.
There is no universal agreement or even similarity
between BoKs and what industry, academia, and professional bodies and associations believe to be the key competencies, learning outcomes and knowledge sets. This may
be a good thing because in the interest of critical debate different perspectives are necessary. Therefore, attempts to
unify BOKs may be actually detrimental to a real understanding of the issues and critical thinking.
Organisations and their managers want to dene and
measure competence. In order to promote and develop
the competence of project managers, education is clearly
necessary.
No clear linkage exists between competencies, LOs,
BoKs and BM. LOs may, in part, be measured by competencies and the desire by industry for particular competen-

Three consortium funded studies being undertaken by

the authors are introduced below. The rst used qualitative
techniques to ascertain the eectiveness of the case study
programme mentioned above (PMPDP). The other two
studies used quantitative techniques. These studies form
the beginning phase of a larger research project investigating BM.
The rst study [35] used the Return On Investment
(ROI) methodology proposed by Philips [18] to investigate
the key objective of the Programme; that is to improve the
competence of employees. The model comprises 4 stages:
(a) evaluation planning; (b) data collection; (c) data analysis; (d) programme evaluation. Qualitative data were gathered from delegates via a questionnaire survey.
The second and third projects [33,36] used the RollsRoyce Knowledge and Experience questionnaire (RRKEQ) [37] (an automated computer based electronic system attached as Appendix A) that facilitates the evaluation
of a persons knowledge and experience in the topics that
make-up the APM BoK. Data in the form of RR computed
knowledge and experience grades were taken from the RRKEQ to compare the competence of delegates at dierent
stages of the programme with a control group which had
not undertaken any comparable professional development
in project management.
17. Sampling
Employees participating in the PMPDP were divided
into cohorts based on when they began their studies. The
rst ve cohorts comprising 83 delegates were chosen as
a working population because they had signicant experience of the programme (response rate 25 per cent).
In the second survey, 358 completed RR-KEQs were
available, of which 55 had been completed by delegates
on the PMPDP, leaving 303 questionnaires to act as a control group. The working population for the third project
comprised 181 employees who completed the RR-KEQ in
2005. Among them, 65 were delegates who had studied
PMPDP modules.
The population for all the three surveys covered dierent business units and management levels. Data were stratied in terms of IPMA project management role levels (A,
B, C and D).

232

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

We could say that those areas where the PMPDP is said

to be making a small contribution are the areas where
employees have sucient knowledge to carry out a particular task already, and hence do not need the additional
knowledge. In other words the PMPDP is training employees for more complex tasks that they may need to undertake in the future. It could be argued that these areas
need to be re-evaluated so that they are better aligned with
the needs of the employee, such as conguration and programme management.
Delegates were also asked to identify an area of project
management in which they believed they need the most
training; i.e. the area which causes them the most problems.
Table 1 shows these problem areas and the frequency with
which they have been selected.
Most of these problem areas can be related to those
selected in relation to benets attributed to studying
PMPDP. Table 2 shows these areas along with the frequency with which they were highlighted as problem areas,
and the percentage contribution of the PMPDP in these
areas. Table 2 aims to show whether or not the most frequently specied problem areas are also those which are
said to be beneting most from PMPDP study.
There is one PMPDP and three non-PMPDP areas that
do not have a match. The People and Culture module, in
particular, is intended to be useful in many dierent practical contexts, so whilst people and culture issues are not
highlighted specically, this module probably contributes
to in solutions in several areas.
The matter of Inexperience is one that cannot be
solved by any taught aspect of the PMPDP. It may be
the case that participation in a learning programme will
lead to employees paying more attention to increasing
their experience. For example, looking for areas where
they can implement a new tool or technique taught on

18. Qualitative analysis

In the rst survey the questions asked were:
(1) Give the title or descriptor of an area of benet.
(2) To what extent is this benet attributable to study on
the PMPDP?
(3) In what way did your learning experience on the
PMPDP relate to the benet reported here?
Twenty one responses to question 1 indicated areas of
benets attributable to the PMPDP (Fig. 7).
The analysis of question 1 reveals that all modules of
PMPDP (apart from module 1, Foundation) are noted
by at least one delegate as being of benet. The second
question ascertained the extent to which delegates believed
that improved performance was due to studying on the
PMPDP (Fig. 8).
Frequency of selection by delegates of
areas which benefit from the PMPDP
6.9%

Commercial & Procurement

17.2%

Configuration Management

13.8%

Cost

Organisation

3.4%

People

3.4%

Planning

10.3%

Programme management

10.3%

Project management
Quality management
Risk analysis

10.3%

Strategy

10.3%
6.9%

6.9%

Fig. 7. Frequency of selection by delegates of areas which benet from the

PMPDP [35].

Average percentage contribution of PMPDP

to each area of benefit

100.0%
90.0%

90.0%

Commercial & Procurement

Percentage

80.0%
70.0%

75.0%

75.0%

70.0%

Cost

61.7%

60.0%
50.0%

62.5%
60.0%

Organisation
People

48.3%
42.5%

40.0%
27.5%

30.0%
20.0%

Configuration Management

Planning
Programme management
Project management
Quality management

20.0%

Risk analysis
Strategy

10.0%
0.0%

PM areas
Fig. 8. Average percentage contribution of PMPDP to each area of benet [35].

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

Table 1
Problem areas in project management [35]
Problem Area

Number of selections (N)

Frequency (%)

Project Denition
Resources
Organisation
Time
Costs
Quality
Risk
Contracts
Change
Inexperience

16
38
73
37
11
1
4
12
13
3

7.69
18.27
35.10
17.79
5.29
0.48
1.92
5.77
6.25
1.44

the programme. There is some anecdotal evidence from the

consortium that supports this claim.
In question 3, individuals were encouraged to describe
how the PMPDP had contributed to their work.
Responses to this question indicate that the PMPDP contributes to the improvement of all types of competence. In
terms of knowledge, individuals reported improvements in
their negotiation, communication and conict resolution
skills. Also, the new tools and techniques learned from
the PMPDP led to better bids to clients, better risk assessments, reduced cost over-runs and earlier damage
detection. In the area of behavioural competence,
improvements were noted in relation to levels of condence in ability and skills. Improved condence was seen
to lead to improved on-the-job performance and as was
mentioned previously, experience can not be taught.
PMPDP was reported to have led to a more active and
closer project management community likely to lead to
improvements in experience.
19. Quantitative analysis
In surveys 2 and 3, mean knowledge scores against each
managerial level were calculated on the basis of the participation of delegates in dierent modules of the programme.

233

Since the data contained no numeric values to measure the

knowledge and understanding of each respondent, equivalent numerical mean scores were assigned to the RR, nonnumeric, knowledge grades. This equivalence is shown in
Table 3.
The mean scores of each individual in the control
group were also calculated to obtain an overall mean
knowledge score against each managerial position. Table
4 quanties the eects on mean knowledge scores of delegates studying PMPDP modules, and also compares the
eects on mean knowledge scores of PMPDP delegates
with non-PMPDP employees. This comparison is graphically represented in Fig. 9. The same analyses were carried out for survey 3. The nal results are graphically
represented in Fig. 10.
The results of surveys 2 and 3 indicate that, on completion of the PMPDP, an increased level of knowledge competence could be expected at all managerial levels. When
the UoM computed mean knowledge scores (see Table 3
for equivalent RR non-numeric grades) for each management level were compared with the IPMA recommended
certication levels, it became evident that by the time
delegates had completed 6 modules on PMPDP, it could
be expected that they either meet or exceed the IPMA certied level for that management position. It appears from

Table 3
RR knowledge grade/UoM computed mean score equivalence [33]
RR knowledge
Grade

UoM
Computed
Mean Score

RR knowledge
Grade

UoM Computed
Mean Score

A+
A
A
B+
B
B
C+
C

14.5
13.5
12.5
11.5
10.5
9.5
8.5
7.5

C
D+
D
D
E+
E
E

6.5
5.5
4.5
3.5
2.5
1.5
0.5

Table 2
Problem areas in project management and contribution of PMPDP [35]
Problem Area

Corresponding area of PMPDP

Frequency of selection of problem area

Contribution of PMPDP

Project Denition
Resources
Organisation
Time
Costs
Quality
Risk
Contracts
Change
Inexperience
None specied
None specied
None specied

Project management
Planning
Organisation
Planning
Cost
Qty management
Risk analysis
Commercial & Procurement
Strategy
None applicable
Conguration Management
People
Programme management

7.69%
18.27%
35.10%
17.79%
5.29%
0.48%
1.92%
5.77%
6.25%
1.44%
-

61.7%
42.5%
75.0%
42.5%
48.3%
60.0%
62.5%
70.0%
75.0%
20.0%
90.0%
27.5%

234

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

Table 4
Comparison of PMPDP and Non-PMPDP employees (Survey No.2) [36]
Position

Participation level

Practitioner D
Professional C
Manager B
Overall Score

PMPDP module 1

Mean Score

PMPDP module 8

Mean Score

Non-PMPDP

Mean Score

C+
C+
B
C+

8.5
8.5
9.5
8.5

B+
B+
A
B+

11.5
11.5
12.5
11.5

C
C+
B
C+

7.5
8.5
9.5
8.5

analysis that nearly all PM practitioners and professionals

exceeded their recommended IPMA certication level (D
and C respectively) upon completing 6 PMPDP modules.
However, the group that showed the greatest improvement
was the Project Managers (IPMA level B). Seventy two
percent of PMs not on the programme fell below the
required IPMA certication level but all PM delegates

14
12
10
8
6

20. Research limitations

4
2
ge

er
ve

ra

ag

lA

an
M

ta
To

Pr

Pr

of

ac

es

tit

si

io

on

ne

al

0
r

Mean knowledge scores

Comparison of PMPDP and non-PMPDP employees

No of modules
Non-PMPDP

PMPDP Module 1

PMPDP Module 8

Fig. 9. Survey No.2 [36].

Mean Knowledge scores

Comparison of PMPDP and Non-PMPDP employees

)
,C

,D

er
B

Al

lr

ol
e

(B

an
ag

Pr

oj

ec
t

es
Pr
of

PM

Pr
ac

tit

si

io

on
a

ne
r

lC

14
12
10
8
6
4
2
0

PM

met or exceeded the certied level upon completion of 6

PMPDP modules.
Similar results were drawn from survey 3. All PM practitioners and professionals were found to exceed their recommended IPMA level after completing 6 modules. In
the case of project managers, 25 per cent met the recommended IPMA level B and 75 percent exceeded it. In comparison with non-PMPDP PMs, 36 per cent fell below, 32
per cent met and 32 per cent exceeded their IPMA certied
level.
Generally, it appears from the results that we can
hypothesise that there may have been a general improvement in competence throughout the organisation due to
delegates studying the PMPDP.

No of Modules
Non-PMPDP

PMPDP Module 1

Fig. 10. Survey No.3 [33].

PMPDP Module 8

There are certain limitations to both qualitative and

quantitative approaches for measuring the eectiveness
of a professional development programme. The qualitative study carried out in the rst survey aimed to measure
the eectiveness of PMPDP using the ROI methodology
[18]. The ROI model comprises 4 levels and demands
complete evaluation of a management development programme by using appropriate research instruments (questionnaires, focus groups and interviews) at every level.
This research project is ongoing and at this stage only
one questionnaire has been fully developed; primarily
investigating level 4 of the ROI model (business results).
Secondly, the model suggests the inclusion of statements
and estimations from delegates, graduates, supervisors,
line mangers, customers and experts. However, the training evaluation undertaken in this survey is based on the
view points of delegates only. However, the data appear
to be rich. Delegates are those individuals who can see
rst hand the positive and negative aspects of the professional development programme. Therefore, the opinions
of this group can be considered meaningful in measuring
the eectiveness of the PMPDP.
The small sample sizes are a limiting factor. However,
this research is ongoing and larger sampling will be conducted. The interim ndings of survey 1 are based on 21
respondents representing 25 per cent of the working population. The results of survey 2 are statistically veried
through chi-squared analysis. However, there are limitations to this study, due to the very nature of quantitative

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

research techniques and the RR-KEQ itself. The RR-KEQ

is a facilitated tool and therefore validity and reliability are
quite strong for this type of instrument. Bias may be positive when the employee is over-condent and assigns him/
herself a higher score, or negative due to negative attitudes
towards work. In survey 3, 17 per cent of the RR-KEQs
submitted were facilitated by managers and 7 percent by
peers.
21. Conclusions
The mode of delivery of the case study programme
(PMPDP), described in this paper, is not complex. A
combination of intensive residential events coupled with
traditional and web enabled blended learning appears
to provide a robust learning and professional development platform. The linkage between all the stakeholders
of the programme: students (delegates), their line managers, senior managers, academic sta and academic related
support sta is maintained through a project managed
approach to this academic-industry partnership. The single point of contact between the industrial consortium
and the university provider is an important feature of
this, together with the inuence of an active steering
group with delegate representation. During the inception
and scope denition stages of the project team building occurred as a result of a common purpose with
respect to the development of learning outcomes arising
from industry led objectives for the programme. Arguably, this is unusual in comparison to the normal relationship between academia and industry in which the
investment of specialist project management resource
from industry whilst possible may not usually be made
available due to considerations of cost and project management capability from a university provider, although
desirable, is frequently impossible. The project managed
stability of project management development does not
come cheap and eectiveness must be measured. The
programme requires the understanding and management

235

of expectations for all stakeholders in the context of a

partnership.
The serious consideration of Benet Metrics (Return on
Investment) is only possible if both qualitative and quantitative sensitive data are available. This is not simple and
requires signicant resources. Therefore, without project
managed partnerships, trust and commitment it is dicult
to see how this could be achieved.
The general trend in all the three surveys indicates that
PMPDP is contributing positively in increasing the knowledge of delegates. It appears from the analysis that the PM
knowledge of employees participating in the PMPDP has
gradually increased as they study more PMPDP modules.
This is found true for all managerial positions.
It is further concluded that studying the PMPDP does
have a positive eect on project managers knowledge
and their proles in relation to the IPMA levels
(A,B,C,D). Although, it cannot be determined that the
PMPDP actually causes an improvement in the knowledge
of employees, it appears from the analysis that employees
attending the PMPDP would nish with a higher knowledge level compared to when they started. Hence, it can
be deduced that professional development programmes like
PMPDP provide an eective way of ensuring improvements in employee competence. It is further observed that
the PMPDP does successfully address most of the identied
areas requiring educational focus in particular project
management areas.
It appears from the results of all the three surveys that
professional development programmes like the PMPDP
can contribute to the development of competence and
competencies in individuals, leading to benets for the
organisation as a whole. The results are in accordance
with the views of a number of published articles on this
subject that professional development programmes do
play a signicant part in improving the competence of
employees. However, they also contradict other views [5]
that competence is dicult to quantify and cannot be
measured.

236

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

Appendix A. Rolls-Royce Knowledge and Experience

Questionnaire (RR-KEQ) sample [37]

M. Alam et al. / International Journal of Project Management 26 (2008) 223237

References
[1] Avots I. Why does project management fail? Calif Manage Rev
1969;12:7782.
[2] Munns AK, Bjeirmi BF. The role of project management in achieving
project success. Int J Project Manage 1996;14(2):817.
[3] Project Management Institute. A Guide to the Project Management
Body of Knowledge, PMI, 1996.
[4] Dixon M. APMBoK, APM UK., 2000.
[5] Crawford L. Proling the Competent Project Manager. Proceedings
of PMI Research Conference, 21-24 June, Paris, France, 2000, p. 315, PMI.
[6] Atkinson R. Project Management: cost, time and quality, two best
guesses and a phenomenon, its time to accept other success criteria.
Int J Project Manage 1999;17(6):33742.
[7] Morris, P.W.G. and Hough, G.H. The preconditions of success and
failures in major projects (Technical Paper No.3). Oxford: Major
Projects Association, 1986.
[8] Lim CS, Mohamed ZM. Criteria of project success: an exploratory
examination. Int J Project Manage 1999;17(4):2438.
[9] Shenhar AJ, Dvir D, Levy O, Maltz AC. Project success: a multidimensional
strategic
concept.
Long
Range
Planning
2001;34(6):699725.
[10] Pinto JK, Slevin D. Project Success: Denitions and Measurement
Techniques. Project Manage J 1988;19(1):6772.
[11] Freeman M, Beale P. Measuring project success. Project Management
Journal 1992;23(1):817.
[12] Shenhar AJ, Levy, et al. Mapping the dimensions of project success.
Project Manage J 1997;28(2):513.
[13] Baccarini D. The logical framework method for dening project
success. Project Manage J 1999;30(4):2532.
[14] Wang X, Huang J. The relationships between key stakeholders
project performance and project success: Perceptions of Chinese
construction supervising engineers. Int J Project Manage
2006;24:25360.
[15] Rowden R, editor. Workplace learning: Debating ve critical
questions of theory and practice. San Francisco: Jossey-Bass; 1996.
[16] Daley JB. Learning and Professionals Practice: A study of four
professions. Adult Education Quarterly 2001;52(1):3954.
[17] Hinds J. Professional Development. Financ Times 2004:1.
[18] Phillips J. Return On Investment in Training and Performance
Improvement Programmes. Burlington, MA, USA: Elsevier Science.;
2003.
[19] Kirkpatrick DL. Techniques for evaluating training programmes.
Training Dev J 1979;33(6):7892.
[20] Thiry M. How can the benets of PM training programmes be
improved? Int J Project Manage 2004;22:138.

237

[21] Gale AW. Competencies: Organizational and Personal. The Wiley

Guide to Managing Projects. New Jersey, USA: John Wiley and Sons,
Inc; 2004.
[22] Black R. and Schell J. Learning within a situated cognition
framework: Implications for adult learning.Paper presented at the
American Vocational Association Convention, Denver, CO. (ERIC
Document Reproduction Service No. ED389 939), 1995.
[23] Eraut M. Developing professional knowledge and competence.
Washington DC: Falmer; 1994.
[24] Grzyb S. Eects of organizational role and culture on participation in
continuing professional education. Paper presented at the Annual
Meeting of the American Educational Research Association, Chicago, IL. (ERIC Document Reproduction Service No. ED409 284),
1997.
[25] Alam M. Benet Metrics Research Project: 1st Year Progress Report
(unpublished), The University of Manchester UK. 2006, 268.
[26] Rowe C. Assessing the eectiveness of Open Learning: The
British Aerospace Experience. Ind Commercial Training
1994;26(4):227.
[27] Boverie P, Mulcahy DS, Zondlo JA. 1995. Evaluating the eectiveness of Training Programmes. [Online] Retrieved 251206, from
http://www.zondlo.com/access/eval.htm.
[28] Gale AW, Brown M. Project Management Professional Development: An industry led programme. J Manage Dev
2003;22(5):41025.
[29] Song S. 2006. Developing Project Managers in China (PhD Thesis).
MACE, Manchester UK, The University of Manchester.
[30] Spencer LM, Spencer SM. Competence at work. NY: John Wiley &
Sons; 1993.
[31] IPMA. International Project Management Association: International
Competence Baseline (version 3). ISBN 0-9553213-0-1, IPMA, 2007.
[32] Crawford L. Senior Management perceptions of project management
competence. Int J Project Manag 2005;23:716.
[33] Alam M. Rolls-Royce Knowledge & Experience Questionnaire
Analysis 2005 (Research Report), (unpublished), The University of
Manchester UK, 2006.
[34] QAAHE. Handbook for Academic Review, Appendix D. Quality
Assurance Agency for Higher Education, available at www.qaa.org.uk., 2002.
[35] Tsopela, M. Return On Investment of the Project Management
Professional Development Programme (PMPDP) (MSc dissertation).
MACE. Manchester UK, The University of Manchester, 2003.
[36] Statter, C. 2004. The development of functional and behavioural
competence in relation to success amongst project managers (Undergraduate Research Project). MACE. Manchester UK, UMIST.
[37] Piling N. Rolls-Royce project management competency assessment
tool. Derby, UK: Rolls-Royce; 2006.