Baklizky2017 PDF

Business Process Management Journal
Business process point analysis: survey experiments

Maruscia Baklizky, Marcelo Fantinato, Lucineia Heloisa Thom, Violeta Sun, Patrick C.K. Hung,
Article information:
To cite this document:
Maruscia Baklizky, Marcelo Fantinato, Lucineia Heloisa Thom, Violeta Sun, Patrick C.K. Hung, (2017)
"Business process point analysis: survey experiments", Business Process Management Journal, Vol.
23 Issue: 2,pp. 399-424, doi: 10.1108/BPMJ-03-2016-0061
Permanent link to this document:
http://dx.doi.org/10.1108/BPMJ-03-2016-0061
Downloaded by KAI NAN UNIVERSITY At 12:38 08 April 2017 (PT)
Downloaded on: 08 April 2017, At: 12:38 (PT)

References: this document contains references to 30 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 42 times since 2017*
Users who downloaded this article also downloaded:
(2017),"Exploring the intersection of business process improvement and BPM capability
development: A research agenda", Business Process Management Journal, Vol. 23 Iss 2 pp. 275-292
http://dx.doi.org/10.1108/BPMJ-05-2016-0095
(2017),"Mapping knowledge networks for organizational re-design in a rehabilitation clinic",
Business Process Management Journal, Vol. 23 Iss 2 pp. 329-348 http://dx.doi.org/10.1108/
BPMJ-01-2016-0028
Access to this document was granted through an Emerald subscription provided by emerald-
srm:584769 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.
*Related content and download information correct at time of download.

The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/1463-7154.htm
Business
Business process point analysis: process point
survey experiments analysis
Maruscia Baklizky and Marcelo Fantinato
University of São Paulo, São Paulo, Brazil
399
Lucineia Heloisa Thom
Federal University of Rio Grande do Sul, Porto Alegre, Brazil Received 20 March 2016
Violeta Sun Revised 27 June 2016
10 September 2016
University of São Paulo, São Paulo, Brazil, and Accepted 20 September 2016
Patrick C.K. Hung

University of Ontario Institute of Technology, Oshawa, Canada
Abstract
Purpose – The purpose of this paper is to present business process point analysis (BPPA), a technique to
measure business functional process size, based on function point analysis (FPA), and using business
process model and notation (BPMN). This paper also discusses the assessment results of BPPA compared
with FPA.
Design/methodology/approach – Two experimental studies with participants from academia and
industry were conducted. The following aspects in the experimental studies were focused: similarity,
application easiness, feasibility, and application benefits. The purpose of the experiment was to assess BPPA
comparing with FPA as the BPPA design followed the FPA pattern.
Findings – Experimental results showed that both academia and industry groups highly rated similarity
and application benefits for BPPA compared with FPA. However, only participants from industry highly
rated BPPA for application easiness and feasibility. The results also showed that participants’ previous
experiences did not influence their ratings on BPPA.
Originality/value – BPPA helps project managers to measure functional process size of business process
management projects. As BPPA is derived from FPA, its mechanism is easily recognizable by project
managers who are used to FPA. These results also show that both techniques are in most cases considered
rather similar.
Keywords Business process management, Business process modelling, Metrics, Experimental studies,
Function point analysis, Functional size measurement
Paper type Research paper
1. Introduction
With the increase in the adoption of business process management (BPM) by companies
(Thom et al., 2009; Weske, 2007; Dumas et al., 2013), new metrics were proposed to help in
control and management of BPM projects. A good project management requires guidance
and control of the artifacts produced in each phase (Project Management Institute (PMI),
2008). Therefore, estimations are needed for project planning and control, where the
following four aspects have to be considered: required resources, effort, cost, and time
(PMI, 2008). also applies to BPM projects. The majority of metrics proposed for BPM
comes from software engineering (González et al., 2010) and are based on parallels
established between software applications and business processes (Baklizky et al., 2013).
Nevertheless, functional complexity techniques are still a challenge in BPM.
Business Process Management

This work was supported by the São Paulo Research Foundation (Fapesp), Brazil, under Grant Journal
Vol. 23 No. 2, 2017
No. 2015/16615-0; the Ministry of Science and Technology (MOST), Taiwan, under MOST Grants pp. 399-424
Nos 105-2923-E-027-001-MY3 and 105-2221-E-027-113; and the Natural Sciences and Engineering © Emerald Publishing Limited
1463-7154
Research Council of Canada (NSERC), under Discovery Grants Program No. RGPIN-2016-05023. DOI 10.1108/BPMJ-03-2016-0061
BPMJ In software engineering, functional size enables the derivation of required resources, effort,
23,2 cost, and time (Sommerville, 2004). Techniques that rely on project functional size calculation are
of functional size measurement type, for which function point analysis (FPA) is the most widely
used technique. FPA was first proposed in 1979 and has been continuously enhanced by the
International Function Point Users Group (IFPUG) (2010). FPA is used in software projects to
estimate their functional size using the function point metric. Similarly, BPM projects need a
400 metric to produce valuable project size information – just as functional sizes are calculated for
software projects – in order to derive other important variables to manage BPM projects.
An extension of the function point metric and FPA technique could be helpful for BPM.
A mapping between software engineering and BPM domains provided knowledge for our
study. Fantinato et al. (2012) compared activities and outcome artifacts of each phase from
both approaches. They found that both domains are very similar in relation to phases,
artifacts, and procedures. The differences mainly relied on the content of their artifacts.
The lifecycles of both software engineering (Sommerville, 2004) and BPM (Weske, 2007)
comprise of the phases presented in Table I.

In software engineering, software projects mainly focus on developing new software
products (Phases 1.1-2.2) and also addressing issues such as operation and maintenance
(Phase 3). In BPM projects, lifecycle is shorter and usually faster and more dynamic during
development (Phases 1 and 2). However, in BPM, a greater importance is given to
post-development phases (Phases 3.1 and 3.2). According to Fantinato et al. (2012), “a given
business process model typically evolves far more often than a given software in the same time.”
Function points and FPA are not directly applicable to BPM projects as they refer to
information only available in the software project documentation and not in the
documentation expected for BPM projects. An adaptation of FPA was recently proposed to
the BPM context (Baklizky et al., 2013). The proposed technique, named business process
point analysis (BPPA), is targeted to business process models in business process model and
notation (BPMN) whereas the original FPA is targeted to the software requirements produced
to drive a software development. BPPA enables functional process size calculation (equivalent
to FPA’s functional size). FPA is a technique to be used in the software engineering domain
while BPPA is a technique to be used in the BPM domain. Thus, BPPA was not proposed
aiming to improve FPA; BPPA was proposed to be a FPA-like technique but to be used in
another domain. FPA was used only as the conceptual framework, i.e., a model, a good
practice to be followed also in BPM. Our intention was to propose a technique that could be
easily applied by project managers, who were used to FPA, when working with BPM.
In the BPPA first version, only development projects were addressed, i.e., projects of business
process automation, with a process model as an input. Application and enhancement projects
(i.e. projects related to calculation of the process size of an automated process already in use or of
a new part to be automated) are still not addressed in the BPPA first version, though existing in
FPA. Although BPPA is based on FPA, the use of BPPA does not require the use of FPA.
Business process point (BPP), that represents a numeric abstraction of the functional
process size, is the base metric. Similarly to function points, a BPP is a “measurement unit”
BPM lifecycle phases Software engineering lifecycle phases
1. Design and analysis 1.1. Requirements specification

1.2. System and software design
Table I. 2. Configuration 2.1. Implementation and unit test
Mapping between 2.2. Integration and systems test
software engineering’s 3. Operation and maintenance 3.1. Enactment
and BPM’s lifecycles 3.2. Evaluation
that expresses the amount of business functionality a business process (as a product) Business
provides to a user. BPPs are used to measure a business process functional size. It can be process point
used to derive, for example, a projects’ estimated cost (either in money or in hours); a single analysis
BPP cost is calculated either from past project experience or from shared reference values.
BPP calculation considers a series of systematic rules. These rules were designed to
represent the functional process size not only based on the number of elements of a process
model, but also considering the different types and features of the elements. 401
Taking as an example, the business process model called “The travel booking” may
represent a business process that some company may need to automate. This business
process covers the activities comprised of a travel agency receiving a travel reservation
request from a client to the booking being successfully made, including several details such
as seat selection and payment as well as error handling. The corresponding business
process model has several tasks, events, gateways – all of them, of different types and at
different complexity levels. A project manager would need to estimate the resources, cost,
and schedule of the process automation project. BPP could be used to support such
estimates. For example, assuming that this business process model has 70 BPP (based on its
elements and characteristics), this value could be used to estimate the total number of hours
necessary for the whole project. To this end, the two approaches could be used (following
the example from software engineering, based on function points). In the preferable
scenario, the company maintains a historical project database which indicates, for example,
that each “1 BPP” is equivalent to “40 hours” (on average) and hence one would need 2,800
hours to carry out the illustrative project. As an alternative, if the company does not count
on a historical database, then it can use open or public databases in order to use the others’
experience while it builds its own database.
According to González et al. (2010), a very reduced set of business process metrics was
validated. This paper presents the BPPA technique and the results of assessing
experimental studies. The following aspects were investigated: similarity, application
easiness, feasibility and application benefits, on BPPA, compared with FPA. We chose
these four aspects since the main purpose of BPPA was to keep compliance and
meaningfulness in relation to the original FPA, as previously performed in a similar study
of other domain (Gonçalves et al., 2011). Participants from two groups, academia and
industry, were involved. Our findings showed that, in general, BPPA was highly rated in
both groups compared with FPA. Since BPPA is an adaptation of the FPA technique,
which provides several benefits for software engineering, we expect that BPPA may work
similarly for BPM. Although there were some specific divergences in the results, we
considered the general outcomes satisfactory. This is a preliminary assessment since
other aspects rather than the four addressed here should also be considered, in order to
validate the proposed BPPA technique.
The remainder of this paper presents the following topics: related work, BPPA proposal,
an example of BPPA application, planning and achieved results of two experimental studies
undertaken to assess BPPA, main lessons learned, and conclusions and future work.
2. Related work
Most initiatives regarding BPM metrics are related to adaptions from software engineering
(González et al., 2010), which include only some size-related metrics, as addressed in our
work. In particular, Gruhn and Laue (2006) point out specific software complexity metrics
that are extensible to analyze the complexity of business process models, including
cognitive complexity metric and fan-in/fan-out metric. Among the extensions proposed by
Gruhn and Laue (2006), they address only one BPM size-related metric – number of
activities (NoA), which is based on the software engineering lines of code metric. More
specifically, none of the extensions proposed by them is based on functional size
BPMJ measurement, as it is proposed in our work. Specifically on size-related BPM metrics,
23,2 Dhammaraska and Intakosum (2009) provide guidelines on how to measure business
process models using use case points (UCP), a metric derived from function points. BPMN
models are transformed in use case descriptions that are then measured using UCP.
In contrast to our approach, these authors attempt to adapt process models to the software
engineering context by rewriting them as use case descriptions, which can cause loss of
402 process models’ properties. Variations in the elaborated use case descriptions can lead to
different resulting metrics to a same business process, making it unreliable.
Regarding metrics designed to directly measure the process size, Kaya and Demirörs
(2011) describe an approach to apply the COSMIC functional size measurement method to
software projects that use BPMN to model software functions. Although their approach
uses BPMN, it is restricted to the software engineering context, where the software
behavior can be modeled using BPMN to represent its internal processes. This means that
their approach does not address the BPM domain, i.e., the automation of business
processes, as proposed in our study. In addition, this method is based on the COSMIC
method and not on the IFPUG’s FPA method, as proposed in our study. More recently,
Viegas and Toledo (2013), Marín and Quinteros (2014) and Salmanoǧlu et al. (2014) also
proposed methods to measure software, considering the software internal processes.
As modeling technique, the first uses no technique in particular, the second uses BPMN
and the third is based on subject-oriented BPM). Moreover, the last two are COSMIC-
related metrics. In a more recent study, Aysolmaz and Demirörs (2015) present a tool to
model business processes related to development of process-aware information system
that automatically generates COSMIC-based software size estimation. We found no
approach based on the standard IFGUP’s FPA as well as using BPMN as modeling
technique to measure process size, such as proposed in our work.
Regarding BPM metrics assessment, we found four works reporting empirical
experiments aiming to assess new metrics and measurement techniques for BPM (Cardoso,
2006; Vanderfeesten et al., 2008; Dumas et al., 2012; Sadowska, 2015). Cardoso (2006) applied
an empirical validation to 19 students with working experience, from the first-year of a
computer science master program with no knowledge of BPM, in order to assess his control-
flow complexity metric, based on McCabe’s cyclomatic complexity. The students had a
training session on BPM and had to apply it in 22 business processes of a bank loan
application. The goal of the experiment was to determine whether any correlation existed
between subjects’ ratings and the proposed metric. Vanderfeesten et al. (2008) conducted an
empirical assessment based on error prediction and understandability in order to assess the
cross-connectivity metric proposed to measure the strength of links among process model
elements. They used correlation analysis and multivariate logistic regression to verify the
error prediction. To validate the understanding, 73 students filled out a questionnaire with
12 process models where they were supposed to answer execution-related questions.
The goal of the experiments was to measure the quality of the new metric in variations of
errors and understandability.
More recently, Dumas et al. (2012) analyzed benefits of structured processes vs
unstructured ones, including an experiment with students exposed to semantically
equivalent unstructured and structured processes. They ran the experiment with
110 students who had knowledge of BPMN, from two business process modeling courses.
The students had to answer six questions about the content of eight process models selected
from an IBM library. The goal was to investigate whether the structure level of process
model influenced its understanding. Finally, Sadowska (2015) analyzed the accuracy of
quality metrics related to BPMN processes. Her proposal was assessed preliminarily for
usefulness through a survey-based experiment, from the view of 14 BPMN experts.
The results showed that the model worked in most cases.
3. BPPA Business
This section gives a brief description of the proposed BPPA technique, based on IFPUG’s process point
FPA, and highlights the most important decisions required to create this functional size analysis
measurement technique for BPM projects. The method to set BPPA consists of mapping out
all FPA stages and steps to corresponding BPPA stages and steps. We aimed to keep the
basic FPA structure and make changes only when adjustments are necessary in order to
consider business process properties instead of software properties. Examples of necessary 403
adjustments are to consider BPMN elements (as BPMN tasks) to identify elementary
processes instead of considering the software’s smallest units of activity. The details of
what BPMN elements were used in spite of software elements are presented later in this
section. Although BPPA is based on FPA, applying BPPA does not depend on also applying
FPA, so that an organization is able to use BPPA even when it does not use FPA. However,
knowledge of FPA concepts is useful for better understanding of BPPA concepts.
We used a technical report to present a methodical description of BPPA (Baklizky and
Fantinato, 2012). Figure 1 shows the BPPA diagram and the levels of changes from the FPA
diagram. The next subsections describe the stages presented in the diagram (along with
their “steps”) as well as the comparison with FPA stages and steps.
3.1 Stage 1 – Gather available documentation

As a reference for this first BPPA stage, in FPA, all available documentation for functional
size measurement is collected. This documentation has to describe system’s functional
requirements and consider what is important for the user. One example is user’s
requirements specification. If relevant, extra artifacts can be useful as support documents,
such as entity diagrams and used cases (IFPUG, 2010). In BPPA, the functional size
measurement documentation is a set of business process models in BPMN (or simply
“BPMN models”). Since this is the main type of artifact used to represent functional
Stages: Steps:
1. Gather available
documentation
2. Determine counting
scope and boundary, and 2.1. Establish 2.2. Determine 2.3. Determine 2.4. Determine the
identify functional purpose of the type of count counting scope boundary
stakeholder requirements count
3.1.2. Classify 3.1.3. Determine 3.1.4. Determine

3.1. Measure data 3.1.1. Identify the
each data funtion the ILFs and EIFs the ILFs and EIFs
funtions data functions
as an ILF or EIF complexity contribution
3.2.3. Classify
3.2.1. Identify each 3.2.2. Determine 3.2.4. Determine 3.2.5. Determine
3.2. Measure each transactional
elementary process unique elementary function as an EI, the Els, EOs and the Els, EOs and
transactional functions
processes EQs complexity EQs contribution
EO or EQ
4. Calculate process size
Key:
Numerous changes
5. Document the business
process point count Few changes
Almost no changes
Figure 1.
6. Report the result of the Stages and steps of
business process point
count
BPPA technique
BPMJ stakeholder requirements in a BPM project, some extra artifacts can also be useful, such as
23,2 activity textual descriptions used to complement BPMN models. When functional
requirements are not represented in BPMN models, BPPA does not apply.
3.2 Stage 2 – Determine counting scope, boundary, and functional stakeholder requirements
The following subsections describe the four steps for this second stage of BPPA.
404 3.2.1 Step 2.1 – establish the purpose of the count. A count starts in response to business
requirements, with purpose determined by the business problem (IFPUG, 2010). FPA and
BPPA take the business scenario as a rule. Although the purpose influences the boundary line
between software products in FPA, the equivalent influence on BPPA is between process
models. For example, a BPP count is an input of the functional process size to configure a
BPM system that is an automated support for the enactment of business processes.
3.2.2 Step 2.2 – determine type of count. The type of count relies on the purpose defined
in the previous step. FPA includes three types of count: development, application, and
enhancement projects, which vary according to the phase where the software is measured
in the software lifecycle (IFPUG, 2010). This first version of BPPA addresses one type of
count: development project, where the “Configuration” phase (refer to Table I) is the only
target for the purpose of the count. In this case, we aim those BPM projects which are
related to automate a business process that have a process model as an input. We chose
the development project type to be part of the first version of BPPA as it refers to the most
often used as well as the most complete type in FPA. Future versions of BPPA should
address application and enhancement projects – since the “Enactment” and “Evaluation”
(Table I) phases may be targets for the purpose of the count. Moreover, the future
versions should also address an additional project type not addressed even by FPA:
non-automation projects, i.e., BPM projects of business process modeling not intended to
process automation (e.g. process models for improving or restructuring an organization).
Non-implementation projects are not addressed by FPA since, in software engineering, it
is not common to model a software without a clear intention to implement it, although this
is common in BPM.
3.2.3 Step 2.3 – determine count scope. In FPA, the count scope defines the groups of
functional user requirements in the function point count (IFPUG, 2010). In BPPA, the BPP
count scope is the group of BPMN models. Since they describe the interactions between two
or more business entities and represent the activities for a business process, they also
represent functional stakeholder requirements in BPM projects. Similarly to function point
count scope, BPP count scope:
(1) defines the BPM project to be measured;
(2) is determined by the purpose of the BPP count; and
(3) identifies the processes to be included in the functional process size and provides
relevant answers to the purpose of the count.
In this version of BPPA, only the BPMN collaboration diagram type is addressable, since it
is, according to the best of our knowledge and experience, the most widely used in
organizations that represent process models. This means that BPMN conversation
diagrams and BPMN choreography diagrams are not addressable in this BPPA version.
3.2.4 Step 2.4 – determine the boundary. For FPA, the boundary is the conceptual
interface between the system under study and its users (IFPUG, 2010). In BPPA, the
boundary is the conceptual interface between the main participants of BPMN models and
the other participants. This BPP count boundary must:
(1) indicate the limit between the main and other participants in a BPMN model;
(2) define the actions and data elements under the responsibility of the main participant; and Business
(3) act as a “membrane” through which data are processed by transactions. process point
The boundary can overlap the area of a pool or specific swimlanes (Business Process analysis
Management Initiative (BPMI), 2011) in the measured BPMN models. In addition, BPMN
models are based on a business view (BPMI, 2011). It is useful to represent the main
participant of a business process and its collaborations with other participants.
The modeling view is one of the determining factors in the process of identifying the
405
boundary, as it influences the interpretation of the modeled business process and highlights
needs, responsibilities, and data required by the main participants.
The procedure to determine the boundary involves:
(1) identifying all the participants in the BPMN model;
(2) checking the modeling view used for the BPMN model being measured;
(3) defining the main participant of the BPMN model, on basis of the previous task; and
(4) defining the boundary, in accordance with the three guidelines defined.
3.3 Stage 3 – measure functions

In this stage, two types of functions are measured: data and transactional functions.
3.3.1 Stage 3.1 – measure data functions. Data function is the first of the two central
items for both FPA and BPPA. Data functions satisfy one or more data action requirements
including data reading, writing, input, output, reception, or transmission within the scope of
a BPMN model. In BPPA, data functions can be internal logical files or external interface
files, as in FPA. The following subsections describe the steps for measuring data functions:
3.3.1.1 Step 3.1.1 – identify the data functions. In FPA, a data function is a set of logically
related data, i.e. data stored persistently for further retrieval, update, and reference.
The amount of logically linked data is grouped in data functions (IFPUG, 2010). In BPPA,
data functions are complex data groupings. Generally, data elements are represented in
BPMN models in a higher granularity level than those directly classifiable as data functions.
As a result, all the data elements found in the BPMN models are initially only classified as
data structures. Each data structure represents a data element processed by one or more
activities or a data element received or transmitted by message flows. The existing data
structures have to be identified and classified to be useful in the next steps. Data structures
are classified based on both representation and complexity. The classification of
representation relies on the type of the element.
Data structures can represent three types of BPMN elements: data object (including
collection, input, and output subtypes), data store, and event message. Regarding the
classification of complexity, the decision depends on whether the data structure is already a
data group or not. For the classification of representation, data objects and event messages
represent volatile data, i.e. data only used in a business process instance. Data stores
represent persistent data, as well as data permanently stored regardless of the enactment of
a specific business process instance. Moreover, data objects and data stores are “directly
represented as data elements” since they are originally designed for this purpose, whereas
the event messages are “indirect represented as data elements” since they also represent
data, although their main purpose is to carry out a message exchange.
The procedure to identify the data functions is:
(1) identify all data structures in the BPMN models scope;
(2) classify each data structure according to its representation and complexity types;
BPMJ (3) name each data structure;
23,2 (4) arrange functionally related and non-complex data structures in different groups, by
taking their representation types as a constraint – data structures that are directly
represented cannot be merged with data structures that are indirectly represented,
i.e. the original representation types are kept in the resulting groups;
(5) count the number of data structures grouped in each resulting group of data
406 structures;
(6) count the number of relationships (associations or flows) linking each data structure,
from the group of data structures, to the activities of the BPMN models; and
(7) classify each resulting group of data structures as a data function.
3.3.1.2 Step 3.1.2 – classify each data function as an internal logical file or external interface
file. In FPA, a data function is classified as an internal logical file or an
external interface file depending on its primary intent and its relative position to the
function point count boundary (IFPUG, 2010). For BPPA, a data function is also classified as
an internal logical file or external interface file depending on its primary intent and the
boundary. However, in BPPA, the types of the data structures compose the data function
based on the following rules:
(1) only data objects within BPP count boundary are included, since data objects
outside this scope do not represent any development effort in a BPM project that
considers the perspective of the main participant;
(2) event messages outside the BPP count boundary are included, since they are useful
to model a communication type between the main participant of the BPMN models
and the other participants; thus, they represent a development effort that also
includes the perspective of the main participant; and
(3) data stores outside the BPP count boundary are included in case they are associated
with activities within the boundary, since they represent the persistent data by the
main participant.
Based on these conventions, the primary intent shows:
(1) the primary intent of an internal logical file is to represent a data function which is
referenced through an association with an activity within the BPP count boundary and
(2) the primary intent of an external interface file is to represent a data function which is
referenced through an association with an activity outside the BPP count boundary.
3.3.1.3 Step 3.1.3 – determine the internal logical files and external interface files
complexity. In FPA, data function complexity is defined by specific information from the
functional software context, not available in the BPM domain. In BPPA, data function
complexity relies on the number of interrelated data structures and the number of
relationships between data function and activities. Interrelated data structures and
relationships between data function and activities are calculated in Step 3.1.1. Interrelated
data structure refers to the number of grouped data structures that form each data function.
Relationships between data function and activities refer to the number of relationships
linking data structures to activities, in each data function. Finally, both values are related
and the complexity of each data function can be defined based on the data in Table II(a).
3.3.1.4 Step 3.1.4 – determine the internal logical files and external interface files
contribution. Using the same FPA values, the size of each data function is calculated via
Table II(b).
3.3.2 Stage 3.2 – measure transactional functions. Transactional function is the second of Business
the two central items for both FPA and BPPA. For BPPA, transactional functions satisfy process point
one or more transactional action requirements such as data processing, validation, and
management in a BPMN model scope. Transactional functions can be external input (EI),
analysis
external output (EO), or external query (EQ). The following subsections describe the steps
for measuring transactional functions.
3.3.2.1 Step 3.2.1 – identify each elementary process. In FPA, elementary processes form 407
transactional functions, i.e. small actions, both atomic or non-atomic, which are grouped to form
a transaction. Similarly, BPPA only requires considering activities in the BPP count boundary
to identify elementary processes. Activities outside this scope do not represent any
development effort during a BPM project that relies on the perspective of the main participant.
Activities represent actions carried out during the enactment of an instance of a BPMN model
and are a generic term for the work that organizations perform in a process. An elementary
process is the smallest set of interrelated activities within a BPP count boundary meaningful to
its stakeholders. Elementary processes are identified before identifying transactional functions.
The following BPMN elements are included in the elementary process identification:
activities (task, transaction, event sub-process, call activity), activity markers (sub-process, loop,
parallel MI, sequential MI, ad hoc, compensation), task types (send, receive, user, manual,
business rule, service, script), gateways, events and sequence flows. Activities can be classified
by degree of complexity defined as follows: atomic activities (e.g. tasks, including the different
task types) are classified as “simple activities”, and non-atomic activities as “complex activities”
(i.e. all other activity types, which are not simple tasks). “Complex activities” are classified as
elementary processes, whereas “simple activities” are grouped to form an elementary process.
The procedure to identify elementary processes is:
(1) identify all activities in the BPP count boundary;
(2) categorize the activities concerning their complexity level (simple and complex);
(3) identify the task type for simple activities and the activity marker for complex activities;
(4) identify links between activities and gateways, and activities and events;
(5) highlight the sequence where the identified activities, gateways and events are
linked through sequence flows;
(6) identify the relations between activities and data functions;
(7) group one or more simple activities when semantically dependent, to form a complex
activity;
(8) classify each resulting complex activity as an elementary process; and
(9) count the number of grouped activities that form the resulting elementary processes.
(a) (b)
RDAs Type
1-19 20-50 W 50 ILF EIF
RDSs Process complexity

1 Low Low Average Low 7 5
2-5 Low Average High Average 10 7
W5 Average High High High 15 10 Table II.
Notes: (a) Process complexity of data functions; (b) Data functions size in BPP. RDA, relationships between data Process complexity
functions and activities; ILF, internal logical file; EIF, external interface file; RDS, interrelated data structures and size in BPP of
Source: Adapted from IFPUG (2010) data functions
BPMJ 3.3.2.2 Step 3.2.2 – determine unique elementary processes. As in FPA, before identifying
23,2 the resulting transactional functions for BPPA, a uniqueness test is performed for the
identified elementary processes, since transactional functions must be unique elementary
processes. To verify whether an elementary process is unique within the set of identified
elementary processes, it cannot:
(1) be both formed by a semantically equivalent set of activities; and
408 (2) be related to the same data functions when compared with one or more elementary
processes.
Despite these rules, the elementary processes represent the same process. After checking all
possible combinations of elementary processes, the resulting unique elementary processes
are classified as transactional functions.
3.3.2.3 Step 3.2.3 – classify each transactional function as an EI, EO, or EQ. Each
transactional function identified in the previous step is classified as an EI, EO, or EQ.
For each transactional function, it is necessary to check: the primary intent, the associated
processing logic, and the relationship to BPP count boundary. The following rules drive the
classification process:
• the primary intent of an EI is to represent a transactional function that processes data
received from outside the BPP count boundary, which can be implicitly or explicitly
represented by some data object, data store or event message;
• the primary intent of an EO is to represent a transactional function that sends data
outside the BPP count boundary and involves a prior processing logic to retrieve and
process internally information from data objects, data stores or event messages,
which does not happen to external queries; and
• the primary intent of an EQ is to represent a transactional function that sends data
outside the BPP count boundary through simple information retrieval from a data
object, data store or event message, i.e. no additional processing logic to retrieve or
process internal information is allowed, otherwise it will be considered an EO.
3.3.2.4 Step 3.2.4 – determine the EI, EO and EQ complexity. In FPA, transactional function
complexity relies on specific information from the functional context of the software, which is
not available on the BPM context. For BPPA, data function complexity depends on:
the number of interrelated activities and the value of the transactional function’s weight.
Interrelated activities are calculated during Step 3.2.1 and refer to the NoA grouped to form each
resulting transactional function. Transactional function’s weight is calculated by analyzing the
task types and activity markers of the activities that compose each resulting transactional
function that were identified in Step 3.2.1. By calculating the total weight of a transactional
function, each activity part of the group forming the transactional function provides a different
individual weight depending on its type. The following individual weights are considered:
(1) task types – send (1), receive (1), user (3), manual (0), business rule (2), script (1),
service (3 for services to be implemented and invoked; or 1 for services to be only
invoked), abstract or not defined (2);
(2) activity markers – loop (2), parallel MI (3), sequential MI (3), compensation (3); and
(3) sub-process activities, including transaction, event sub-process, call activity, regular
sub-process, ad hoc – the types of activities forming these sub-processes are
considered to calculate their individual weights.
Finally, both values are related and the complexity of each data function is found using
Table III(a) (for EIs) or Table III(b) (for EOs and EQs).
3.3.2.5 Step 3.2.5 – determine the EI, EO, and EQ contribution. Using FPA values, Business
the size of transactional functions is calculated via Table IV. process point
analysis
3.4 Stage 4 – calculate functional process size
According to IFPUG (2010), “the purpose and count scope shall be considered when
selecting and using the appropriate formula to calculate the functional size,” as this is a
result from the counting process. Since this version of BPPA considers only the 409
development project type, the following formula is used to calculate its functional process
size: PSP ¼ CDF+CTF, where PSP is the functional process size of a given BPM project,
CDF is the total value as a result of the final contribution of all the data functions identified
for the given count scope, and CTF is the total value as a result of the final contribution of
all the transaction functions identified for the given count scope.
3.5 Stage 5 – document the BPP count

Finally, the BPP count can be documented with the following information: the count
purpose and type; the count scope and boundary; the count data; a list of all data functions
and transactional functions identified, including their types, complexities, and number of
BPPs assigned to each one; the final result of the count (Stage 6); and any assumptions made
on the BPMN models, as well as any issues that were resolved.
3.6 Stage 6 – report the result of the BPP count

According to IFPUG (2010), “[…] consistently reporting the results of functional point
counts enables the readers to identify the standard to which they comply”. As the last stage
of BPPA, the report of the results of the BPP count is defined as follows: S BPP (BPPA-v),
where: S is the result of the BPP count (Stage 4); BPP is the size unit of BPPA; and v
represents the used version of BPPA.
(a) (b)
RAs RAs
1-4 5-15 W 15 1-5 6-19 W19
TFW (EI) TFW (EO/EQ)

0-1 Low Low Average 0-1 Low Low Average
2 Low Average High 2-3 Low Average High
W2 Average High High W3 Average High High Table III.
Notes: (a) Complexity of external inputs; (b) complexity of external outputs and external queries. RA, interrelated Process complexity of
activities; TFW, transaction function’s weight; EI, external input; EO, external output; EQ, external query transactional
Source: Adapted from IFPUG (2010) functions
Type
EI EO EQ
Process complexity
Low 3 4 3
Average 4 5 4 Table IV.
High 6 7 6 Size in BPP of
Notes: EI, external input; EO, external output; EQ, external query transactional
Source: Adapted from IFPUG (2010) functions
BPMJ 4. Application example
23,2 This section provides an overview of an illustrative example of the application of BPPA,
an applicability analysis based on the authors’ observations. The scenario chosen is
“The travel booking” process model (BPMI, 2011), a BPM development project composed of
only one BPMN models, presented in Figure 2.
In the business process model (Figure 2), there are ten activities (two sub-processes and
410 eight simple tasks), 12 events (four of which are message events) – excluding those inside
the sub-processes, four gateways – excluding those inside the sub-processes, and sequence
flows between the activities, events and gateways.
The application of BPPA to the travel booking process model indicated a count of
53 BPPs. This value includes the addition of three data functions (two low internal logical
files and one low external interface file) and nine transactional functions (two low EIs,
one high EI, three low EQs, one average EQ, one low EO and one average EO). The number
of BPPs, such as the number of function points, does not represent any information in itself.
On the other hand, it is important in comparative situations (e.g. comparing the number of
BPPs from one project with others and checking whether it requires more effort or not).
In addition, BPP can be useful to plan and manage a BPM project associated to comparative
historical data.
The main result of this application is that BPPA can be applied as designed and
proposed. We used a typical scenario to test the application of BPPA and were able to
calculate the BPP metric and present its use’s feasibility. We discuss some additional results
as follows:
(1) The chosen scenario only includes a small number of data elements that are
observable by comparing the number of data functions with the number of
transactional functions. In fact, we identified only data elements of the “event
message” type and no data object or store. Based on our knowledge, this is a
common picture for business processes modeled through BPMN where no data are
explicitly modeled. Unlike in software engineering, both data and transactions have
almost the same importance in initial representation. In BPM, it is only addressed
the transactional issue related to different types of activities.
(2) In this specific scenario, most of the data functions and transactional functions were
identified and calculated as “low” complexity (75 percent). We believe that this is the
result of using the same factors used in FPA to calculate the data functions and
transactional functions complexities (see Tables II and III). Although this is not
necessarily an issue, typical scenarios in BPM projects are scattered at different
levels of complexity.
(3) In this application, we noticed that, when measuring transactional functions, only
activities are included to compose the final BPP metric, whereas other elements in
the sequence flows, such as gateways and messages, are useful to understand the
context of the problem modeled and to identify elementary processes and
transactional functions. However, there are some implementation efforts based on
these two types of BPMN elements in the business process development phase.
As discussed in previous item, there is clearly a need to include these elements in the
transactional function’s weight count as shown in Table III.
5. Experimental studies for BPPA

Validation is one of the most important aspects to describe a measure (Zelkowitz and
Wallace, 1998). Experimentation helps to determine effectiveness of proposed theories
and methods (González et al., 2010). In this context, this section presents the planning and
3
2
5 6
Source: Adapted from BPMI (2011)

analysis
Business
411
process point
Figure 2.
Travel booking
model
business process
BPMJ results of two experimental studies conducted to assess BPPA: the first one includes
23,2 participants from academia and the second one includes participants from industry.
Initially, Section 5.1 presents the studies’ design and Sections 5.2 and 5.3 discuss the results.
Both experimental studies were based on the Goal-Question-Metric approach (Basili et al.,
1994). Accordingly, they were organized as follows:
• Goal: assessing BPPA by comparing it with FPA. BPPA was designed following the
412 same FPA pattern (having FPA as a model, a conceptual framework), including
minimal necessary changes so that the well-known benefits of FPA could be
transferred to BPPA.
• Question: is BPPA similar to FPA in terms application easiness, feasibility, and
potential benefits, since they share the same basic concepts.
• Metrics: opinion from studies’ participants (as defined in next subsections).
We designed these experimental studies bearing in mind that BPPA was not proposed to
present advantages regarding FPA since they are used in different domains, i.e., software
engineering (with respect to FPA) and BPM (with respect to BPPA). Seeing that FPA could
not be directly applied in BPM, we proposed BPPA based on the same principles of
FPA with the necessary changes. Through this premise, we intended to define BPPA
as similar as possible to FPA in terms of “application easiness,” “feasibility” and
“benefits”. Therefore, BPPA was not expected to be better than FPA in any of these
attributes; equal or similar would be satisfactory. The only unsatisfactory scenario would
be to have BPPA worse than FPA, which would demonstrate that our principle-based
design failed.
5.1 Planning of the experimental studies

We carried out two studies to assess BPPA: study I, involving participants from academia;
and study II, involving participants from industry. The initial step of the experimental
analysis consisted of defining a plan including an experimental design. This section
presents the planned steps for both studies. We used the following analysis mechanisms:
descriptive statistics, statistical non-parametric Yates’ χ2 test (a correction of χ2 test used in
case of small populations), and hypothesis test (Student’s t-test). Section 5.2 presents the
results of the two studies.
5.1.1 Study I planning: participants from academia. Study I was conducted in a
controlled environment. It was based on the Goal-Question-Metric approach, aiming to:
• analyzing BPPA for measuring BPMN models;
• with the purpose of comparing it with FPA from software engineering in terms of
basic principles;
• with respect to similarity, application easiness, feasibility, and potential application
benefits;
• according to the viewpoint of the user; and
• in the context of measurement of BPM projects for organizations.
The study was conducted with 28 undergraduate students from the University of São Paulo,
Brazil – all with background in BPM and software engineering – enrolled in a “software
quality” course from an undergraduate program in information systems. All students
enrolled in the course were selected for the study. Participants were distributed in 14 groups,
two people per group, so that they could share experiences in order to simulate a business
environment where analysts usually do not work isolated.
We conducted the experiment as one of the course’s assignments. One of the authors Business
provided a 90-minute FPA and BPPA training section to the participants. Then, the students process point
were encouraged to apply each technique into specific scenarios. They had a four-week time analysis
frame to complete the task, since this is the usual time set for their assignments. During this
time, they could interact with an instructor by e-mail or in person. Finally, we asked the
groups to answer a questionnaire reporting their opinions on their own background in BPM,
BPMN, software metrics and FPA as well as their opinion about BPPA compared with FPA, 413
and then we compared both BPPA and FPA techniques. This survey had a quantitative focus
although modeled across subjective grades (Dumas et al., 2013). For each of the four assessed
aspects (i.e. similarity, application easiness, feasibility and potential application benefits),
one question asked the students to score their opinions on a 0 to 10 scale, where 5 meant
equality between BPPA and FPA, less than 5 meant BPPA proportionally worse than FPA,
and numbers greater than 5 meant BPPA proportionally better than FPA. The application
domains of the experiment were: a human resources web application, for FPA (IFPUG, 2010),
and a BPMN model for choosing the Nobel Prize, for BPPA (Object Management Group, 2010).
The instruments in this study include:
(1) explanatory material on both FPA and BPPA;
(2) two scenario specifications for applying the techniques (one for each technique); and
(3) one questionnaire split in two parts to identify: participants’ experience and
participants’ opinions on BPPA compared with FPA.
Four pairs of null and alternative hypotheses were stated:
(1) Similarity:
H0. BPPA is different from FPA concerning objectives, stages and steps; and
H1. BPPA is similar or equal to FPA concerning objectives, stages and steps.
(2) Application easiness:
H0. BPPA is more difficult to be applied than FPA considering the proposed scenarios
and the provided support material; and
H1. BPPA is as easy as or easier to be applied rather than FPA considering the proposed
scenarios and the provided support material.
(3) Feasibility:
H0. BPPA is less feasible to be applied than FPA, comparing two potential organizational
contexts for software development (for FPA) and BPM (for BPPA); and
H1. BPPA is as feasible as, or more feasible than FPA, comparing two potential
organizational contexts for software development (for FPA) and BPM (for BPPA).
(4) Application benefits:
H0. BPPA provides fewer benefits for BPM than FPA does for software engineering; and
H1. BPPA provides benefits for BPM as much as or more than FPA does for software
engineering.
The independent variables were:
(1) BPPA applied to BPMN models; and
(2) the participants’ levels of education and experience.
BPMJ The dependent variables were:
23,2 (1) similarity between BPPA and FPA;
(2) application easiness of BPPA compared with FPA;
(3) feasibility of BPPA compared with FPA; and
(4) application benefits of BPPA compared with FPA.
414
The activities carried out by the participants in this study included all stages of BPPA as
well as FPA’s. Nevertheless, the main task of the participants was to identify the boundaries
of each count and measure the intrinsic data and transactional functions for each technique.
The procedure followed to perform the experiment was:
(1) taking a class on BPPA and FPA;
(2) receiving from the instructor: the explanatory material (slideshow used in class,
BPPA’s technical report, link for FPA’s detailed information, two measurement
examples) and two scenario specifications for applying each technique;
(3) applying both techniques in the respective scenarios to measure them;
(4) receiving and answering a questionnaire given by the instructor; and
(5) delivering the measurement reports for each scenario as well as the answered
questionnaire to the instructor.
We applied the non-parametric χ2 for independence test to assess whether there were
significant differences among the groups (experienced, median and inexperienced
participants). This test allowed to know if the degree observed for each aspect of BPPA
compared with FPA is dependent or independent of the participants’ experience level. After
the data analysis, we conducted a hypothesis test (a proportion population test) for the
decision of acceptance or denial of the four null hypotheses, by using a significance level of
5 percent. We applied the Student’s t-test.
These actions were set to mitigate the identified validity threats and mitigate inherent bias:
(1) Internal validity: providing same treatment to all groups to avoid different reactions
to different treatments; providing training on necessary concepts by assuming that
few participants had good domain knowledge; and ensuring that the questionnaire
could show the cases when the two members of a group decided to apply each
technique individually (i.e. a simple work split between the group members), since
these participants might not know how to compare BPPA with FPA;
(2) External validity: selection of participants from the end term of the undergraduate
program on information systems, since it was expected that these students had BPM
and software engineering backgrounds, mainly in project management, as well as
potential for industrial practice and adequacy of instrumentation to industrial practice;
(3) Building validity: non-disclosure of the experiment’s hypotheses to the participants
so not to have assumptions on these hypotheses; disclosure that the goal of the
experiment was to assess BPPA and not the participants; disclosure that BPPA was
recently proposed so there could be some shortcomings; and, disclosure that the
responses to the questionnaire would not influence the participants’ grading; and
(4) Conclusion validity: application of the statistical non-parametric Yates’ χ2 test to
assess whether there are significant differences among the various groups or the
differences are by chance; and application of hypothesis tests to decide on
acceptance or denial of the null hypotheses, using a significance level of 5 percent.
5.1.2 Planning of study II: participants from industry. Study II was conducted with Business
participants from industry. This study was conducted in an independent and process point
complementary way regarding study I, which was carried out with participants from
academia. However, both studies have similar purposes, i.e., they share the same
analysis
Goal-Question-Metrics points previously defined for study I. Therefore, we highlight here
only the differences from study I in order to avoid repeated information.
Study II was conducted in an uncontrolled environment. It was completely conducted 415
in an offline setting, with a quantitative focus modeled through subjective grades.
We selected 13 certified function point specialists, who work with FPA on a daily basis
with background in BPM and software engineering. These professionals were selected
based on one of the authors personal contacts, who has a close relationship with
employees of a metric-specialized company. All those invited to participate in this study
accepted and participated.
We conducted this second study in three steps. First, we presented a summary of the
BPPA proposal to the participants. Then, we provided a detailed description of the proposal
so they could deeply read and then assess it. We gave them two weeks to complete this
assignment, since we could not demand a shorter time due to their professional schedules.
During this time, they could interact with the instructor by e-mail to ask questions. Finally,
we requested the participants to answer a questionnaire reporting their opinions on the
experience, mainly comparing BPPA with FPA.
The instruments were:
(1) explanatory material concerning BPPA; and
(2) one questionnaire split in two parts to identify – participants’ experience and
participants’ opinions on BPPA compared with FPA on similarity, application
easiness, feasibility and application benefits.
We considered the same hypotheses from study I for this study as well as the same
dependent variables. However, we slightly changed the independent variables to:
(1) knowledge of BPPA after evaluating it; and
(2) participants’ experience.
Activities in this study were simpler than in study I due to the professionals’ time
constraints. They had to understand BPPA and answer the questionnaire. The procedure
followed to conduct the experiment was:
(1) receiving from the instructor: the explanatory material (slideshow covering BPPA
and BPPA’s technical report with detailed information);
(2) reading and evaluating the explanatory material related to BPPA;
(3) receiving the questionnaire from the instructor and answering it; and
(4) submitting the answered questionnaire to the instructor.
The non-parametric Yates’ χ2 for independence test and a hypothesis test were conducted in
the same way as in study I. Student’s t-test was also applied.
Actions set to mitigate validity threats in this study were similar to study I, with the
following differences:
• for internal validity – only a slideshow was provided for self-study on BPPA
concepts, with no formal training; in addition, no threat was considered when
comparing both techniques since the questionnaires were always answered
individually;
BPMJ • for external validity – selection of participants with very high knowledge of FPA; and
23,2 • for building validity – disregard the disclosure that the responses to the questionnaire
would not influence on the participants’ grading for that course’s assignment.
5.2 Results of the experimental studies

The results from both studies considered the aspects of similarity, application easiness,
416 feasibility, and application benefits. First, we present a general analysis of the participants’
background and expertise. Then, we discuss the application of Yates’ χ2 test to check the
independency among participants’ experience and the aspects addressed; and, present the
application of Student’s t-test to test the experiment hypotheses.
5.2.1 Results of study I: participants from academia. We collected data on the
participants’ experience in topics related to BPM and metrics from the responses to the first
part of the questionnaire. Figure 3 presents the average grades. The participants had to
score their knowledge in these four topics ranging from 0 to 10 (0 – no knowledge;

5 – medium; and 10 – expert)[1].
Figure 3 presents data in two groups – “All students” and “Only integrated students.”
The latter includes only those where participants informed to have performed all the steps
together whereas the former includes also those groups that split the total work in individual
parts (one student executed the BPPA part; and the other, the FPA part). We investigate the
influence of such a difference in the outcome of the study analysis afterwards.
We collected data from the participants’ opinions on BPPA from the responses of the
second part of the questionnaire. Figure 3 presents these data, regardless the participants’
previous experience. We asked each group to conceptualize their opinion in each topic grading
from 0 to 10, where 5 represents equilibrium between BPPA and FPA and 10 represents the
maximum grade that BPPA could receive comparing with FPA (i.e. representing that BPPA is
better than FPA). We grouped and counted as option “1” answers ranging from 0-3, meaning
that “BPPA is worse than FPA,” as presented in Table V. We grouped and counted as “2”
answers ranging from 4 to 6, which means “BPPA is similar to FPA,” and answers ranging
from 7 to 10, meaning “BPPA is better than FPA”, i.e., “similar or better.”
We linked each observed aspect, presented in Table V, to each of the four domains presented
in Figure 3. These 16 relations were set to verify whether the main results of the study, i.e.,
participants’ opinions, depended on their experience level (i.e. inexperienced (grades 0-3), median
(grades 4-6) and experienced (grades 7-10)). We applied a non-parametric analysis to calculate
the observed and expected frequencies for each aspect, and then we performed the Yates’ χ2 test,
which showed the values presented in Table VI. Considering 6 as the freedom degree and that
the calculated χ2 values are less than the 1 from χ2 table (i.e. 12.59), at a significance level of
5 percent, it was concluded that all four aspects observed in this study are independent of the
participants’ level of experience.
8.00
All Students Only Integrated Students
7.00
6.00
5.00
4.00
3.00
2.00
Figure 3. 1.00
Participants’ 0.00
experience (academia) BPM BPMN Software FPA
metrics
% (only
Business
% (all integrated process point
Aspect observed Answer students) students) analysis
Similarity 1. BPPA is different of FPA 0 0
2. BPPA is similar or equal to FPA 93 100
3. No means to judge 7 0
Application easiness 1. BPPA is more difficult to be applied than FPA 50 80 417
2. BPPA is as easy as or easier to be applied than FPA 43 20
3. No means to judge 7 0
Feasibility 1. BPPA is less feasible than FPA 22 60
2. BPPA is as feasible as or more feasible than FPA 64 40 Table V.
3. No means to judge 14 0 Participants’ opinions
Application benefits 1. BPPA provides fewer benefits than FPA 0 0 regarding BPPA
2. BPPA provides as much as or more benefits than FPA 86 80 compared with FPA
3. No means to judge 14 20 (academia)
Participants’ experience “All students” χ2 “Only integrated students” χ2

Aspect observed topic value value
Similarity BPM knowledge 1.53 0.28

BPMN knowledge 2.75 5.07
Software metrics
knowledge 1.69 0.52
FPA knowledge 1.42 7.10
Application easiness BPM knowledge 2.28 0.28
software metrics
knowledge 1.57 0.52
FPA knowledge 4.98 6.09
Feasibility BPM knowledge 3.20 1.23
Software metrics
knowledge 0.84 0.52
FPA knowledge 2.93 4.12 Table VI.
Application benefits BPM knowledge 1.36 0.42 Relations between
BPMN knowledge 2.05 0.625 participants’
Software metrics experience and
knowledge 11.50 0.42 aspects observed
FPA knowledge 1.60 1.17 (academia)
Finally, we conducted a population proportion test for the four aspects observed to test the
null hypotheses considered in this study. Table VII presents the values calculated by the
test based on the values in Table V, considering a total of 28 participants for “All students”
and ten participants for “Only integrated students” and a probability of 50 percent of
acceptance or denial for each null hypothesis.
Aspect observed “All students” Student’s t-value “Only integrated students” Student’s t-value
Table VII.
Similarity 4.54 3.16 Calculated values of
Application easiness −0.75 −1.90 the population
Feasibility 1.51 −0.63 proportion test
Application benefits 3.78 1.90 (academia)
BPMJ Following the significance value of 5 percent (for the Student’s t-test distribution table), the
23,2 proportion values achieved should be greater than 1.64 so that the null hypotheses were
denied in favor of the alternative ones. For both groups, similarity’s and application benefits’
null hypotheses were denied and application easiness’s and feasibility’s null hypotheses
could not be denied.
5.2.2 Results of study II: participants from industry. Figure 4 presents the average grades
418 concerning the participants’ experience in BPM, BPMN, software metrics, and FPA.
In general, the participants from industry showed more knowledge in the software
engineering field, i.e. software metrics and FPA, compared with the BPM field, i.e. BPM in
general and BPMN language. Whereas the average grade for the software engineering field
was about 8.7, the average grade for the BPM field was 5.5. We investigate the influence of
these data in the outcome of the study analysis afterwards.
We collected data about participants’ opinions on BPPA from the second part of the
questionnaire. Table VIII presents these data, regardless the participants’ previous experience.
We applied a non-parametric analysis to calculate observed and expected frequencies for

each aspect in Table VIII, and then we performed Yates’ χ2 test producing the values
presented in Table IX. Considering six as the freedom degree and that the calculated
χ2 values are less than the 1 from χ2 table (i.e. 12.59), at a significance level of 5 percent, all four
aspects observed in this study are independent of the participants’ level of experience.
Finally, we conducted a population proportion test for the four aspects observed to
assess the null hypotheses. Based on the values in Table VIII and considering the total
number of 13 participants, in addition to a probability of 50 percent of acceptance or denial
of each null hypothesis, the values calculated by the test are presented in Table IX.
10.00
9.00
8.00
7.00
6.00
5.00
4.00
3.00
Figure 4. 2.00
Participants’ 1.00
experience (industry) 0.00
BPM BPMN Software metrics FPA
Aspect observed Answer %
Similarity BPPA is not similar to FPA 0

BPPA is similar or equal to FPA 100
No means to judge 0
Application easiness BPPA is not easier to be applied than FPA 23
BPPA is as easy as or easier to be applied than FPA 77
No means to judge 0
Feasibility BPPA is less feasible than FPA 0
Table VIII. BPPA is as feasible as or more feasible than FPA 92
Participants’ opinions No means to judge 8
regarding BPPA Application benefits BPPA produces fewer benefits than FPA 8
compared with BPPA produces as much as or more benefits than FPA 84
FPA (industry) No means to judge 8
Aspect observed Participants’ experience topic χ2 value Student’s t-value
Business
process point
Similarity BPM knowledge 0.88 3.61 analysis
BPMN knowledge 0.44
Software metrics knowledge 1.05
FPA knowledge 1.88
Application easiness BPM knowledge 0.30 1.94
BPMN knowledge 1.54 419
software metrics knowledge 0.82
FPA knowledge 1.10
Table IX.
Feasibility BPM knowledge 1.00 3.05 Relations between
BPMN knowledge 0.20 participants’
Software metrics knowledge 0.94 experience and
FPA knowledge 1.91 aspects observed, and
Application benefits BPM knowledge 0.27 2.50 calculated values of
BPMN knowledge 0.20 the population
Software metrics knowledge 0.36 proportion test

FPA knowledge 0.61 (industry)
Following the significance value of 5 percent (for the Student’s t-test distribution table),
the proportion values calculated should be greater than 1.64 so that the null hypotheses were
denied in favor of the alternative hypotheses. The result denied all the four null hypotheses.
5.2.3 Comparing results from studies I and II. The analysis showed that BPPA, when
compared with FPA, was highly rated in most of the aspects (i.e. similarity, application
easiness, feasibility, and potential application benefits) covered in the experiment, except for
having been considered more difficult to be applied than FPA (but only by participants from
academia) and less feasible than FPA (but particularly by the “Only integrated students”
from academia). These results indicate that BPPA was in general well assessed when
comparing with FPA regardless of the nature of the participants, either from academia or
from industry, mainly regarding similarity and potential application benefits.
Both application easiness and feasibility were well assessed only by one of the two
groups, indicating that these aspects may be not completely addressed in a proper way in
our proposed approach.
More participants from academia than from industry had no means to judge some
aspects. Although these differences are not significant enough to affect the conclusion of
these studies, they can be possibly explained due to the distinct viewpoints of each type of
participants, since both groups showed different knowledge and professional backgrounds.
In relation to participants’ previous experience, whereas participants from academia pointed
out higher knowledge in BPM and BPMN, participants from industry pointed out higher
knowledge in software metrics and FPA.
6. Lessons learned from the experimental studies

This section presents a summary of an additional and concluding analysis on the results
achieved with both studies. We first structured the analysis on the four covered aspects:
similarity, application easiness, feasibility and application benefits of BPPA compared with
FPA. Moreover, we also present a brief analysis on the influence of the differences among
participants’ experiences in the context of the assessed approach. Finally, we address
limitations of the undertaken studies:
• Similarity: BPPA is similar or equal to FPA in both studies. This is an important
result since BPPA was proposed and conceived based on FPA. Although there are
some important and necessary differences between them, BPPA is designed as a
BPMJ specific type of FPA. We did it so that the known suitable features of FPA were
23,2 transferred to the BPM context.
• Application easiness: BPPA was considered “as easy as,” or “easier than” FPA to be
applied by participants from industry rather than academia. Their different levels of
previous knowledge on FPA shall explain this difference. It is difficult to apply the
FPA technique, therefore, it also naturally applies to BPPA due to similarity.
420 However, since metric experts from industry are already used to FPA, they probably
see neither FPA nor BPPA as difficult techniques. In contrast, since students from
academia are not so used to FPA, they probably would find complex aspects in both
FPA and BPPA.
• Feasibility: BPPA was considered “as feasible as” or “more feasible than” FPA only
by participants from industry and not from academia. Although, according to data in
Table V, the “All students” group from academia presents a higher percentage
indicating the opposite, the Student’s t-test did not allow rejecting the null hypothesis
for it. This difference between both types of participants for the feasibility aspect is
explained in a similar way as presented for the application easiness aspect (in the
previous item). Since metric experts are more used to metric techniques, they tend to
be not as idealistic as students expect to be.
• Application benefits: BPPA was considered as providing benefits “as much as” or
“more than” FPA in both studies. Both groups could see benefits of BPPA for the
BPM context in the same way as FPA provides benefits for the software engineering
context. The result is entirely consistent to participants from industry since it
matches the results of three previous aspects for those participants. On the other
hand, participants from academia considered BPPA more difficult to be applied and
less feasible than FPA; but, in summary, they ended up highly rating BPPA on
benefits comparing with FPA. This students’ behavior in overall perceived benefits is
explained considering the software quality theory learned during the course.
Moreover, although there was a large difference between participants’ previous experiences
for both studies – indeed, they have quite complementary experiences – these differences
did not change the studies’ results. For all aspects assessing BPPA compared with FPA,
results were independent of previous experience, for both studies.
A specific issue noticed in study I refers to the inexperience of participants with FPA that
could have impacted the comparison between BPPA and FPA. In order to mitigate this
specific experiment validity threat, we provided FPA training to participants. In study II,
the little experience of the participants with BPMN was a similar issue, which can also have
influenced the comparison between BPPA and FPA. We had not anticipated this issue.
Nevertheless, explanatory material concerning BPPA provided for the participants showed
some basic information on BPMN.
As a general limitation of this study, we only compared BPPA with FPA in similarity,
application easiness, feasibility, and application benefits, i.e., no further aspect was used in
this comparison as well as no other type of assessment was performed besides the
comparison of BPPA with FPA. We chose first to assess BPPA in contrast to FPA because
our main purpose was to conceive a BPM-specific technique as compliant and meaningful as
possible with the original FPA. Therefore, such a comparison was considered the most basic
analysis to be performed. The four aspects were based on a previous study (Gonçalves et al.,
2011). Our study refers to a preliminary assessment since other assessment aspects should
still be considered in order to completely validate the proposed BPPA technique.
BPPA should be further assessed mainly considering its contributions to the BPM field as
well as its accuracy and costs regardless of being compliant to FPA.
Finally, we highlight some limitations regarding the types of BPM projects that were Business
assessed in this experimental studies. The results achieved here can be analyzed only under process point
the perspective of development projects, i.e., those projects related to automate a business analysis
process, taking a process model as input. No other type of project, including application and
enhancement projects, was assessed here since the first version of BPPA still does not
address them. Moreover, projects where the purpose is not related to the automation of the
modeled process were also not addressed in the studies presented here, for the same reason 421
explained above. All these additional project types are important for the BPM context, and
hence one needs to know if BPPA would also behave similarly to FPA for them. In order to
extend BPPA to address these types of BPM projects, it is necessary to follow and adapt the
respective guidelines presented by FPA for software engineering projects. In summary, in a
general way, the weights used to measure data functions and transactional functions should
be properly adjusted, including those presented in Tables II, III and IV. Other fine
adjustments should be considered so that the process size measured could be properly used
to all these project types.
7. Conclusions and future works

This paper addressed a new technique, called BPPA, for measuring the size of BPM projects
based on BPMN models. BPPA was proposed in order to provide BPM project managers
with a specific metric – called BPP – which enables them to estimate the functional size of a
business process that needs to be automated. Based on the calculated BPP metric, project
managers can derive other important estimates such as time, effort, and cost necessary to
automate such a business process.
We built BPPA based on the FPA framework, having FPA as a model of good practices
to be followed, and hence the procedures of both are very similar, although BPPA was
adapted to fit BPM context. Two studies were addressed to assess BPPA, especially
comparing it with FPA since we intended to design BPPA as similar to FPA as possible.
These studies contributed to provide experimental data in a field that still lacks examples of
such initiatives. We assessed four main aspects during the studies: similarity, application
easiness, feasibility, and application benefits. We conducted the studies with participants
from both academia and industry. The outcomes provided mostly positive evidences of the
aspects observed, showing that BPPA overall conforms to FPA.
For future works, we plan to carry out other studies to perform additional analyses by
controlled experiments or case studies. These further studies should strengthen the
assessment as follows:
(1) including impartial specialists from the BPM domain as well as more specialists
from the software metrics field;
(2) running an experiment in a real-world setting;
(3) increasing validity assurance by defining a set of objective metrics to be applied
through a clear protocol; and
(4) leading to a comparative approach between BPPA and other techniques related to
BPM measurement.
Note
1. We used an 11-point range to allow freedom of opinion, with the value five as the middle. We used
the same scale for all the evaluations in order to standardize and facilitate comparison between
different questions.
BPMJ References
23,2 Baklizky, M., Fantinato, M., Thom, L.H., Sun, V., Prado, E.P.V. and Hung, P. (2013), “Business process
points – a proposal to measure BPM projects” Proceedings of the 21st European Conference on
Information Systems, Springer, Utrecht, p. 2.
Baklizky, M. and Fantinato, P. (2012), “FPA4BPM – Function point analysis for business process
management (v.1.0)” Technical Report PPgSI-003/2012, Master of Science Graduate Program in
422 Information Systems, School of Arts, Sciences and Humanities, University of São Paulo, available
at: http://ppgsi.each.usp.br/arquivos/RelTec/PPgSI-003_2012.pdf (accessed 7 March 2017).
Aysolmaz, B. and Demirörs, O. (2015), “Unified process modeling with UPROM tool”, Proceedings of the
International Conference on Advanced Information Systems Engineering, CAiSE Forum 2014,
Thessaloniki, pp. 250-266.
Basili, V.R., Caldiera, G. and Tombach, H.D. (1994), “Goal question metric paradigm”, Encyclopedia of
Software Engineering, Vol. 2, 1st ed., John Wiley & Sons, Inc., Hoboken, NJ, pp. 528-532.
Business Process Management Initiative (BPMI) (2011), “Business process model and notation
(BPMN)”, Release 2.0. Object Management Group (OMG), available at: www.omg.org/spec/
BPMN/2.0/PDF/ (accessed 20 March 2016).
Cardoso, J. (2006), “Process control-flow complexity metric: an empirical validation”, Proceedings of the
3rd IEEE International Conference on Services Computing, IEEE Computer Society, Chicago, IL,
pp. 167-173.
Dhammaraska, K. and Intakosum, S. (2009), “Measuring size of business process from use case
description”, Proceedings of the 2nd IEEE International Conference on Computer Science and
Information Technology, IEEE Computer Society, Beijing, pp. 600-604.
Dumas, M., La Rosa, M., Mendling, J. and Reijers, H.A. (2013), Fundamentals of Business Process
Management, 1st ed., Springer, Heidelberg.
Dumas, M., La Rosa, M., Mendling, J., Mäesalu, R., Reijers, H.A. and Semenenko, N. (2012),
“Understanding business process models: the costs and benefits of structuredness”, Proceedings
of the 24th International Conference on Advanced Information Systems Engineering, Springer,
Gdańsk, pp. 31-46.
Fantinato, M., Toledo, M.B.F., Thom, L.H., Gimenes, I.M.S., Rocha, R.S. and Garcia, D.Z.G. (2012),
“A survey on reuse in the business process management domain”, International Journal of
Business Process Integration and Management, Vol. 6 No. 1, pp. 52-76.
Gonçalves, T.L., Gimenes, I.M.S., Fantinato, M., Travassos, G.H. and Toledo, M.B.F. (2011),
“Experimental studies of e-contract establishment in the PL4BPM context”, International
Journal of Web Engineering and Technology, Vol. 6 No. 3, pp. 243-265.
González, L.S., García, F., Ruiz, F. and Piattini, M. (2010), “Measurement in business processes:
a systematic review”, Business Process Management Journal, Vol. 16 No. 1, pp. 114-134.
Gruhn, V. and Laue, R. (2006), “Complexity metrics for business process models”, Proceedings of the 9th
International Conference on Business Information Systems, Springer, Klagenfurt, pp. 1-12.
International Function Point Users Group (IFPUG) (2010), Function Point Counting Practices Manual,
Release 4.3.1, International Function Point Users Group, Westerville, OH.
Kaya, M. and Demirörs, O. (2011), “E-COSMIC: a business process model based functional size
estimation approach”, Proceedings of the 37th EUROMICRO Conference on Software
Engineering and Advanced Applications, IEEE Computer Society, Oulu, pp. 404-410.
Marín, B. and Quinteros, J. (2014), “A COSMIC measurement procedure for BPMN diagrams”,
Proceedings of the International Conference on Software Engineering and Knowledge
Engineering, Vancouver, pp. 408-411.
Object Management Group (2010), “BPMN 2.0 by example”, Version 1.0, pp. 25-26, available at: www.
omg.org/spec/BPMN/2.0/examples/PDF (accessed 20 March 2016).
Project Management Institute (PMI) (2008), A Guide to the Project Management Body of Knowledge
(PMBOK Guide), 4th ed., Project Management Institute Inc, Newtown Township, PA.
Sadowska, M. (2015), “Process models expressed in BPMN”, e-Informatica Software Engineering Business
Journal, Vol. 9 No. 1, pp. 57-77. process point
Salmanoǧlu, M., Demirörs, O. and Türetken, O. (2014), “Exploration of a method for COSMIC size analysis
estimation from S-BPM”, Communications in Computer and Information Science, Vol. 422,
pp. 55-66, available at: https://link.springer.com/ bookseries/7899
Sommerville, I. (2004), Software Engineering, 6th ed., Addison Wesley, Boston, MA.
Thom, L.H., Reichert, M. and Iochpe, C. (2009), “Activity patterns in process-aware information 423
systems: basic concepts and empirical evidence”, International Journal of Business Process
Integration and Management, Vol. 4 No. 2, pp. 93-110.
Vanderfeesten, I., Reijers, H.A., Mendling, J., van der Aalst, W.M.P. and Cardoso, J. (2008), “On a quest
for good process models: the cross-connectivity metric”, Proceedings of the 20th International
Conference on Advanced Information Systems Engineering, Springer-Verlag, Montpellier,
pp. 480-494.
Viegas, C.F.O. and Toledo, M.B.F. (2013), “A new metric to estimate project development time: Process
points”, Proceedings of the 7th International Conference on Knowledge Management in

Organizations, KMO 2013, Salamanca, pp. 197-208.
Weske, M. (2007), Business Process Management: Concepts, Languages, Architectures, 1st ed., Springer,
Heidelberg.
Zelkowitz, M.V. and Wallace, D.R. (1998), “Experimental models for validating technology”, Computer,
Vol. 31 No. 5, pp. 23-31.
Further reading
Aversano, L., Bodhuin, T., Canfora, G. and Tortorella, M. (2004), “A framework for measuring business
processes based on GQM”, ‘Proceedings of the Hawaii International Conference on System
Sciences, IEEE Computer Society, Big Island, HI, pp. 163-172.
Chergui, M.E.A. and Benslimane, S.M. (2014), “Services derivation from business process: a PSO-based
multi-objective approach”, Proceedings of the 2014 International Conference on Multimedia
Computing and Systems, Marrakesh, pp. 582-588.
Clemmons, R.K. (2006), “Project estimation with use case points”, Crosstalk – the Journal of Defense
Software Engineering, Vol. 19 No. 2, pp. 18-22.
Common Software Measurement International Consortium (COSMIC) (2009), The COSMIC Functional Size
Measurement Method, Version 4.0.1, Measurement Manual (The COSMIC Implementation Guide
for ISO/IEC 19761: 2003), Common Software Measurement International Consortium, available at:
http://cosmic-sizing.org/ publications/measurement-manual-401/ (accessed 20 March 2016).
About the authors

Maruscia Baklizky is a System Analyst at JPMorgan Chase, Brazil. She has a Bachelor in Information
Systems (2013) from the University of São Paulo, Brazil. Her main research interests are information
technology, software metrics, and business process management.
Marcelo Fantinato is an Associate Professor at the University of São Paulo, Brazil. He Has PhD in
Computer Science (2007) and Master of Engineering (2002) from the University of Campinas;
and Bachelor in Computer Science (1999) from the State University of Maringa, Brazil. He worked as a
Software Testing Specialist at the CPqD Foundation (2001-2006) and as a Specialist in Research and
Development at Motorola (2006-2008), Brazil. He has a Green Belt certification in the Motorola Six Sigma
program for process improvement. His main research interests are business process management,
service-oriented computing, software product line, and software testing. Marcelo Fantinato is the
corresponding author and can be contacted at: m.fantinato@usp.br
Lucineia Heloisa Thom is an Assistant Professor at the Federal University of Rio Grande do Sul
(UFRGS), Brazil. She was a Visiting Scientist at the University of Grenoble (2010-2011), France, and at
the University of Ulm (2007-2009), Germany. She has a Bachelor in Computer Science from the
University of Santa Cruz do Sul (1999), Master (2002) and PhD (2006) in Computer Science from
UFRGS, Brazil. Part of her PhD was developed at the University of Stuttgart (2004-2005), Germany.
BPMJ Her research interests are in the area of workflow patterns, process design, IT support for healthcare
23,2 processes and ontology.
Violeta Sun is an Assistant Professor at the University of São Paulo, Brazil. She has a PhD (2010)
and Master (2005) in Business Administration from the University of São Paulo, Brazil. Part of her PhD
was developed at the University of Oslo, Norway. She also has a Bachelor in Business Administration
(1986) from the Fundação Armando Álvares Penteado, Brazil. Her main research areas are project
management, information technology management, information technology, IT infrastructure,
424 IT governance, management systems in health, computer application in public administration.
Patrick C.K. Hung is an Associate Professor at the University of Ontario Institute of Technology,
Canada. He has a PhD in Computer Science (2001) and Master of Philosophy Science in Computer
Science (1995) from the Hong Kong University of Science and Technology, Hong Kong; a Master of
Applied Science in Management Sciences (2002) from the University of Waterloo, Canada; and a
Bachelor of Science in Computer Science (1993) from the University of New South Wales, Australia.
Dr Hung has been working with Boeing Research and Technology in Seattle, Washington on aviation
services-related research projects. He has a US patent on the Mobile Network Dynamic Workflow
Exception Handling System and a continuation-in-part pending patent application with Boeing.
For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com

Baklizky2017 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Baklizky2017 PDF

Uploaded by

Copyright:

Available Formats

Business Process Management Journal

Business process point analysis: survey experiments

Downloaded on: 08 April 2017, At: 12:38 (PT)

*Related content and download information correct at time of download.

Patrick C.K. Hung

Business Process Management

comprise of the phases presented in Table I.

BPM lifecycle phases Software engineering lifecycle phases

1. Design and analysis 1.1. Requirements specification

3.1 Stage 1 – Gather available documentation

3.1.2. Classify 3.1.3. Determine 3.1.4. Determine

4. Calculate process size

3.3 Stage 3 – measure functions

RDSs Process complexity

3.5 Stage 5 – document the BPP count

3.6 Stage 6 – report the result of the BPP count

TFW (EI) TFW (EO/EQ)

5. Experimental studies for BPPA

Source: Adapted from BPMI (2011)

5.1 Planning of the experimental studies

5.2 Results of the experimental studies

score their knowledge in these four topics ranging from 0 to 10 (0 – no knowledge;

Participants’ experience “All students” χ2 “Only integrated students” χ2

Similarity BPM knowledge 1.53 0.28

We applied a non-parametric analysis to calculate observed and expected frequencies for

Aspect observed Answer %

Similarity BPPA is not similar to FPA 0

Software metrics knowledge 0.36 proportion test

6. Lessons learned from the experimental studies

to all these project types.

7. Conclusions and future works

points”, Proceedings of the 7th International Conference on Knowledge Management in

About the authors

You might also like