Professional Documents
Culture Documents
Assessment Questionnaire
Document History
Drafted Reviewed Approved
V. Modification Date Who Date Who Date Who
1.00 Initial
Document Information
Author: Daniel Frutschi
Manager: Martin Andenmatten
Document Name: GLF_COBIT5_eng_T11_Questionnaire_BAI07_r2_v2.0.doc
Title: Assessment Questionnaire
Subject: BAI07 Manage Change Acceptance & Testing
Category: Assessment Questionnaire
Comment: Questionnaire for the COBIT5 process assessment.
2
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Summary
1. Individual Data ........................................................................................................ 4
2. BAI07 Manage Change Acceptance & Testing .................................................... 5
2.1. Base Practices .......................................................................................................... 7
2.2. Process Work Products .......................................................................................... 15
2.3. Process Maturity Level 1 Assessment .................................................................... 17
2.4. Process Maturity Level 2 Assessment .................................................................... 18
2.5. Work Products for Level 2 ...................................................................................... 22
2.6. Process Maturity Level 2 Assessment .................................................................... 23
2.7. Work Products for Level 3 ...................................................................................... 29
2.8. Process Maturity Level 4 Assessment .................................................................... 30
2.9. Work Products for Level 4 ...................................................................................... 34
2.10. Process Maturity Level 5 Assessment .................................................................... 35
2.11. Work Products for Level 5 ...................................................................................... 38
3
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
1. Individual Data
Seniority in the
Name Current Position Organization
and History of Positions
4
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Process ID BAI07
5
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
6
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
7
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
8
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
1. Develop and document the test plan, which aligns to the programme and
project quality plan and relevant organisational standards. Communicate
and consult with appropriate business process owners and IT stakeholders.
2. Ensure that the test plan reflects an assessment of risk from the project and
that all functional and technical requirements are tested. Based on
assessment of the risk of system failure and faults on implementation, the
plan should include requirements for performance, stress, usability, pilot
and security testing.
3. Ensure that the test plan addresses the potential need for internal or
external accreditation of outcomes of the test process (e.g., financial
regulatory requirements).
4. Ensure that the test plan identifies necessary resources to execute testing
and evaluate the results. Examples of resources include construction of test
environments and use of staff time for the test group, including potential
temporary replacement of test staff in the production or development
environments. Ensure that stakeholders are consulted on the resource
implications of the test plan.
N/P/L/F/N.A.
5. Ensure that the test plan identifies testing phases appropriate to the
operational requirements and environment. Examples of such testing
phases include unit test, system test, integration test, user acceptance test,
performance test, stress test, data conversion test, security test, operational
readiness test, and backup and recovery tests.
6. Confirm that the test plan considers test preparation (including site
preparation), training requirements, installation or an update of a defined
test environment, planning/performing/documenting/retaining test cases,
error and problem handling, correction and escalation, and formal approval.
7. Ensure that the test plan establishes clear criteria for measuring the
success of undertaking each testing phase. Consult the business process
owners and IT stakeholders in defining the success criteria. Determine that
the plan establishes remediation procedures when the success criteria are
not met (e.g., in a case of significant failures in a testing phase, the plan
provides guidance on whether to proceed to the next phase, stop testing or
postpone implementation).
8. Confirm that all test plans are approved by stakeholders, including business
process owners and IT, as appropriate. Examples of such stakeholders are
application development managers, project managers and business
process end users.
9
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
10
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Test changes independently in accordance with the defined test plan prior to
migration to the live operational environment.
Are the following activities planned, done, checked and improved?
1. Review the categorised log of errors found in the testing process by the
development team, verifying that all errors have been remediated or
formally accepted.
2. Evaluate the final acceptance against the success criteria and interpret the
final acceptance testing results. Present them in a form that is
understandable to business process owners and IT so an informed review
and evaluation can take place.
3. Approve the acceptance with formal sign-off by the business process
owners, third parties (as appropriate) and IT stakeholders prior to promotion
to production.
4. Ensure that testing of changes is undertaken in accordance with the testing
plan. Ensure that the testing is designed and conducted by a test group
independent from the development team. Consider the extent to which
business process owners and end users are involved in the test group.
Ensure that testing is conducted only within the test environment.
5. Ensure that the tests and anticipated outcomes are in accordance with the
defined success criteria set out in the testing plan.
N/P/L/F/N.A. 6. Consider using clearly defined test instructions (scripts) to implement the
tests. Ensure that the independent test group assesses and approves each
test script to confirm that it adequately addresses test success criteria set
out in the test plan. Consider using scripts to verify the extent to which the
system meets security requirements.
7. Consider the appropriate balance between automated scripted tests and
interactive user testing.
8. Undertake tests of security in accordance with the test plan. Measure the
extent of security weaknesses or loopholes. Consider the effect of security
incidents since construction of the test plan. Consider the effect on access
and boundary controls.
9. Undertake tests of system and application performance in accordance with
the test plan. Consider a range of performance metrics (e.g., end-user
response times and database management system update performance).
10. When undertaking testing, ensure that the fallback and rollback elements of
the test plan have been addressed.
11. Identify, log and classify (e.g., minor, significant, mission-critical) errors
during testing. Ensure that an audit trail of test results is available.
Communicate results of testing to stakeholders in accordance with the test
plan to facilitate bug fixing and further quality enhancement.
11
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
12
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Provide early support to the users and IT operations for an agreed-on period of
time to deal with issues and help stabilise the new solution.
Are the following activities planned, done, checked and improved?
13
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
14
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
15
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
16
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
17
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Is there a plan of actions to execute the process? What type of plan is it?
Is there any monitoring of process activities against that plan?
Are activities appropriately and regularly monitored?
Is the time to perform the process estimated and tracked?
GP 212 Plan
and Monitor the
Performance of
the Process to
Fulfill the
Identified
Objectives
18
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Is someone tracking the process activities? Does someone notice and react if
they are not performed?
Are there corrective actions when process results dont meet objectives?
(More resources, new planning)?
GP 213 Adjust
the
Performance of
the Process
What are the roles and responsibilities that have been identified for this
process?
Are they clearly defined (even not formally at this stage)? Are they
communicated to everyone?
Who is responsible for that?
GP 214 Define
Responsibilities
and Authorities
for Performing
the Process
Do you think that existing HR are well trained and efficient? Are they
numerous enough and available? Are other resources available when needed
(monitoring, tracking, and reporting tools)?
GP 215 Identify
and Make
Available
Resources to
Perform the
Process
according to
Plan
19
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Who are involved parties in the process? Who uses the outputs from this
process (reporting, documents or information)?
What type of communication is there between the different stakeholders?
How is the coordination between the organizational functions involved in the
process managed?
GP 216
Manage the
Interfaces
Between
Involved
Parties
PA 2.2 Work The Work Product Management attribute is a measure of the extent to which
Product the Work Products produced by the process are appropriately managed.
Management
Attribute
What are the documents/information used or resulting from the process?
Do you have defined requirements and quality criteria for these documents
(and information)?
Do you know what they should contain?
GP 221 Define Are the input and output of the process based on templates (at your level)?
the
Requirements
for the Work
Products
Do you have specific rules to manage the process inputs and outputs?
Are there rules for versioning? Document naming?
Are there access rules for documents, information, databases?
GP 222 Define
the
Requirements
for
Documentation
and Control of
the Work
Products
20
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
GP 223
Identify,
Document, and
Control the
Work Products
Are there reviews of the documents and other outputs produced by the
process? Do you compare the actual documents to their requirements and
quality criteria?
Is there a frequency for reviewing the documents and other outputs produced
by the process?
If any problem is detected, are corrective actions taken? How?
GP 224 Review
and Adjust
Work Products
to Meet the
Defined
Requirements
21
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Work Products
ID Name Used Name in the Location
(X) Organization
PA 2.1 Performance Management Attribute
3-00 Plan
5-00 Record
6-00 Report
PA 2.2 Work Product Management Attribute
1-00 Object
3-00 Plan
5-00 Record
8-00 Specification
22
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
GP 311 Define
the Standard
Process that will
support the
Deployment of
the Process
Are the connections between the process activities and other processes
clearly identified and defined? Where?
Do you have a diagram describing the relationships?
GP 312
Determine the
Sequence and
Interaction
between
Processes so
that they work
as an Integrated
System of
Processes
23
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
GP 313 Identify
the Roles and
Competencies
for Performing
the Standard
Process
GP 314 Identify
the Required
Infrastructure
and Work
Environment for
Performing the
Standard
Process
24
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Have you identified the different audiences for the service management
reports?
Have you defined the layout, contents, and frequency of Service
Management reports (according to the audience)?
[ITIL 2011: Was it agreed with the business?
Service
Reporting]
GP 316 Define
and Agree upon
the Content of
Service
Management
Reports
PA 3.2 Process The Process Deployment attribute is a measure of the extent to which the
Deployment standard process is effectively deployed to achieve its process Expected
Attribute Results.
Does the process implemented correspond to the one that is described
formally?
How do you organize the deployment of the process across the organization?
GP 321 Deploy
a Standard
Process that
satisfies the
context-specific
Requirements
from the
Standard
Process
Definition
25
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Are all the defined roles for this process assigned to someone?
Are all the responsibilities identified and communicated? With appropriate
authority lead?
GP 322 Assign
and
Communicate
Roles,
Responsibilities,
and Authorities
for Performing
the Standard
Process
Do you think that people working on this process have the right level of
knowledge or training?
How do you ensure that the competence level is adequate?
Do you look at particular skills when hiring people?
Is there a training plan for the actors of this process? Are there training
sessions organized to improve the competence level?
GP 323 Ensure
necessary
Competencies
for Performing
the Standard
Process
26
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
GP 324 Provide
Resources and
Information to
Support the
Performance of
the Standard
Process
Does the work environment correspond to what has been defined formally for
implementing this process?
Do you have enough technical (HW, SW) resources?
GP 325 Provide Do you have the appropriate tools?
Adequate
Process
Infrastructure to
Support the
Performance of
the Standard
Process
27
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
[ITIL 2011:
Service
Reporting]
GP 327
Produce and
Publish Service
Management
Reports
28
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Work Products
ID Name Used Name in the Location
(X) Organization
PA 3.1 Process Definition Attribute
2-00 Description
3-00 Plan
4-00 Procedure
5-00 Record
8-00 Specification
PA 3.2 Process Deployment Attribute
2-00 Description
3-00 Plan
5-00 Record
6-00 Report
8-00 Specification
29
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Level 4 The Established Process now operates within defined limits to achieve its
Predictable process Expected Results.
Process
PA 4.1 The Process Measurement attribute is a measure of the extent to which
Process measurement results are used to ensure that the performance of the process
Measurement supports the achievement of the relevant process performance objectives in
Attribute support of defined business goals.
Are business goals known?
What process information is necessary to determine whether business goals
are met?
GP 411
Identify
Process
Information
Needs, in
Relation with
Business
Goals
GP 412
Derive
Process
Measurement
Objectives
from Process
Information
Needs
30
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
GP 413 Can you give examples of process quantitative objectives that you follow?
Establish
Quantitative
Objectives for
the
Performance
of the
Standard
Process,
according to
the Alignment
of the
Process with
the Business
Goals
What metrics are measured and what is the frequency of those measurements?
With these measures can you determine whether the process reaches its
quantitative objectives?
GP 414
Identify
Product and
Process
Measures
that support
the
Achievement
of the
Quantitative
Objectives for
Process
Performance
How are those measures collected? Who is responsible for collecting, and
reporting measures?
Are they measured automatically? How hard is it? How long does it take to get
those measures?
GP 415
Collect
Product and
Process
Measurement
Results
through
performing
the Standard
Process
31
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
PA 4.2 The Process Control attribute is a measure of the extent to which the process is
Process quantitatively managed to produce a process that is stable, capable, and
Control predictable within defined limits.
Attribute
Are there regular statistical analyses of the process performance?
Have you defined techniques to control the process performance?
GP 421
Determine
Analysis and
Control
Techniques,
appropriate
to control the
Process
Performance
GP 422
Define
Parameters
suitable to
Control the
Process
Performance
32
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
How are special causes of variations treated? How are corrective action
implemented?
GP 424
Identify and
Implement
Corrective
Actions to
Address
Assignable
Causes
After the implementation of corrective actions, are new limits defined for
process performance?
GP 425 Re-
Establish
Control Limits
following
Corrective
Actions
33
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Work Products
ID Name Used Name in the Location
(X) Organization
PA 4.1 Process Measurement Attribute
2-00 Description
3-00 Plan
5-00 Record
6-00 Report
8-00 Specification
PA 4.2 Process Control Attribute
2-00 Description
3-00 Plan
5-00 Record
6-00 Report
34
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
GP 511 Define
the Process
Improvement
Objectives for
the Process
that Support
the Relevant
Business
Goals
GP 512
Analyze
Measurement
Data of the
Process to
Identify Real
and Potential
Variations in
the Process
Performance
35
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Do you watch innovation and the evolution of best practices in the process
domain to identify improvement opportunities?
GP 513
Identify
Improvement
Opportunities
of the Process
Based on
Innovation and
Best Practices
GP 514 Derive
Improvement
Opportunities
of the Process
from New
Technologies
and Process
Concepts
GP 515
Define an
Implementatio
n Strategy
based on long-
term
Improvement
Vision and
Objectives
36
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
How are the process improvements implemented? What are the steps?
GP 522
Manage the Who implements the process improvement changes? Who will follow and
Implementatio check the positive or negative effect?
n of agreed
changes to
select areas of
the Defined
Standard
Process
according to
the
Implementatio
n Strategy
37
COBIT5
Assessment Questionnaire
BAI07 Manage Change Acceptance & Testing
Work Products
ID Name Used Name in the Location
(X) Organization
PA 5.1 Process Innovation Attribute
2-00 Description
3-00 Plan
4-00 Procedure
5-00 Record
6-00 Report
8-00 Specification
PA 5.2 Process Optimization Attribute
2-00 Description
3-00 Plan
5-00 Record
6-00 Report
8-00 Specification
38