You are on page 1of 39

CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,

Development and Implementation

Domain 3

Domain 3 Provide assurance that the practices for


the acquisition, development, testing and
implementation of information systems
Information Systems Acquisition,
Development and Implementation objectives.

©Copyright 2016 ISACA. All rights reserved. 2 © Copyright 2016 ISACA. All rights reserved.

Domain 3 Domain Objectives


The focus of Domain 3 is to provide an overview of key The objective of this domain is to ensure that the CISA
processes and methodologies used by organizations candidate understands and can provide assurance that
when creating and changing application systems and the practices for the acquisition, development, testing
infrastructure components. and implementation of information systems meet the

The CISA candidate must understand how an


organization evaluates, develops, implements, maintains
and disposes of its IT systems and related components.

3 © Copyright 2016 ISACA. All rights reserved. 4 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 1


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

On the CISA Exam Domain Tasks


Domain 3 represents 18% of the questions on the CISA 3.1 Evaluate the business case for the proposed
exam (approximately 27 questions). investments in information systems acquisition,
Domain 3 incorporates seven tasks related to development, maintenance and subsequent retirement
information systems acquisition, development and to determine whether it meets business objectives.
implementation. 3.2 Evaluate IT supplier selection and contract
management processes
service levels and requisite controls are met.
3.3 Evaluate the project management framework and
controls to determine whether business requirements are
achieved in a cost-effective manner while managing
risks to the organization.

5 © Copyright 2016 ISACA. All rights reserved. 6 © Copyright 2016 ISACA. All rights reserved.

3.4 Conduct reviews to determine whether a project is 3.6 Evaluate the readiness of information systems for
progressing in accordance with project plans, is implementation and migration into production to
adequately supported by documentation, and has timely determine whether project deliverables, controls and the
and accurate status reporting. requirements are met.
3.5 Evaluate controls for information systems during the 3.7 Conduct post-implementation reviews of systems to
requirements, acquisition, development and testing determine whether project deliverables, controls and the
requirements are met.
standards, procedures and applicable external
requirements.

7 © Copyright 2016 ISACA. All rights reserved. 8 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 2


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Task 3.1 Key Terms


Key Term Definition
Business case Documentation of the rationale for making a
business investment, used both to support a
Evaluate the business case for the business decision on whether to proceed with the

proposed investments in information


investment and as an operational tool to support
management of the investment through its full
systems acquisition, development, economic life cycle

maintenance and subsequent retirement


Return on A measure of operating performance and
investment (ROI) efficiency, computed in its simplest form by dividing
to determine whether it meets business net income by the total investment over the period
being considered
objectives.

9 © Copyright 2016 ISACA. All rights reserved. 10 © Copyright 2016 ISACA. All rights reserved.

Task to Knowledge Statements


How does Task 3.1 relate to each of the following How does Task 3.1 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.1 Knowledge of benefits realization Understanding the business case K3.5 Knowledge of risk management The IS auditor needs to understand how
practices (e.g., feasibility studies, business development approach for program practices applied to projects risk management processes are integrated
cases, total cost of ownership [TCO], return management and SDLC processes. throughout program management
on investment [ROI]) processes and system and software
K3.3 Knowledge of project governance The IS auditor needs to understand development activities.
mechanisms (e.g., steering committee, program management governance
project oversight board, project concepts and how to evaluate the program
management office) office and/or project steering committee
integration within the organization.

11 © Copyright 2016 ISACA. All rights reserved. 12 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 3


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

How does Task 3.1 relate to each of the following How does Task 3.1 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.7 Knowledge of enterprise architecture Enterprise architectures are supported or K3.13 Knowledge of project success The IS auditor needs to understand the
related to data, applications and technology served by IT architectures (e.g., n-tier, criteria and project risk specific success criteria for the
(e.g., web-based applications, web client-server, web-based and distributed system/software programs and respective
services, n-tier applications, cloud services, components). The IS auditor must projects and how management is
virtualization) understand the role of these components evaluating progress toward meeting these
and how control objectives are met across criteria along with addressing deficiencies
all components to determine whether risk is that put the program and respective
sufficiently mitigated by these controls. projects at risk.

13 © Copyright 2016 ISACA. All rights reserved. 14 © Copyright 2016 ISACA. All rights reserved.

Benefits Realization Benefits Realization Objectives


To assess whether IT management is fulfilling its value The objectives of benefits realization include:
management responsibilities, the IS auditor must o IT-enabled business investments achieve the
understand how the business defines value or an ROI for promised benefits and deliver measurable business
development-related projects. value.
Because IT-related initiatives have high expenditures, o Required capabilities (solutions and services) are
these projects must be evaluated on: delivered on time and within budget.
o Cost o IT services and assets continue to contribute to
o Quality business value.
o Development/delivery time
o Reliability and dependability

15 © Copyright 2016 ISACA. All rights reserved. 16 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 4


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Benefits Realization Techniques


Benefits realization requires a planned approach. It is a Key elements include:
continuous process that must be part of the governance o Describing benefits management or benefits
and management of projects. realization
o Assigning a measure and target
o Establishing a tracking/measuring regimen
o Documenting the assumption
o Establishing key responsibilities for realization
o Validating the benefits predicted in the business
o Planning the benefit that is to be realized

17 © Copyright 2016 ISACA. All rights reserved. 18 © Copyright 2016 ISACA. All rights reserved.

Benefits Realization Phases Business Case


A business case provides the information required for an
organization to decide whether a project should proceed.
Understand: The
organization defines

Benefits Realization Phases


specific objectives

It allows for a comparison of costs and business benefits


expected from a
project and outcomes
needed to achieve the

and provides justification for setting up or continuing a


defined objectives.

project.
Report: Actual versus
the planned results Plan: Based upon It is often the first step in a project and normally derives
from a feasibility study.
are reported to senior Understand phase, a
management and plan is developed to
Provide accountability achieve the objectives
for the performance of and outcomes.
the program.

Realize: Progress
toward achieving the
goals is monitored.
Deviations can be
discovered, and
corrective action can
be taken. The plan Source: New South Wales Government
and the business case
Department of Finance & Services,
should be updated
and maintained as Benefits Realisation Guideline, Version 1.2,
changes occur. Australia, 2011

19 © Copyright 2016 ISACA. All rights reserved. 20 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 5


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Feasibility Study
Define the project scope. During the feasibility study, the IS auditor should perform
the following:
o Review the documentation for the phase to ensure
Conduct a current analysis.
that it is reasonable.
o Determine whether all cost justifications/benefits are
Identify requirements based on stakeholder needs. verifiable and that they show the anticipated costs
and expected benefits.
Provide a recommended approach. o Identify and determine the criticality of the need.
o Determine if a solution can be achieved with systems
Evaluate the cost-effectiveness of the approach. already in place. If not, review the evaluation of
alternative solutions for reasonableness.
Conduct a formal review with stakeholders.
o Determine the suitability of the chosen solution.

21 © Copyright 2016 ISACA. All rights reserved. 22 © Copyright 2016 ISACA. All rights reserved.

In the Big Picture Discussion Question


Normally, it would be essential to involve which of the
following stakeholders in the initiation stage of a project?
A. System owners
The Big B. System users
Task 3.1 Picture
Evaluate the business case for the
The IS auditor must be
C. System designers
D. System builders
proposed investments in information
able to evaluate
systems acquisition, development,
acquisition and
maintenance and subsequent
development of business
retirement to determine whether it
case selection and
meets business objectives.
management life cycle
methodologies.

23 © Copyright 2016 ISACA. All rights reserved. 24 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 6


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Discussion Question Task 3.2


When reviewing an active project, an IS auditor observed
that the business case was no longer valid because of a
reduction in anticipated benefits and increased costs. The
IS auditor should recommend that the:
A. project be discontinued. Evaluate IT supplier selection and
B. business case be updated and possible corrective contract management processes to
actions be identified.
C. project be returned to the project sponsor for
ensure that service
reapproval. levels and requisite controls are met.
D. project be completed and the business case be
updated later.

25 © Copyright 2016 ISACA. All rights reserved. 26 © Copyright 2016 ISACA. All rights reserved.

Key Terms Task to Knowledge Statements

Key Term Definition How does Task 3.2 relate to each of the following
Request for A document distributed to software vendors, knowledge statements?
proposal (RFP) requesting them to submit a proposal to develop or Knowledge Statement Connection
provide a software product. K3.2 Knowledge of IT acquisition and The IS auditor must understand the variety
vendor management practices (e.g., of vendor provided services (commercial
Requirements A technique used in which the affected user groups evaluation and selection process, contract off-the-shelf hardware/software products,
definition define the requirements of the system for meeting management, vendor risk and relationship outsourced services to include cloud
the defined needs. Some of these are business, management, escrow, software licensing), offerings, managed services, etc.).
regulatory and security-related requirements as including third-party outsourcing
relationships, IT suppliers and service
well as development-related requirements. providers

27 © Copyright 2016 ISACA. All rights reserved. 28 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 7


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

System Acquisition Factors System Specifications


Factors impacting whether to develop or acquire a system When acquiring a new system, the specifications should
include: include the following:
o The date the system needs to be functional o Organizational description (centralized/decentralized,
o The cost to develop the system as opposed to buying it distributed, outsourced, manned or lights-out)
o The resources, staff and hardware required o Hardware and software evaluation assurance levels for
o In a vendor system, the license characteristics (e.g., yearly security robustness
renewal, perpetual) and maintenance costs o Information processing requirements
o Other systems that will need the ability to interface with the o Hardware requirements
new system
o System software applications
o Compatibility with strategic business plans, risk appetite,
o Support requirements
regulatory compliance requirements and the
IT infrastructure o Adaptability and conversion requirements
o Likely future requirements for changes to functionality o System constraints

29 © Copyright 2016 ISACA. All rights reserved. 30 © Copyright 2016 ISACA. All rights reserved.

Requirements Definition
Requirements definition should include descriptions of what a In order to successfully complete a requirements definition,
system should do, how users will interact with a system, the project team will complete tasks such as:
conditions under which the system will operate and the o Identify stakeholders.
information criteria the system should meet. o Record requirements in a structured format and consult
with stakeholders.
o Verify requirements are complete, consistent,
unambiguous, verifiable, modifiable, testable and
traceable.
o Detect and correct conflicts.
o Identify any constraints.
o Resolve conflicts.

31 © Copyright 2016 ISACA. All rights reserved. 32 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 8


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Request For Proposal (RFP)


When determining system requirements, the IS auditor should perform
the following: Product vs. system Product scalability Customer
Vendor
viability/financial
requirements and interoperability references
o Obtain the detailed requirements definition document, and verify its stability
accuracy through interviews.
o Identify the key team members on the project team.
Availability of
o Verify that project initiation and cost have received proper complete and
Vendor support
Source code
Number of years of
experience in
management approval. reliable availability
offering the product
documentation
o Review the conceptual design specifications to ensure that they
address the needs of the user. A list of recent or
Number of client
o Review the conceptual design to ensure that control specifications planned
sites using the Acceptance testing
enhancements to
have been defined. the product, with
product with a list of of the product
current users
o Review the UAT specification. dates

o Determine whether a reasonable number of vendors received a


proposal covering the project scope and user requirements.
o Determine whether an embedded audit routine can be used.
Source: ISACA, CISA Review Manual 26th Edition, figure 3.14

33 © Copyright 2016 ISACA. All rights reserved. 34 © Copyright 2016 ISACA. All rights reserved.

Software Acquisition Process Physical Architecture Analysis


During software acquisition, the IS auditor should perform the
following: Vendor selection

o Analyze the documentation from the feasibility study to


determine whether the decision to acquire a solution was 1. Review of
existing
2. Analysis
3. Draft
4. Functional
5. Define
final 6. Proof of

appropriate.
and design functional
requirements requirements functional concept
architecture
requirements

o Review the RFP to ensure that it covers the items listed and Architecture Architecture Presentation and Architecture Delivery of

whether the selected vendor is supported by the RFP


Workshop 1 Workshop 2 discussion of Workshop 3 prototype
functional

documentation.
requirements

o Attend agenda-based presentations and conference room pilots


to ensure Requirements are validated using a proof of concept.
RFP.
The proof of concept should deliver a working prototype
o Review the vendor contract prior to its signing.
that demonstrates basic setup and functionality.
o Ensure the contract is reviewed by legal counsel before it is
signed.

Source: ISACA, CISA Review Manual 26th Edition, figure 3.24

35 © Copyright 2016 ISACA. All rights reserved. 36 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 9


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Implementation Planning In the Big Picture

1. Procurement Establish the communication process, and determine

Phase
the deliverables, contracts and SLAs. Requirements
statement is produced.

The Big
2. Delivery Time Picture
Develop delivery plan: priorities, goals, key facts,
principles, communication strategies, key indicators,
progress on key tasks and responsibilities. Task 3.2 The IS auditor must
Evaluate IT supplier selection and
understand existing and

3. Installation
contract management processes to
emerging vendor services
and the control needed to
Plan
Develop and review the plan with involved parties. levels and requisite controls are met.
adequately address
associated risks.

4. Installation Develop test plan to include test cases, basic

Test Plan
requirements specifications, definition of processes
and metrics.

Source: ISACA, CISA Review Manual 26th Edition, figure 3.25

37 © Copyright 2016 ISACA. All rights reserved. 38 © Copyright 2016 ISACA. All rights reserved.

Discussion Question Discussion Question


During the audit of an acquired software package, an IS A company has contracted with an external consulting firm
auditor finds that the software purchase was based on to implement a commercial financial system to replace its
information obtained through the Internet, rather than from existing system developed in-house. In reviewing the
responses to a request for proposal (RFP). The IS auditor proposed development approach, which of the following
should FIRST: would be of GREATEST concern?
A. test the software for compatibility with existing A. Acceptance testing is to be managed by users.
hardware. B. A quality plan is not part of the contracted
B. perform a gap analysis. deliverables.
C. review the licensing policy. C. Not all business functions will be available on initial
D. ensure that the procedure had been approved. implementation.
D. Prototyping is being used to confirm that the system
meets business requirements.

39 © Copyright 2016 ISACA. All rights reserved. 40 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 10


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Task 3.3 Key Terms


Key Term Definition
Project A structured set of activities concerned with
delivering a defined capability (that is necessary,
Evaluate the project management but not sufficient, to achieve a required business
outcome) to the enterprise based on an agreed-on
framework and controls to determine schedule and budget.

whether business requirements are Project Portfolio The set of projects owned by a company. It usually
includes the main guidelines relative to each
achieved in a cost-effective manner while project, including objectives, costs, time lines and

managing risks to the organization.


other information specific to the project.
Program A project management technique used in the
Evaluation and planning and control of system projects.
Review Technique
(PERT)

41 © Copyright 2016 ISACA. All rights reserved. 42 © Copyright 2016 ISACA. All rights reserved.

Task to Knowledge Statements


How does Task 3.3 relate to each of the following How does Task 3.3 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.1 Knowledge of benefits realization The IS auditor should understand how the K3.4 Knowledge of project management The IS auditor must understand the need
practices (e.g., feasibility studies, business business defines business cases, control frameworks, practices and tools for an established development
cases, total cost of ownership [TCO], return processes used during feasibility studies management framework within the
on investment [ROI]) and resultant determinations with regard to organization, the constituent elements of a
ROI for development related projects. standard methodology, and the contents
K3.3 Knowledge of project governance The IS auditor needs to understand and deliverables of each phase in order to
mechanisms (e.g., steering committee, program management governance ascertain the degree of necessary audit
project oversight board, project concepts and how to evaluate the program involvement.
management office) office and/or project steering committee K3.5 Knowledge of risk management The IS auditor needs to understand how
integration within the organization practices applied to projects risk management processes are
integrated throughout program
management processes and system and
software development activities.

43 © Copyright 2016 ISACA. All rights reserved. 44 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 11


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

How does Task 3.3 relate to each of the following How does Task 3.3 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.6 Knowledge of requirements analysis The IS auditor must understand the life K3.13 Knowledge of project success The IS auditor needs to understand the
and management practices (e.g., cycle of program, project and unique criteria and project risk specific success criteria for the
requirements verification, traceability, gap system and software development system/software programs and respective
analysis, vulnerability management, requirements. projects.
security requirements)
K3.8 Knowledge of system development
methodologies and tools, including their system/software development
strengths and weaknesses (e.g., agile methodologies and tools enable him or her
development practices, prototyping, rapid to better evaluate the existence and
application development [RAD], effectiveness of critical system
object-oriented design techniques, secure development controls.
coding practices, system version control)

45 © Copyright 2016 ISACA. All rights reserved. 46 © Copyright 2016 ISACA. All rights reserved.

Projects vs. Programs Project Management


The project management approach is dependent on the
Project Programs
size of the organization and complexity of the business.
Has specific objectives, Group of projects and Prior to project involvement, the IS auditor must become
deliverables, and start and time-based tasks closely familiar with the standard or structure used by the
end dates linked through a common
Always time-bound objective organization.
Usually broken into explicit More complex Project management processes include:
phases Usually have a longer
duration, higher budget and o Initiating
higher risk o Planning
Have higher strategic
importance o Executing
o Controlling
o Closing

47 © Copyright 2016 ISACA. All rights reserved. 48 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 12


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Project Context
When analyzing the context of a project, the IS auditor Understanding the environment and context of the
must consider: projects help to identify:
o Importance of the project in the organization o Common objectives for the organization
o o Risk
the project o Resource connections
o Relationship between the project and other projects
o Connection between the project and the underlying
business case

49 © Copyright 2016 ISACA. All rights reserved. 50 © Copyright 2016 ISACA. All rights reserved.

Project Organization Roles and Responsibilities


Influence project organization The audit function should have an active part in application
development projects, often as control experts.
The project manager has only a staff function without formal
management authority.
The CISA should be familiar with general roles and
responsibilities in project management, including:
Pure project organization
The project manager has formal authority over those taking part in the
Senior User Project steering
project. management management committee
Project sponsor

Matrix project organization


Systems Security officer
Management authority is shared between the project manager and the Project manager
development User project and information
department heads. management team system security
and project team engineer

Quality
assurance

51 © Copyright 2016 ISACA. All rights reserved. 52 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 13


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Project Communication Project Culture


Communicate project initiation through: A project culture is comprised of shared norms, beliefs,
o One-on-one meetings values and assumptions of the project team.
o Kick-off meetings The project culture can be defined through a mission
o Project start workshops statement, project name and logo, project office or
meeting place, communication protocols, project
o Combination of the above intranet, etc.
Communication should be open, clearly presented and
documented.

53 © Copyright 2016 ISACA. All rights reserved. 54 © Copyright 2016 ISACA. All rights reserved.

Project Objectives Object Breakdown Structure


Project objectives are the specific action statements that The object breakdown structure (OBS) represents
support the project goals. individual components of the solution and their
Project objectives should always begin with an action hierarchical relationship to each other.
verb.
S
OBS

mart
Customer
Serv ices
Online

M easurable
A project needs clearly
defined results that are: A ttainable
WBS Sales
Application
Development

R ealistic

T imely
WP1 Web WP2 Sales
Page Interface
Development Code
Development

Source: ISACA, CISA Review Manual 26th Edition, figure 3.5

55 © Copyright 2016 ISACA. All rights reserved. 56 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 14


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Work Breakdown Structure Project Management Elements


The work breakdown structure lists all necessary tasks
and groups them into manageable and controllable units.
New System
Implementation Project

Project
System
Management
Deliverables
Deliverables

System
Communication Solution Application Changeover
Infrastructure Requirements Test Cases
Plan Design Development Plan
Setup

Subsystem Design Application


QA Plan
Requirements Documents Code

Data
Conversion
Scope Plan Conversion
Scripts
Specifications

Overall characteristics of successful project planning are that


it is a risk-based management process and iterative in nature.
Risk Plan

Schedule

Source: ISACA, CISA Review Manual 26th Edition, figure 3.6 Source: Personas & Tecnicas Multimedia SL copyright 2009. All rights reserved. Used by permission.

57 © Copyright 2016 ISACA. All rights reserved. 58 © Copyright 2016 ISACA. All rights reserved.

In the Big Picture


The IS auditor should review the adequacy of the following
project management activities:
o Levels of oversight by project committee/board
o Risk management methods The Big
o Issue management Task 3.3 Picture
Evaluate the project management
o Cost management framework and controls to determine
Proper selection of the
project management
o Processes for planning and dependency management
whether business requirements are
approach and software
achieved in a cost-effective manner
development framework is
o Reporting processes while managing risks to the
critical to the success of
organization.
o Change control processes IT initiatives.

o Stakeholder management involvement


o Sign-off process

59 © Copyright 2016 ISACA. All rights reserved. 60 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 15


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Discussion Question Discussion Question


An organization is implementing an enterprise resource While evaluating software development practices in an
planning (ERP) application. Of the following, who is organization, an IS auditor notes that the quality assurance (QA)
PRIMARILY responsible for overseeing the project to function reports to project management. The MOST important
ensure that it is progressing in accordance with the project concern for an IS auditor is the:
plan and that it will deliver the expected results? A. effectiveness of the QA function because it should interact
between project management and user management.
A. Project sponsor
B. efficiency of the QA function because it should interact
B. System development project team (SDPT) with the project implementation team.
C. Project steering committee C. effectiveness of the project manager because the project
D. User project team (UPT) manager should interact with the QA function.
D. efficiency of the project manager because the QA function
will need to communicate with the project implementation
team.

61 © Copyright 2016 ISACA. All rights reserved. 62 © Copyright 2016 ISACA. All rights reserved.

Task 3.4 Key Terms


Key Term Definition
Computer-aided The use of software packages that aid in the development of all
software engineering phases of an information system. System analysis, design
(CASE) programming and documentation are provided. Changes

Conduct reviews to determine whether a introduced in one CASE chart will update all other related charts
automatically. CASE can be installed on a microcomputer for

project is progressing in accordance with easy access.

project plans, is adequately supported by


System development The phases deployed in the development or acquisition of a
life cycle (SDLC) software system. SDLC is an approach used to plan, design,

documentation, and has timely and


develop, test and implement an application system or a major
modification to an application system. Typical phases of the

accurate status reporting.


SDLC include the feasibility study, requirements study,
requirements definition, detailed design, programming, testing,
installation and post-implementation review.
Waterfall development Also known as traditional development, a procedure-focused
development cycle with formal sign-off at the completion of
each level.

63 © Copyright 2016 ISACA. All rights reserved. 64 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 16


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Task to Knowledge Statements


How does Task 3.4 relate to each of the following How does Task 3.4 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.1 Knowledge of benefits realization The IS auditor should understand how to K3.4 Knowledge of project management The IS auditor must be able to evaluate the
practices (e.g., feasibility studies, business measure the project progress with original control frameworks, practices and tools aspects within development frameworks,
cases, total cost of ownership [TCO], return business cases, feasibility studies and the constituent elements of each
on investment [ROI]) resultant determinations (with regard to methodology, and the contents and
ROI) for the development projects. deliverables of each phase.
K3.3 Knowledge of project governance The IS auditor needs to understand how to K3.5 Knowledge of risk management The IS auditor needs to apply and evaluate
mechanisms (e.g., steering committee, evaluate the program management practices applied to projects the risk management processes during the
project oversight board, project governance and the program office and/or evaluation of program management
management office) project steering committee integration processes and system and software
within the organization. development activities.

65 © Copyright 2016 ISACA. All rights reserved. 66 © Copyright 2016 ISACA. All rights reserved.

Project Planning
How does Task 3.4 relate to each of the following When planning a project, the project manager needs to
knowledge statements? determine the various tasks to be performed, as well as
Knowledge Statement Connection the following:
K3.8 Knowledge of system development o Task sequence
methodologies and tools, including their differing methodologies enables
strengths and weaknesses (e.g., agile them to better evaluate the existence and o Task duration
development practices, prototyping, rapid effectiveness of critical system
application development [RAD], development controls with each o Task priority
object-oriented design techniques, secure
coding practices, system version control)
methodology.
o Task budget
K3.13 Knowledge of project success The IS auditor needs to evaluate the o Task resources
criteria and project risk specific success criteria for the
system/software programs and respective During project execution, the project manager must
projects and how management is control the scope, resource usage and risk.
evaluating progress toward meeting these
criteria along with addressing deficiencies
that put the program and respective
projects at risk.

67 © Copyright 2016 ISACA. All rights reserved. 68 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 17


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Project Budgeting Tools Project Scheduling Tools


System Dev. This includes four estimating methodologies: analogous estimating, The following techniques and tools can be used for
estimating project schedules:
Project Cost parametric estimating, bottom-up estimating and actual costs.
Estimation

Software Size This method determines the relative physical size of the application
Estimation software. Critical Path This method lays out project activities as a network of branches and
Methodology calculates the longest path of planned activities to determine the shortest
possible completion.
Function Point FPA measures the size of an information system based on the number and
Analysis complexity of the inputs, outputs, files, interfaces and queries. Gantt Charts This tool charts when an activity should begin and when it should end.

Cost Budgets This method estimates work effort, including personnel hours, machine
hours and other external costs, and multiplies the effort by the hourly rate. Program PERT uses three different estimates of each activity duration and then
Evaluation applies a CPM algorithm to reduce it to a single number.
Review Technique
Software Cost This tool defines all cost drivers and then develops a cost estimate of the
Estimation system and total project. Timebox This technique defines software deliverables that have short and fixed
Management timeframes.

69 © Copyright 2016 ISACA. All rights reserved. 70 © Copyright 2016 ISACA. All rights reserved.

SDLC
SDLC critical success factors include:
o Productivity
Phase 1 Feasibility Study

Phase 2 Requirements Definition o Quality


o Economic value
o Customer service
Phase 3A Software Selection
Phase 3B Design
and Acquisition

The main advantage of SDLC is that it provides a


template into which methods for the requirements can be
Phase 4A Configuration Phase 4B Development

Phase 5 Final Testing and


placed.
Implementation

Phase 6
Postimplementation

Source: ISACA, CISA Review Manual 26th Edition, figure 3.12

71 © Copyright 2016 ISACA. All rights reserved. 72 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 18


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

IS Auditor Role in SDLC


The IS auditor should be aware that merely following an The IS auditor should ensure that:
SDLC management approach does not ensure the o
successful completion of a development project. objectives.
o Project planning is performed, including effective
estimates of resources, budget and time.
o Scope creep is controlled and there is a software
baseline.
o Management is tracking software design and
development activities.
o Senior management support is provided.
o Periodic review and risk analysis is performed in each
project phase.

73 © Copyright 2016 ISACA. All rights reserved. 74 © Copyright 2016 ISACA. All rights reserved.

Business Application Development Business Application Systems


Two major categories include: Business application systems may reside in the following environments:

o Organization- objective is to o
o
E-commerce
Electronic data interchange
collect, collate, store, archive and share information o Email
with business users and various applicable support o Point-of-sale (POS) systems
functions. o
o
Electronic banking and electronic finance
Payment systems and electronic funds transfer (EFT)
o End-user- The objective is to o Automated teller machines (ATM)
provide different views of data for their performance o Purchase accounting systems
o Integrated manufacturing systems
optimization. o Industrial control systems (ICS)
o Interactive voice response (IVR)
o Image processing
o Artificial intelligence (AI) and business intelligence systems
o Decision support system (DSS)
o Customer relationship management (CRM)
o Supply chain management (SCM)

75 © Copyright 2016 ISACA. All rights reserved. 76 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 19


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Development Methods
Agile Development Prototyping Development
A family of similar development processes that espouse a The process of quickly putting together a working model (a
nontraditional way of developing complex systems. These prototype) in order to test various aspects of a design, illustrate
ideas or features and gather early user feedback. Prototyping
flexibly handle changes to the system being developed or the uses programmed simulation techniques to represent a model
project that is performing the development. of the final system to the user for advisement and critique. The
emphasis is on end-user screens and reports. Internal controls
are not a priority item since this is only a model.
Rapid Application Development (RAD)
A methodology that enables enterprises to develop strategically
important systems faster, while reducing development costs
and maintaining quality by using a series of proven application
development techniques, within a well-defined methodology.

77 © Copyright 2016 ISACA. All rights reserved. 78 © Copyright 2016 ISACA. All rights reserved.

Object-Oriented System Development Web-Based Application Development


OOSD is a programming technique that groups data and This approach uses XML languages (SOAP, WSDL, UDDI) to
procedures into objects, which permits analysts, developers provide more effective integration of code modules within and
and programmers to consider larger logical chunks of a system between enterprises.
and clarify the programming process. Software Reengineering
OOSD allows for the management of an unrestricted variety of
data, the ability to model complex relationships and the ability to This is a process involving the extraction of components from
meet demands of a changing environment. existing systems and restructuring these components to develop
new systems or to enhance the efficiency of existing systems.
Component-Based Development Existing software systems thus can be modernized to prolong their
functionality.
This method assembles applications from cooperating
packages of executable software that make their services Reverse Engineering
available through defined interfaces.
This is a software engineering technique whereby existing
It reduces development time and cost, improves quality, application system code can be redesigned and coded using
promotes modularity and simplifies reuse. computer-aided software engineering (CASE) technology.

79 © Copyright 2016 ISACA. All rights reserved. 80 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 20


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

System Development Tools


Computer-aided software engineering (CASE) Code generators generate program code
of automated tools to aid in the software development based on parameters defined by a systems analyst or on
process. The IS auditor must be able to recognize data/entity flow diagrams developed by the design
changes in the development process brought on by module of a CASE product. The IS auditor should be
CASE and may use CASE as an audit tool. aware of source code generated by such tools.

81 © Copyright 2016 ISACA. All rights reserved. 82 © Copyright 2016 ISACA. All rights reserved.

Fourth-generation languages (4GLs) During the design and development phases, the IS auditor should do
the following:
languages that are environmentally independent and
o Review the system flowcharts for adherence to the general design.
have simple language subsets and a workbench
o Verify that appropriate approvals were obtained for any changes.
approach.
o Review the input, processing and output controls designed into the
system for appropriateness.
o Interview the key users to determine their understanding of how the
system will operate.
o Assess the adequacy of audit trails to provide traceability and
accountability of system transactions.
o Verify the integrity of key calculations and processes.
o Verify that the system can identify and process erroneous data
correctly.
o Review the quality assurance results.
o Verify that all recommended corrections were made.

83 © Copyright 2016 ISACA. All rights reserved. 84 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 21


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

In the Big Picture Discussion Question


Which of the following would BEST help to prioritize project
activities and determine the time line for a project?
A. A Gantt chart
The Big B. Earned value analysis (EVA)
Task 3.4 Picture C. Program evaluation review technique (PERT)
Conduct reviews to determine whether Both project management
a project is progressing in accordance and software D. Function point analysis (FPA)
with project plans, is adequately development
supported by documentation, and has implementation are
timely and accurate status reporting. critical to project success.

85 © Copyright 2016 ISACA. All rights reserved. 86 © Copyright 2016 ISACA. All rights reserved.

Discussion Question Task 3.5


An IS auditor has found time constraints and expanded
needs to be the root causes for recent violations of
corporate data definition standards in a new business Evaluate controls for information
intelligence project. Which of the following is the MOST
appropriate suggestion for an auditor to make? systems during the requirements,
A. Achieve standards alignment through an increase of acquisition, development and testing
resources devoted to the project. phases for compliance with the
B. Align the data definition standards after completion
of the project.
procedures and applicable external
C. Delay the project until compliance with standards
can be achieved. requirements.
D. Enforce standard compliance by adopting punitive
measures against violators.
87 © Copyright 2016 ISACA. All rights reserved. 88 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 22


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Key Terms Task to Knowledge Statements

Key Term Definition How does Task 3.5 relate to each of the following
Application The policies, procedures and activities designed to knowledge statements?
controls provide reasonable assurance that objectives Knowledge Statement Connection
relevant to a given automated solution (application) K3.2 Knowledge of IT acquisition and Identify key controls required to mitigate
are achieved vendor management practices (e.g., risks associated with vendor evaluation and
evaluation and selection process, contract selection processes and terms and
Input control Techniques and procedures used to verify, validate management, vendor risk and relationship conditions within vendor contracts.
and edit data to ensure that only correct data are management, escrow, software licensing),
entered into the computer including third-party outsourcing
relationships, IT suppliers and service
providers
K3.4 Knowledge of project management The acquisition process keys upon
control frameworks, practices and tools proactive and responsive project
management practices and tools ensuring
the services and material are acquired to
meet project goals and objectives.

89 © Copyright 2016 ISACA. All rights reserved. 90 © Copyright 2016 ISACA. All rights reserved.

Task to Knowledge

How does Task 3.5 relate to each of the following How does Task 3.5 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.5 Knowledge of risk management Through focused and integrated risk K3.7 Knowledge of enterprise architecture Based on the defined system
practices applied to projects management analysis, the IS auditor can related to data, applications and technology requirements, the project team must then
proactively identify issues that can (e.g., web-based applications, web translate these requirements into defined
negatively impact a project. services, n-tier applications, cloud services, system architectures. The IS auditor must
K3.6 Knowledge of requirements analysis Failure to accurately and completely virtualization) understand how the requirements map to
and management practices (e.g., document all applicable technical, the resultant architectures and that the
requirements verification, traceability, gap operational and functional requirements will selected architecture is properly selected.
analysis, vulnerability management, lead to project delays, overruns and even K3.8 Knowledge of system development The project team needs to select the
security requirements) failure. methodologies and tools, including their correct system development method based
strengths and weaknesses (e.g., agile on system complexity and the need to
development practices, prototyping, rapid implement new systems more quickly to
application development [RAD], object- achieve benefits before the business
oriented design techniques, secure coding changes.
practices, system version control)

91 © Copyright 2016 ISACA. All rights reserved. 92 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 23


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Task to Knowledge Task to Knowledge

How does Task 3.5 relate to each of the following How does Task 3.5 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.9 Knowledge of control objectives and For each phase of the system development K3.11 Knowledge of configuration and Project management must establish
techniques that ensure the completeness, project, specific control objectives must be release management relating to the configuration management processes from
accuracy, validity, and authorization of documented and a control mechanism in development of information systems the very start through post-implementation
transactions and data place for project success. IS auditors are turnover to operations and the subsequent
the control engineers based on their deep system upgrades and decommissioning.
understanding of risk management K3.12 Knowledge of system migration and The project team must plan and develop
practices. infrastructure deployment practices and the tools and processes for migrating new,
K3.10 Knowledge of testing methodologies The project team must plan, develop and data conversion tools, techniques and upgraded and modified systems to ensure
and practices related to the information complete appropriate testing in order to procedures desired system functionality is retained
system development life cycle (SDLC) confirm that all documented system throughout these activities.
requirements are met.

93 © Copyright 2016 ISACA. All rights reserved. 94 © Copyright 2016 ISACA. All rights reserved.

Virtualization Virtualization Controls


. . . The IS auditor will need to understand the following concepts:
. . .
. . . o Hypervisors and guest images (OS and networks) are securely
. . configured according to industry standards. Apply hardening to
. . Application Application Application
. . these virtual components as closely as one would to a physical
Application Application Application Guest OS Guest OS server, switch, router, firewall or other computing device.
Guest OS Guest OS Application Hypervisor
o Hypervisor management communications should be protected on a
dedicated management network.
Hypervisor Host OS o The hypervisor should be patched as the vendor releases the fixes.
Hardware Hardware
o The virtualized infrastructure should be synchronized to a trusted
authoritative time server.
Bare metal Hosted o Unused physical hardware should be disconnected from the host
system.
To develop effective audit programs, the IS auditor must obtain a clear understanding of
both virtualization and cloud service provider (CSP) architectures supporting the o All hypervisor services should be disabled unless they are needed.
and processes.
o Host inspection capabilities should be enabled to monitor the
security of each guest OS and of each activity occurring between
Source: Reprinted courtesy of the National Institute of Standards and Technology, U.S. Department of Commerce. Not copyrightable in the United States . guest OSs.

95 © Copyright 2016 ISACA. All rights reserved. 96 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 24


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Application Controls Input Controls


Application controls ensure that: Input controls ensure that only valid and authorized
o Only complete, accurate and information are input and that these transactions are only
processed once.
valid data are entered and Input

updated in a computer
system.
o Processing accomplishes
the correct task. Application

o Processing results
Controls

meet expectations.
o Data are maintained.
Output Processing

97 © Copyright 2016 ISACA. All rights reserved. 98 © Copyright 2016 ISACA. All rights reserved.

Input authorization verifies that all transactions have Batch controls and balancing group input transactions to
been authorized and approved by management. Types provide control totals. Types of batch controls and
of authorization include: balances include:
o Signatures on batch forms or source documents o Total monetary amount
o Online access controls o Total items
o Unique passwords o Total documents
o Terminal or client workstation identification o Hash totals
o Source documents o Batch registers
o Control accounts
o Computer agreements

99 © Copyright 2016 ISACA. All rights reserved. 100 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 25


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Processing Controls
Input error handling verifies that only correct data is Processing procedures and controls are meant to ensure
accepted into a system. It can be processed by the the reliability of application program processing.
following:
o Rejecting only transactions with errors
o Rejecting the whole batch of transactions
o Holding the batch in suspense
o Accepting the batch and flagging error transactions

101 © Copyright 2016 ISACA. All rights reserved. 102 © Copyright 2016 ISACA. All rights reserved.

Data validation and editing procedures ensure that input data Processing controls are meant to ensure the
are validated and edited as close to the time and point of
origination as possible. completeness and accuracy of accumulated data.
o Sequence check o Manual recalculations
o Limit check o Editing
o Range check
o Validity check o Run-to-run totals
o Reasonableness check o Programmed controls
o Table lookups o Reasonableness verification of calculated amounts
o Existence check
o Key verification o Limit checks on amounts
o Check digit o Reconciliation of file totals
o Completeness check o Exception reports
o Duplicate check
o Logical relationship check

103 © Copyright 2016 ISACA. All rights reserved. 104 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 26


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Output Controls
Data file controls ensure that only authorized processing
occurs to stored data.
o Before and after image reporting Logging and storage of negotiable, sensitive
Output controls
o Maintenance error reporting and handling provide
and critical forms in a secure place
Computer generation of negotiable
o Source documentation retention assurance that instruments, forms and signatures
o Internal and external labeling the data delivered Report accuracy, completeness and
to users will be timeliness
o Version usage
presented, Reports generated from the system
o Data file security formatted and Report distribution
Balancing and reconciling
o One-for-one checking delivered in a Output error handling
o Prerecorded input consistent and Output report retention
secure manner.
o Transaction logs Verification of receipt of reports

o File updating and maintenance authorization


o Parity checking
105 © Copyright 2016 ISACA. All rights reserved. 106 © Copyright 2016 ISACA. All rights reserved.

Application Control Documentation

The IS auditor should review the following


o Identifying significant application components and the documentation to gain an understanding of the
flow of transactions
o Identifying the application control strengths and
evaluating the impact of the control weaknesses System
Functional
o Developing a testing strategy development
design
Program
methodology changes
o Testing the controls to ensure their functionality and documents
specifications
effectiveness
o Evaluating the control environment by analyzing the Technical
test results and other audit evidence to determine that User manuals reference
documentation
control objectives were achieved
o Considering the operational aspects of the application
to ensure its efficiency and effectiveness
107 © Copyright 2016 ISACA. All rights reserved. 108 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 27


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Application Control Testing Continuous Online Auditing


The IS auditor must test application controls to ensure Systems Control Audit Review
their functionality and effectiveness. File and Embedded Audit Embeds specially written software in the host
application system to monitor it on a selective basis
Modules (SCARF/EAM)
Some of the methods and techniques to test the
application system include: Snapshots
Captures the processing path a transaction follows
and applies identifiers for subsequent reviews

o Snapshot o Integrated test facility


o Mapping o Parallel simulation Audit Hooks
Embeds hooks in the application system to function as
o Tracing and tagging o Transaction selection red flags, which allows IS or the auditor to intervene

o Test data/deck programs


o Base-case system o Embedded audit data Integrated Test Facility (ITF)
Sets up dummy entities on the production files to
confirm the correctness of the processing
evaluation collection
o Parallel operation o Extended records
Continuous and Intermittent Simulates the execution of an application and audits
Simulations (CIS) the transaction if it meets predetermined criteria

109 © Copyright 2016 ISACA. All rights reserved. 110 © Copyright 2016 ISACA. All rights reserved.

In the Big Picture Discussion Question


Which of the following would be the BEST approach to
ensure that sufficient test coverage will be achieved for a
project with a strict end date and a fixed time to perform
The Big testing?
Task 3.5
Evaluate controls for information Picture A. Requirements should be tested in terms of
systems during the requirements, importance and frequency of use.
acquisition, development and testing during the system
phases for compliance with the acquisition and B. Test coverage should be restricted to functional
requirements.
development processes is
procedures and applicable external a key success factor for all
requirements. projects.
C. Automated tests should be performed through the
use of scripting.
D. The number of required test runs should be reduced
by retesting only defect fixes .

111 © Copyright 2016 ISACA. All rights reserved. 112 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 28


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Discussion Question Task 3.6


Who should review and approve system deliverables as
they are defined and accomplished to ensure the
successful completion and implementation of a new
business system application?
Evaluate the readiness of information
A. User management
systems for implementation and
B. Project steering committee
C. Senior management
migration into production to determine
D. Quality assurance staff whether project deliverables, controls
.

113 © Copyright 2016 ISACA. All rights reserved. 114 © Copyright 2016 ISACA. All rights reserved.

Key Terms
Key Term Definition Key Term Definition
Quality assurance A planned and systematic pattern of all actions Test data Simulated transactions that can be used to test
(QA) necessary to provide adequate confidence that an item processing logic, computations and controls actually
or product conforms to established technical programmed in computer applications. Individual
requirements (ISO/IEC 24765). programs or an entire system can be tested. This
System testing Testing conducted on a complete, integrated system to technique includes integrated test facilities (ITFs) and
base case system evaluations (BCSEs).
requirements. System test procedures typically are Test programs Programs that are tested and evaluated before approval
performed by the system maintenance staff in their into the production environment. Test programs, through
development library. a series of change control moves, migrate from the test
environment to the production environment and become
production programs.

115 © Copyright 2016 ISACA. All rights reserved. 116 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 29


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Task to Knowledge Statements


How does Task 3.6 relate to each of the following How does Task 3.6 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.5 Knowledge of risk management Based on defined project schedules, K3.8 Knowledge of system development Each type of system development
practices applied to projects system and resource requirements, the IS methodologies and tools, including their methodology has specific sequencing of
auditor must perform risk analysis strengths and weaknesses (e.g., agile development activities and respective
throughout all phases of the project to development practices, prototyping, rapid deliverable. The IS auditor must be able to
confirm the impacts of any deficiencies application development [RAD], understand how to trace these deliverables
identified during the evaluation. object-oriented design techniques, secure to system specifications and requirements
K3.6 Knowledge of requirements analysis The ability to efficiently trace documented coding practices, system version control) prior to production acceptance.
and management practices (e.g., requirements to system architecture, K3.9 Knowledge of control objectives and Based on the established control objective
requirements verification, traceability, gap design and completed testing is critical to techniques that ensure the completeness, for each phase of the system development
analysis, vulnerability management, production cut-over and go no-go accuracy, validity, and authorization of project, the IS auditor needs to evaluate
security requirements) decisions. transactions and data the respective controls that are in place to
meet these objectives prior to a go-live
decision.

117 © Copyright 2016 ISACA. All rights reserved. 118 © Copyright 2016 ISACA. All rights reserved.

Task to Knowledge Task to Knowledge

How does Task 3.6 relate to each of the following How does Task 3.6 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.10 Knowledge of testing methodologies Based on the testing methods applied, K3.12 Knowledge of system migration and One critical project management decision
and practices related to the information testing content and delivery schedules will infrastructure deployment practices and involves the method selected for migrating
system development life cycle (SDLC) differ, and the IS auditor must be able to data conversion tools, techniques and the completed system into the production
determine the timing and specific testing procedures environment. This is based on size,
that needs to be completed prior to system complexity and business-driven
go-live. requirements.
K3.11 Knowledge of configuration and Maintaining an accurate and complete K3.13 Knowledge of project success Through the use of establishing KPIs
release management relating to the hardware, software and process baseline is criteria and project risk related to system requirements, at the start
development of information systems critical to ensure the system not only is of the project the project management
ready for migration to production office can use these as the benchmark to
operations but can also be maintained over determine project success.
its product life.

119 © Copyright 2016 ISACA. All rights reserved. 120 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 30


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Testing Types of Testing


Testing determines that the user requirements have
been validated, the system is performing as anticipated Unit testing
and internal controls work as intended. Tests program logic within a particular program or module
Ensures that the internal operation of the program performs according to
The two primary approaches to testing include: specification
Uses a set of test cases that focus on the control structure of the procedural design
o Interface or integration testing
upward until a complete system testing has taken A hardware or software test that evaluates the connection of two or more
place. components that pass information from one area to another

o System testing
work downward to individual units. A series of tests designed to ensure that modified programs, objects, database
schema, etc., which collectively constitute a new or modified system, function
properly
Final acceptance testing
System testing that takes place during the implementation phase and applies the

121 © Copyright 2016 ISACA. All rights reserved. 122 © Copyright 2016 ISACA. All rights reserved.

Final Acceptance Testing Other Types of Testing


Final acceptance testing has two major parts: Test Type Description
Alpha and beta The first stage, called alpha testing, is often performed on an
Quality Assurance Testing testing early version of the application system only by users within
User Acceptance Testing (UAT)
(QAT) the organization developing the software (i.e., systems
Focuses on technical aspects Focuses on functional aspect testing). The second stage, called beta testing, a form of
of the application of the application user acceptance testing, generally involves a limited number
Verifies that the application Ensures that the system is of external users and involves real-world exposure.
works as documented by production-ready and satisfies Pilot testing A preliminary test that focuses on specific and
testing the logical design and all documented requirements predetermined aspects of a system, such as a proof of
the technology itself Performed in a secure testing concept.
Ensures that the application or staging environment that
meets the documented mimics production as close as White box A testing approach that uses knowledge of a
technical specifications and possible testing
deliverables intervals to verify its expected behavior.
Involves minimal perspective
end-user participation Black box A testing approach that focuses on the functionality of the
Performed by the IT
Performed by IT department department and the end user testing application or product and does not require knowledge of the
code intervals.

123 © Copyright 2016 ISACA. All rights reserved. 124 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 31


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

User Performance Testing


Test Type Description Some of the user procedures that should be observed and tested include:
Function/validation Tests the functionality of the system against the detailed SoD
testing requirements to ensure that the software that has been than one of the following processes: origination, authorization,
built is traceable to customer requirements verification or distribution.
Regression testing The process of rerunning a portion of a test scenario or
test plan to ensure that changes or corrections have not unique passwords.
introduced new errors -to-run control totals and other application
Parallel testing The process of feeding test data into two systems the
totals are reconciled on a timely basis.
modified system and an alternative system (possibly the
original system) and comparing the results appropriate review, research, timely correction and resubmission.
Sociability testing Test to confirm that the new or modified system can secure manner.
operate in its target environment without adversely
impacting existing systems information on access levels by individuals.

attempts.

125 © Copyright 2016 ISACA. All rights reserved. 126 © Copyright 2016 ISACA. All rights reserved.

Data Integrity Testing


Data integrity testing is a set of substantive tests that During testing, the IS auditor should perform the following:

examine accuracy, completeness, consistency and o Review the test plan, error reports, end user documentation and procedures
used for completeness and accuracy.
authorization of data presently held in a system. Two o Reconcile control totals and converted data.
common types include: o Verify cyclical processing and critical reports for accuracy.

o Relational integrity o Interview end users of the system for their understanding of new methods,
procedures and operating instructions.
performed at the data element and record-based o Verify that system security is functioning as designed.
levels. o Review parallel testing results and the user acceptance testing.
o Referential integrity Define existence o Review unit and system test plans to determine whether tests for internal
controls are planned and performed.
relationships between entities in different tables of a o Review the user acceptance testing and ensure that the accepted software
database that needs to be maintained by the DBMS. has been delivered to the implementation team. The vendor should not be
able to replace this version.
o Review procedures used for recording and following through on error
reports.

127 © Copyright 2016 ISACA. All rights reserved. 128 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 32


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

In the Big Picture Discussion Question


An IS auditor is reviewing the software development
process for an organization. Which of the following
functions would be appropriate for the end users to
The Big perform?
Task 3.6 Picture A. Program output testing
Evaluate the readiness of information
B. System configuration
Prior to system
systems for implementation and
production cut-over, IS
migration into production to determine
whether project deliverables, controls
auditors must be able to
effectively provide
C. Program logic specification
met.
management with their D. Performance tuning
assessment as to system
readiness.

129 © Copyright 2016 ISACA. All rights reserved. 130 © Copyright 2016 ISACA. All rights reserved.

Discussion Question Task 3.7


From a risk management point of view, the BEST approach
when implementing a large and complex IT infrastructure
is:
A. a major deployment after proof of concept.
B. prototyping and a one-phase deployment. Conduct post-implementation reviews of
C. a deployment plan based on sequenced phases. systems to determine whether project
D. to simulate the new infrastructure before deployment.
requirements are met.

131 © Copyright 2016 ISACA. All rights reserved. 132 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 33


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Key Terms Task to Knowledge Statements


Key Term Definition How does Task 3.7 relate to each of the following
Business process The thorough analysis and significant redesign of business knowledge statements?
reengineering (BPR) processes and management systems to establish a better
performing structure, more responsive to the customer base Knowledge Statement Connection
and market conditions, while yielding material cost savings. K3.1 Knowledge of benefits realization Once the system go-live has occurred, the
Change management A holistic and proactive approach to managing the transition practices (e.g., feasibility studies, business IS auditor needs to determine if the system
from a current to a desired organizational state, focusing cases, total cost of ownership [TCO], return has delivered services and value as
on investment [ROI]) documented in the business case and ROI
Includes activities such as culture change (values, beliefs and calculations.
attitudes), development of reward systems (measures and K3.4 Knowledge of project management Project management closure process
appropriate incentives), organizational design, stakeholder control frameworks, practices and tools should determine whether project
management, human resources (HR) policies and procedures, objectives were met or excused and should
executive coaching, change leadership training, team building, identify lessons learned to avoid mistakes
and communication planning and execution. and encourage repetition of good practices.
Configuration The control of changes to a set of configuration items over a K3.9 Knowledge of control objectives and Review controls built into the system to
management system life cycle. techniques that ensure the completeness, ensure that they are operating according to
accuracy, validity and authorization of design.
transactions and data

133 © Copyright 2016 ISACA. All rights reserved. 134 © Copyright 2016 ISACA. All rights reserved.

How does Task 3.7 relate to each of the following How does Task 3.7 relate to each of the following
knowledge statements? knowledge statements?
Knowledge Statement Connection Knowledge Statement Connection
K3.10 Knowledge of testing methodologies Review evidence (test plans and test K3.14 Knowledge of post-implementation
and practices related to the information results) to ensure that procedures are review objectives and practices (e.g., requirements were achieved. Careful
system development life cycle (SDLC) carried out as prescribed by organizational project closure, control implementation,
standards. benefits realization, performance utilization, trouble tickets, work orders and
K3.13 Knowledge of project success Based on established project KPIs being measurement) overall satisfaction with the system. This
criteria and project risk
objectives, the IS auditor can use under objectives and requirements were
performance of project KPIs to correlate to achieved.
higher project risk.

135 © Copyright 2016 ISACA. All rights reserved. 136 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 34


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Implementation Planning Implementation Planning Steps


After successful testing, the system is implemented

procedures. Develop a gap analysis process.


Define required roles.
An implementation plan should be prepared well in
advance of the implementation date.
Develop service level agreements (SLAs). SLAs should consider:
Each step of setting up the production environment Operating time
should be documented, including who will be Support time
responsible, how the step will be verified and the Meantime between failures (MTBF)
back-out procedure. Meantime to repair (MTTR)
Technical support response time
Implementation plan/knowledge transfer plan
Develop training plans:
Staff training
End user training

137 © Copyright 2016 ISACA. All rights reserved. 138 © Copyright 2016 ISACA. All rights reserved.

Post-implementation
During the implementation phase, the IS auditor should Post-implementation reviews are typically conducted
perform the following: after the project has been in use long enough to realize
o Verify appropriate sign-offs have been obtained. its business benefits and costs and to measure the
o Review the programmed procedures used for
scheduling and running the system. units.
o Review all system documentation to ensure its Metrics include:
completeness. o Total cost of ownership (TCO)
o Verify all data conversion to ensure that they are o Return on investment (ROI)
correct and complete.

139 © Copyright 2016 ISACA. All rights reserved. 140 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 35


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Post-implementation Review Project Close


During the post-implementation review, the IS auditor should Projects have a finite life. Once the project is closed, it is
perform the following: handed over to end users.
o were
achieved. During project closure:
o Determine if the cost benefits are being measured, analyzed and o Assign outstanding issues.
accurately reported to management. o Assign custody of contracts.
o Review program change requests performed to assess the type
of changes required of the system.
o Archive or hand off documentation.
o Review controls to ensure that they are operating according to o Discuss lessons learned.
design. o Conduct a post-project review.
o any
resource or operating problems.
o Review input and output control balances and reports to verify
that the system is processing data accurately.

141 © Copyright 2016 ISACA. All rights reserved. 142 © Copyright 2016 ISACA. All rights reserved.

Certification and Accreditation System Maintenance


Certification is a process by which an assessor Following implementation, a system enters into the
performs a comprehensive assessment against a ongoing development or maintenance stage.
standard of management and operational and technical System maintenance practices refer primarily to the
controls and determines the level of compliance. process of managing change to application systems
o The goal is to determine the extent to which controls while maintaining the integrity of both the production and
are implemented correctly, operating as intended and application source and executable code.
producing the desired outcome. A standard change management process needs to be in
Accreditation authorizes operation of an information place for recording and performing changes, which is
system, thereby accepting the risk. A senior official typically established during the project design phase.
accepts responsibility and is fully accountable for any
adverse impacts.

143 © Copyright 2016 ISACA. All rights reserved. 144 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 36


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Change Management
Change management is a process to document and A change management process should include the
authorize any change requests. procedures for the following:
Change requests are initiated from the end user, o A formal change request process
operational staff, and system development and o Documentation
maintenance staff. o Testing of changes
o Emergency changes
o Deploying changes into production
o Handling unauthorized changes

145 © Copyright 2016 ISACA. All rights reserved. 146 © Copyright 2016 ISACA. All rights reserved.

Configuration Management
Configuration management uses change management The IS auditor should review the change management
processes along with checkpoints, reviews and sign-off process for possible improvements in the following:
procedures. o Change request methodology and procedures
o Response time and response effectiveness
Develop the Baseline Analyze and Develop o User satisfaction
o Security access restrictions
configuration applicable report on the configuration
management plan. components. results. status reports.

o Emergency procedures
o Acknowledgement and resolution of items on the
change control log
Update the
Perform
configuration Develop release
configuration
status accounting procedures.
control activities.
database.

147 © Copyright 2016 ISACA. All rights reserved. 148 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 37


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

BPR Methods and Techniques


BPR Steps BPR Method Description
Benchmarking A continuous, systematic process for evaluating the products,
Define the areas Develop a Process services or work processes of organizations recognized as a
to be reviewed. project plan. world-
ISO 9126 An international standard to assess the quality of software
products
Capability Maturity A model used by many organizations to identify best practices
Redesign and
Gain an Model Integration useful in helping them assess and increase the maturity of their
understanding of
streamline the
the process (CMMI) software development processes
process.
under review. ISO/IEC 330xx A series of standards that provide guidance on process
assessment
Business Process A technique to evaluate controls at the process and activity level
Establish a Control Assurance and the controls specific to the business process owner
Implement and
continuous
monitor the new
improvement
process.
process.

149 © Copyright 2016 ISACA. All rights reserved. 150 © Copyright 2016 ISACA. All rights reserved.

In the Big Picture Discussion Question


During a postimplementation review, which of the following
activities should be performed?
A. User acceptance testing (UAT)
The Big B. Return on investment (ROI) analysis
Task 3.7 Picture C. Activation of audit trails
Effective
D. Updates of the state of enterprise architecture (EA)
Conduct post-implementation reviews
post-implementation
of systems to determine whether
diagrams
evaluations determine if
project deliverables, controls and
objectives
and requirements were
achieved.

151 © Copyright 2016 ISACA. All rights reserved. 152 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 38


CISA Review Course 26th Edition Domain 3: Information Systems Acquisition,
Development and Implementation

Discussion Question Domain 3 Summary


The PRIMARY objective of conducting a In this domain we have covered the following:
postimplementation review for a business process o Evaluating the business case for the proposed information
automation project is to: systems acquisition and development
o Evaluating IT supplier selection and contract management
A. ensure that the project meets the intended business processes
requirements. o Evaluating the project management framework and controls
B. evaluate the adequacy of controls. o Conducting reviews to determine whether a project is
C. confirm compliance with technological standards. progressing in accordance with project plans
o Evaluating controls for information systems during the
D. confirm compliance with regulatory requirements. requirements, acquisition, development and testing phases
o Evaluating the readiness of information systems for
implementation and migration into production
o Conducting post-implementation reviews of systems to
determine whether project deliverables, controls and
requirements are met

153 © Copyright 2016 ISACA. All rights reserved. 154 © Copyright 2016 ISACA. All rights reserved.

Discussion Question Discussion Question


A legacy payroll application is migrated to a new An IS auditor should ensure that review of online electronic
application. Which of the following stakeholders should be funds transfer (EFT) reconciliation procedures should
PRIMARILY responsible for reviewing and signing-off on include:
the accuracy and completeness of the data before going A. vouching.
live?
B. authorizations.
A. IS auditor C. corrections.
B. Database administrator D. tracing.
C. Project manager
D. Data owner

155 © Copyright 2016 ISACA. All rights reserved. 156 © Copyright 2016 ISACA. All rights reserved.

© 2016. ISACA. All Rights Reserverd. 39

You might also like