CSQE 2002 001 Transp S

QUALITY COUNCIL OF INDIANA INTRO-1 (1)
CSQE 2002
THE SOFTWARE
QUALITY ENGINEER
PRIMER
2002 by Quality Council of Indiana
All rights reserved
Third Edition
Bill Wortman
Quality Council of Indiana
602 West Paris Avenue
West Terre Haute, IN 47885
TEL: 812-533-4215
FAX: 812-533-4216
qci@qualitycouncil.com
http://www.qualitycouncil.com
001
CSQE 2002
CSQE Primer Contents

I. CERTIFICATION OVERVIEW . . . . . . . . . . . . . . . . . I-1
SOFTWARE QUALITY ENGINEER EXAM . . . . . I-3
CSQE BODY OF KNOWLEDGE . . . . . . . . . . . . I-10
II. GENERAL KNOWLEDGE, CONDUCT AND

ETHICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . II-1
QUALITY PHILOSOPHIES & PRINCIPLES . . . . II-2
BENEFITS OF SOFTWARE QUALITY . . . . II-2
PREVENTION VERSUS DETECTION . . . . II-15
ORGAN. & PROCESS BENCHMARKING II-16
STANDARDS, SPECIFICATIONS & MODELS II-18
QUALITY MANAGEMENT SYSTEM
MODELS . . . . . . . . . . . . . . . . . . . . . . . II-18
SOFTWARE STANDARDS . . . . . . . . . . . . II-50
SOFTWARE ASSESSMENT MODELS . . . II-55
OTHER SOFTWARE STANDARDS . . . . . II-80
LEADERSHIP TOOLS & SKILLS . . . . . . . . . . II-85
ORGANIZATIONAL LEADERSHIP . . . . . . II-85
TEAM MANAGEMENT . . . . . . . . . . . . . . II-101
TEAM TOOLS . . . . . . . . . . . . . . . . . . . . . II-112
FACILITATION SKILLS . . . . . . . . . . . . . . II-119
COMMUNICATION SKILLS . . . . . . . . . . II-129
ETHICS & PROFESSIONAL DEVELOPMENT II-137
ASQ CODE OF ETHICS . . . . . . . . . . . . . II-137
SOFTWARE LIABILITY AND SAFETY . . II-144
TRAINING AND DEVELOPMENT . . . . . . II-146
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . II-151
CSQE 2002
III. SOFTWARE QUALITY MANAGEMENT . . . . . . . III-1

GOALS & OBJECTIVES . . . . . . . . . . . . . . . . . . . . . III-2
QUALITY GOALS AND OBJECTIVES . . . . III-2
OUTSOURCED SERVICES . . . . . . . . . . . . . III-7
PLANNING . . . . . . . . . . . . . . . . . . . . . . . . . III-13
TRACKING . . . . . . . . . . . . . . . . . . . . . . . . III-19
SQM SYSTEM DOCUMENTATION . . . . . . III-20
CUSTOMER REQUIREMENTS . . . . . . . . . III-22
METHODOLOGIES . . . . . . . . . . . . . . . . . . . . . . III-28
REVIEW, INSPECTION & TESTING . . . . . III-28
CHANGE MANAGEMENT METHODS . . . III-30
COST OF QUALITY . . . . . . . . . . . . . . . . . . III-36
QUALITY DATA TRACKING . . . . . . . . . . . III-42
REPORTING & CORRECTIVE ACTION . . III-45
QUALITY IMPROVEMENT PROCESSES . III-48
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . III-53
IV. SOFTWARE AUDITS . . . . . . . . . . . . . . . . . . . . . . IV-1

PROGRAM DEVELOPMENT & ADMIN . . . . . . . IV-2
AUDIT RESPONSIBILITIES . . . . . . . . . . . . IV-5
AUDIT PREPARATION & EXECUTION . . . . . . IV-10
AUDIT TYPES . . . . . . . . . . . . . . . . . . . . . . IV-10
AUDIT SCHEDULE . . . . . . . . . . . . . . . . . . IV-19
AUDITING TOOLS & PROCEDURES . . . . IV-20
PREPARATION PHASE . . . . . . . . . . . . . . IV-23
PERFORMANCE PHASE . . . . . . . . . . . . . IV-24
AUDIT REPORTING & FOLLOW-UP . . . . . . . . IV-25
REPORTING PHASE . . . . . . . . . . . . . . . . . IV-25
CORRECTIVE ACTION FOLLOW-UP . . . . IV-26
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . IV-28
CSQE 2002
V. SOFTWARE ENGINEERING PROCESSES . . . . . V-1

ENVIRONMENTAL CONDITIONS . . . . . . . . . . . V-2
LIFE CYCLES . . . . . . . . . . . . . . . . . . . . . . . V-2
SYSTEM ARCHITECTURE . . . . . . . . . . . . V-19
REQUIREMENTS MANAGEMENT . . . . . . . . . . V-24
REQUIREMENTS ENGINEERING . . . . . . . . . . V-32
REQUIREMENTS ELICITATION . . . . . . . . V-38
REQUIREMENTS ANALYSIS . . . . . . . . . . V-51
REQUIREMENTS SPECIFICATIONS . . . . V-60
ANALYSIS, DESIGN & DEVELOPMENT . . . . . V-65
SOFTWARE DESIGN METHODS . . . . . . . V-65
TYPES OF SOFTWARE REUSE . . . . . . . . V-72
CLEANROOM & FORMAL METHODS . . . V-76
SOFTWARE DEVELOPMENT TOOLS . . . V-82
MAINTENANCE MANAGEMENT . . . . . . . . . . . V-85
OPERATIONAL MAINTENANCE . . . . . . . V-94
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . V-102
VI. PROGRAM & PROJECT MANAGEMENT . . . . . VI-1

PLANNING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VI-2
PROJECT PLANNING ELEMENTS . . . . . . VI-3
GOAL-SETTING & DEPLOYMENT . . . . . . VI-12
PROJECT PLANNING TOOLS . . . . . . . . . VI-14
COST AND VALUE DATA . . . . . . . . . . . . . VI-30
TRACKING & CONTROLLING . . . . . . . . . . . . . VI-33
PHASE TRANSITION TECHNIQUES . . . . VI-33
INTERPRETING COQ DATA . . . . . . . . . . . VI-38
TRACKING ELEMENTS & METHODS . . . VI-44
PROJECT REVIEWS . . . . . . . . . . . . . . . . . VI-46
RISK MANAGEMENT . . . . . . . . . . . . . . . . . . . . VI-49
RISK MGMT PLANNING METHODS . . . . VI-49
CSQE 2002
RISK PROBABILITY . . . . . . . . . . . . . . . . . VI-53

PRODUCT RELEASE DECISIONS . . . . . . VI-56
SECURITY & HAZARD ANALYSIS . . . . . . VI-59
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . VI-67
VII. SOFTWARE METRICS . . . . . . . . . . . . . . . . . . . VII-1

METRICS & MEASUREMENT THEORY . . . . . VII-4
DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . VII-6
BASIC MEASUREMENT THEORY . . . . . VII-10
PSYCHOLOGY OF METRICS . . . . . . . . . VII-15
PROCESS & PRODUCT MEASUREMENT . . VII-17
PROCESS & RESOURCE METRIC . . . . VII-17
COMMONLY USED METRICS . . . . . . . . VII-21
SOFTWARE QUALITY ATTRIBUTES . . . VII-25
DEFECT DETECTION MEASURES . . . . VII-36
PERF. & PROCESS EFFECTIVENESS . VII-39
ANALYTICAL TECHNIQUES . . . . . . . . . . . . . VII-43
DATA INTEGRITY . . . . . . . . . . . . . . . . . . VII-43
QUALITY TOOLS . . . . . . . . . . . . . . . . . . VII-48
SAMPLING THEORY & TECHNIQUES . . VII-74
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . VII-78
VIII. SOFTWARE VERIFICATION & VALIDATION VIII-1

THEORY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VIII-2
V&V PROGRAM . . . . . . . . . . . . . . . . . . . . VIII-9
INTERFACES . . . . . . . . . . . . . . . . . . . . . VIII-14
REVIEWS AND INSPECTIONS . . . . . . . . . . . VIII-16
TYPES . . . . . . . . . . . . . . . . . . . . . . . . . . . VIII-16
PROCESSES . . . . . . . . . . . . . . . . . . . . . . VIII-21
DATA COLLECTIONS & REPORTS . . . . VIII-26
TEST PLANNING & DESIGN . . . . . . . . . . . . . VIII-29
TYPES OF TESTS . . . . . . . . . . . . . . . . . . VIII-29
TEST STRATEGIES . . . . . . . . . . . . . . . . VIII-38
TEST DESIGN . . . . . . . . . . . . . . . . . . . . . VIII-44
CSQE 2002
TEST ENVIRONMENTS . . . . . . . . . . . . . VIII-53

TEST EXECUTION & EVALUATION . . . . . . . VIII-58
TEST IMPLEMENTATION . . . . . . . . . . . . VIII-58
TEST DOCUMENTATION . . . . . . . . . . . . VIII-65
CODE COVERAGE METRICS . . . . . . . . . VIII-73
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . VIII-81
IX. SOFTWARE CONFIGURATION MANAGEMENT IX-1

DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . IX-5
CONFIGURATION INFRASTRUCTURE . . . . . . IX-7
CONFIGURATION MANAGEMENT . . . . . . IX-7
LIBRARY/REPOSITORY PROCESS . . . . . IX-12
DEFECT TRACKING/LIBRARY TOOLS . . IX-14
CONFIGURATION IDENTIFICATION . . . . . . . . IX-20
BASELINES . . . . . . . . . . . . . . . . . . . . . . . . IX-27
IDENTIFICATION METHODS . . . . . . . . . . IX-32
SOFTWARE BUILDS . . . . . . . . . . . . . . . . . IX-36
CONFIGURATION CONTROL . . . . . . . . . . . . . IX-39
PROPOSED MODIFICATIONS . . . . . . . . . IX-43
TRACEABILITY . . . . . . . . . . . . . . . . . . . . . IX-50
VERSION CONTROL . . . . . . . . . . . . . . . . . IX-53
CONFIGURATION STATUS ACCOUNTING . . IX-56
CHANGES TO ITEM/BASELINES . . . . . . . IX-62
CONFIGURATION AUDITS . . . . . . . . . . . . . . . IX-68
RELEASE & DISTRIBUTION ISSUES . . . . . . . IX-71
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . IX-77
X. INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-1
ANSWERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-19
CSQE 2002
CSQE Primer Question Contents

QUESTIONS
PRIMER SECTION % CSQE EXAM PRIMER CD
I. Certification Overview
II. General Knowledge,

Conduct and Ethics 10.0% 16 40 100
III. Software Quality

Management 12.5% 20* 50 125
IV. Software Audits 6.25% 10* 25 63
V. Software Engineering
Process 16.25% 26 65 162
VI. Program & Project

Management 15.0% 24 60 150
VII. Software Metrics 15.0% 24 60 150
VIII. Software Verification

& Validation 15.0% 24 60 150
IX. Software Configuration

Management 10 % 16 40 100
X. Index
TOTALS 100% 160 400 1000
QUALITY COUNCIL OF INDIANA I-1 (8)
CSQE 2002
I. CERTIFICATION OVERVIEW
Professionalizing Quality Education
I KNOW OF NO MORE
ENCOURAGING FACT THAN
T H E U N Q U E S T I O N AB L E
ABILITY OF MAN TO ELEVATE
HIS LIFE BY A CONSCIOUS
ENDEAVOR.
HENRY DAVID THOREAU

CSQE 2002
CSQE Exam
Objective
To provide recognized software quality engineer

fundamental training, and to prepare persons
interested in taking the CSQE examination.
Eligibility
Eligibility entails a combination of eight years work

experience and/or higher education. Three years of
this requirement must be in a decision making,
technical, professional or management position.
Duration
The CSQE examination lasts 4 hours and begins at

an advised time (typically 8 or 9 A.M.). There are
currently 160 multiple choice questions on the open
book examination.
Other Details
Contact ASQ at (800) 248-1946 or

http://www.asq.org
CSQE 2002
CSQE Exam (Continued)

Bibliography Sources
The CSQE student should obtain the bibliography

furnished by ASQ. The sources recommended by the
authors include:
Humphrey, W. (1990). Managing the Software Process.

Reading, MA: Addison Wesley.
ISO 9000-3-1997, Quality Management and Quality

Assurance Standards - Part 3: Guidelines for the
Application of ISO 9001 to the Development, Supply
and Maintenance of Software.
ISO 10006:1997 Quality Management - Guidelines to

Quality in Project Management.
ISO 10007:1995 Quality Management - Guidelines for

Configuration Management.
Kan, S. (1995). Metrics and Models in Software Quality

Engineering, 1st ed. Reading, MA: Addison Wesley.
Kaner, C., Falk, J., & Nguyen, H. (1999). Testing

Computer Software, 2nd ed. Van Nostrand
Reinhold.
CSQE 2002
CSQE Exam (Continued)

Bibliography Sources (Continued)
Pressman, R.S. (1992). Software Engineering: A

Practitioners Approach, 3rd ed. New York:
McGraw-Hill, Inc.
Pressman, R. (1993). A Manager's Guide to Software

Engineering. New York: McGraw-Hill.
Schulmeyer, G. G., & McManus, J. I. (1999). Handbook

of Software Quality Assurance, 3rd. ed. Upper
Saddle River, NJ: Prentice Hall.
Additional references are listed at the end of each

Primer Section.
CSQE 2002
ASQ Certified Software Quality Engineer

Body of Knowledge
The detailed Body of Knowledge is given in the CSQE
Primer pages I-6 through I-21.
I. General Knowledge, Conduct, and Ethics

(16 Questions)
II. Software Quality Management (30 Questions)
III. Software Engineering Processes (26 Questions)
IV. Program and Project Management (24 Questions)
V. Software Metrics, Measurement, and Analytical

Methods (24 Questions)
VI. Software Verification and Validation (V&V)

(24 Questions)
VII. Software Configuration Management

(16 Questions)
CSQE 2002
Six Levels of Cognition

Based on Blooms Taxonomy (1956)
In addition to content specifics, the subtext detail also
indicates the intended complexity level of the test
questions for that topic. These levels are based on
Levels of Cognition (from Blooms Taxonomy, 1956)
and are presented below in rank order, from least
complex to most complex.
C Knowledge Level
C Comprehension Level
C Application Level
C Analysis
C Synthesis
C Evaluation
QUALITY COUNCIL OF INDIANA II-1 (14)
CSQE 2002
II. GENERAL KNOWLEDGE, CONDUCT AND ETHICS
IT IS EASY TO BE TOLERANT OF
THE PRINCIPLES OF OTHER
PEOPLE, IF YOU HAVE NONE OF
YOUR OWN.
SIR HERBERT SAMUEL

CSQE 2002

A. QUALITY PHILOSOPHY AND PRINCIPLES
1. BENEFITS OF SOFTWARE QUALITY
Introduction
General Knowledge, Conduct and Ethics is presented in
the following areas:
C Quality Philosophy and Principles

C Standards, Specifications, and Models
C Leadership Tools and Skills
C Ethical Conduct and Professional Development
Quality Philosophy and Principles

Quality Philosophy and Principles is presented in the
following topic areas:
C Benefits of Software Quality

C Prevention vs. Detection
C Organizational and Process Benchmarking
CSQE 2002

Benefits of Software Quality

There are two principal categories of software quality
measurement: functional and nonfunctional.
Software quality: The conformance to explicitly stated

functional and performance requirements, explicitly
documented development standards, and implicit
characteristics, such as maintainability and modularity.
The simplest definition is a generalization given by

Crosby: Quality is conformance to requirements.
It is important to note that both definitions cover

products and the processes that create them. They also
cover the functional and the nonfunctional categories of
quality measures.
Two other definitions of quality are:
C The degree to which a system, component, or

process meets specified requirements.
C The degree to which a system, component, or

process meets customer or user needs or
expectations.
CSQE 2002

Benefits of Software Quality (Continued)

Quality software plays a significant role in almost
everything we do.
The benefits of software quality from a customer*s

perspective are:
C Customer satisfaction
C Improved software reliability
C Reduced errors during software operation
C Matching the customer*s requirements
The benefits of software quality from an organization*s

perspective are:
C The customer*s requirements are met

C Stability of requirements
C Verification that feature requirements have
been implemented
C Processes are applied in a consistent manner
C The process is improved over time
CSQE 2002

Quality Philosophies
Philip B. Crosby Senior manager involvement
4 absolutes of quality management
Quality cost measurements
W. Edwards Plan-do-study-act (wide usage)
Deming Top management involvement
Concentration on system improvement
Constancy of purpose
Armand V. Total quality control/management
Feigenbaum Top management involvement
Kaoru Ishikawa 4M (5M) or cause and effect diagram
Company wide quality control
Next operation as customer
Joseph M. Juran Top management involvement
Quality Trilogy (project improvement)
Quality cost measurement
Pareto analysis
Walter A. Assignable cause versus chance causes
Shewhart Control charts
Plan-Do-Check-Act
Use of statistics for improvement
Genichi Taguchi Loss function concepts
Signal to noise ratio
Experimental design methods
Concept of design robustness
CSQE 2002

2. PREVENTION VERSUS DETECTION
Prevention Versus Detection

The basic philosophy of detection is based on the
premise that when defects are identified, then the defect
will be corrected and eliminated.
The prevention philosophy is based on applying

techniques earlier in the life cycle that prevent the later
identification of defects. This principle can be applied
to software defect prevention.
Total Quality Management (TQM) is a management

process that incorporates several quality concepts:
continuous improvement, management by fact,
measurement of progress, quality teams, management
of resources, and leadership. Examples of Software
TQM activities include the following:
C Plan-Do-Check-Act
C SEI Capability Maturity Model
C Goal/Question/Metric Paradigm
CSQE 2002

3. ORGANIZATIONAL & PROCESS BENCHMARKING
Organizational & Process Benchmarking

Benchmarking is the process of comparing the current
project, methods, or processes with the best practices
and using this information to drive improvement of
overall company performance, with other projects,
within the organization and outside the organization.
Process Benchmarking
Process benchmarking focuses on discrete work

processes and operating systems. This form of
benchmarking seeks to identify the most effective
operating practices from many companies that perform
similar work functions.
Performance Benchmarking
Performance benchmarking usually focuses on

elements of price, technical quality, ancillary product or
service features, speed, reliability, and other
performance characteristics.
CSQE 2002

3. ORGANIZATIONAL & PROCESS BENCHMARKING
Benchmarking (Continued)
Project Benchmarking
Benchmarking of project management is easier than

many business processes, because of the opportunities
for selection outside of the group of direct competitors.
Project benchmarking is used to select new techniques
for planning, scheduling, and controlling the project.
Strategic Benchmarking
Strategic benchmarking examines how companies

compete and is seldom industry-focused. It moves
across industries seeking to identify the winning
strategies that have enabled high-performing
companies to be successful in their marketplaces.
The sequence of activities for benchmarking follows a

pattern such as the following:
C Determine Current Practices

C Identify Best Practices
C Analyze Best Practices
C Model Best Practices
CSQE 2002

B. STANDARDS, SPECIFICATIONS, AND MODELS
1. QUALITY MANAGEMENT SYSTEM MODELS
Standards, Specifications, and Models
Standards, Specifications, and Models is presented in

the following topic areas:
C Quality Management System Models

C Software Standards
C Software Assessment Models
C Other Software Standards
CSQE 2002

Model Situations and Applications

Of significance to the software quality engineering
professional is the realization that software process
models, specifications and standards are multifaceted.
In effect they are used as tools in a variety of
situations in the Software Engineering Industry and the
associated Software Quality Engineering activities.
The Capability Maturity Model (CMM), which is

commonly used as an assessment model, can also be
used as a model to guide in the development of a sound
software engineering process.
It can then be used as a model to continue to augment

the maturity level of this software engineering process
(i.e. the continuous aspect of the model).
The model can be used to assess the maturity level of

the software processes that have been developed and
to identify gaps where further process development is
needed (i.e. assessment and continuous improvement).
The CMM has also been used as a platform in the

industry to develop standards such as TL 9000 Quality
System Requirements.
CSQE 2002

Quality Management System Models

International Organization for Standardization
ISO (the International Organization for Standardization)

is a worldwide federation of national standards bodies.
The American National Standards Institute (ANSI) is the
national member body representing the United States.
A main purpose of ISO is to promote the development of
standardization.
ISO Technical Committee 176 (ISO/TC176) was formed

in 1979 to harmonize the increasing international
activity in quality management and quality assurance.
The United States has input through a Technical

Advisory Group (TAG) to ISO/TC 176.
CSQE 2002

ISO 9000/Q9000-2000 Quality Standards

ISO 9000-2000, ANSI/ASQ Q9000- 2000
Quality management systems - Fundamentals and
vocabulary
ISO 9001-2000, ANSI/ASQ Q9001- 2000

Quality management systems - Requirements
ISO 9004-2000, ANSI/ASQ Q9004-2000

Quality management systems - Guidance for
performance improvement
ISO 9000 - provides an introduction to Quality

Management System (QMS)
ISO 9001 - specifies requirements for a QMS where

capability to provide product that meets customer
requirements needs to be demonstrated
ISO 9004 - Provides guidance on QMS that contributes

to satisfaction of the Customer and other interested
parties
CSQE 2002

ISO 9000/Q9000-2000 (Continued)

Process Approach Model
This standard promotes a process approach when

developing, implementing and improving the
effectiveness of a quality management system. This
approach enhances customer satisfaction by meeting
customer requirements by ensuring that quality
management system requirements are complementary
to product requirements.
Model of Process Based Quality Management System

CSQE 2002

ISO 9000 Quality Management Principles

ISO 9001-2000 and ISO 9004-2000 form a consistent pair
of QMS standards based on eight principles of quality
management:
1. Customer focus
2. Leadership
3. Involvement of people
4. Process approach
5. System approach to management
6. Continual improvement
7. Factual approach to decision making
8. Mutually beneficial supplier relationship

CSQE 2002

ISO 9000-3-1997
ISO 9000-3 and ANSI/ISO/ASQ Q9000-3-1997 are titled:
Quality Management and Quality Assurance Standards -
Part 3: Guidelines for the Application of ANSI/ISO/ASQC
Q9001-1994 to the Development, Supply, Installation and
Maintenance of Computer Software.
C Extensive reference is made to ANSI/IEEE/EIA

12207.0, Standard for Information Technology-
Software Life Cycle Processes.
It is anticipated that ISO 9000-3 will undergo another

revision to make it compatible with ISO 9001:2000.
An outline and synopsis of ISO 9000-3 and

ANSI/ISO/ASQ Q9000-3-1997 is given in the Primer.
CSQE 2002

2. SOFTWARE STANDARDS
Software Standards
Bellcore TR-179/Telcordia (TR-179)

The Bellcore TR-179 document presents a
comprehensive set of software quality generic
requirements that addresses the need for greater
supplier accountability for demonstrating compliance to
quality system requirements, and the need for improved
quality and reliability of the software systems used in
telecommunications networks. It is based on the
framework established by the ISO 9001/Q9001
standards.
TR-179 is intended to apply to stand-alone software and

software embedded in telecommunications systems and
to non-deliverable design, test, support, and operational
software used to assist in the development,
implementation, operation, and change of deliverable
software.
CSQE 2002

Bellcore Model
The Bellcore model contains 25 generic requirements
sets, which are organized to align with the ISO 9000-3
descriptive guideline structure. Two additional sets
have been added, for ongoing quality improvement in
the quality system and basic software processes. They
provide purchasers with operational assistance in
analyzing and resolving field problems.
CSQE 2002

ISO/IEC Joint Technical Committee 1

JTC1 is the joint technical committee established
between ISO and the International Electrotechnical
Commission (IEC), which has a broad charter to develop
standards for information technology.
ISO/IEC JTC 1/SC7

ISO/IEC JTC1/SC7 is an international technical
standards committee working on standards for software
engineering. It is a subcommittee of JTC 1.
The SC7 charter is to reduce the cost of international

software engineering, to improve communications
between suppliers and buyers, and to improve the
quality of delivered software and systems: all through
the development of software engineering standards
which are relevant, coherent, complete and effective in
use.
The work of SC7 is accomplished through projects

assigned to working groups (WG).
CSQE 2002

IEEE Software Engineering Standards

The Computer Society of the IEEE formed a committee
to codify these norms of professional software
engineering practices into standards. Currently, there
are 27 approved software engineering standards. Some
of these standards that are of interest to Software
Quality Engineers include the following:
C ANSI/IEEE Standard 730: Standard for Software

QA Plans
C ANSI/IEEE Standard 828: Standard for Software

CM Plans
C ANSI/IEEE Standard 830: Guide to Software

Requirement Specifications
C ANSI/IEEE Standard 1012: Standard for

Software V&V Plans

Software Reviews and Audits

Developing Software Life Cycle Processes
CSQE 2002

3. SOFTWARE ASSESSMENT MODELS
Software Assessment Models
Capability Maturity Model

The Capability Maturity Model (CMM) developed by the
Software Engineering Institute (SEI), represents a well-
structured compilation of best practices that can be
gradually implemented in a company wishing to develop
or maintain quality software within projects that are
consistently on time and within budget.
The CMM provides a reliable basis for consistent

evaluation of software organizations. It also serves as
a guide for organizations to improve their management
and development of software products.
The CMM describes five maturity levels which provide

a successive, evolutionary foundation for continuous
improvement and a scale for measuring the maturity of
organizations.
Key process areas are identified within the CMM as

critical indicators of a companys progress away from
ad hoc program management and toward a more
managed process.
CSQE 2002

Capability Maturity Model (Continued)
Process
Maturity Levels indicate
Capability
contain
Key Process Areas achieve Goals
organized by
Common Implementation or
address
Features Institutionalization
contain
Key Infrastructure
describe
Practices or Activities
CMM Structure
CSQE 2002

Capability Maturity Model (Continued)

Level 1: Initial - No defined goals or activities. The
organization typically does not provide a stable
environment for developing and maintaining software.
Level 2: Repeatable - Basic project management

processes are established to track cost, schedule, and
functionality.
Level 3: Defined - The software process for both

management and engineering activities is documented,
standardized, and integrated into an organization-wide
understanding of the software process.
Level 4: Managed - The software processes and

products are quantitatively understood and controlled
using detailed measures.
Level 5: Optimizing - The entire organization is focused

on continuous process improvement.
CSQE 2002

CMM Evaluation Methods

Software Capability Evaluation (SCE)
This method is used to determine an organizations
level of maturity against a predefined standard as
defined by the CMM.
Interim Profile
This approach is also based on CMM. It is a method to
measure progress between two extensive evaluations.
Software Process Assessment (SPA)

This method was developed to evaluate internal
software capability of an organization wishing to initiate
or reinforce an improvement effort internally.
CBA IPI
The CBA IPI method (CMM-Based Appraisal for Internal
Process Improvement) has two objectives: establishing
an accurate picture of the strengths and weaknesses of
an organizations software process relative to the CMM,
and creating an internal commitment of personnel to the
ongoing improvement process.
CSQE 2002

Capability Maturity Model Integration

The Capability Maturity Model Integration (CMMIK)
helps organizations appraise its organizational maturity
or process area capability, establish priorities for
improvement, and implement these improvements. The
CMMI reflects content from bodies of knowledge for
both systems engineering and software engineering.
Each organization needs to decide which CMMI

representation, either continuous or staged, best fits
their process improvement needs.
CSQE 2002

CMMI - Continuous Representation

CMMI models are designed to describe discrete levels
of process improvement. In the continuous
representation, capability levels provide a
recommended order for approaching process
improvement within each process area. At the same
time, the continuous representation allows some
flexibility for the order in which the process areas are
addressed.
Specific goals are organized into specific practices and

generic goals are organized into generic practices.
Each specific and generic practice corresponds to a
capability level. Specific goals and specific practices
apply to individual process areas.
Generic goals and generic practices apply to multiple

process areas. The generic goals and generic practices
define a sequence of capability levels that represent
improvements in the implementation and effectiveness
of all the processes an organization chooses to
improve.
CSQE 2002

CMMI - Continuous Representation (Cont.)
CMMI Model Components Continuous Representation

CSQE 2002


Capability Levels
All CMMI models, with a continuous representation,

reflect capability levels in their design and content. A
capability level consists of related specific and generic
practices for a process area that can improve the
organizations processes associated with that process
area.
Capability Level 0: Incomplete - An incomplete process

is a process that is either not performed or partially
performed.
Capability Level 1: Performed - Satisfies the specific

goals of the process area. It supports and enables the
work necessary to produce identified output work
products using identified input work products.
Capability Level 2: Managed - A process that is planned

and executed in accordance with policy; employs skilled
people having adequate resources to produce
controlled outputs; involves relevant stakeholders; is
monitored, controlled, and reviewed; and is evaluated
for adherence to its process description.
CSQE 2002


Capability Level 3: Defined - A process that is tailored
from the organization's set of standard processes
according to the organizations tailoring guidelines, and
contributes work products, measures, and other
process-improvement information to the organizational
process assets.
Capability Level 4: Quantitatively Managed - A process

that is controlled using statistical and other quantitative
techniques. Quantitative objectives for quality and
process performance are established and used as
criteria in managing the process.
Capability Level 5: Optimizing - A process that is

changed and adapted to meet relevant current and
projected business objectives. An optimizing process
focuses on continually improving the process
performance through both incremental and innovative
technological improvements.
CSQE 2002

CMMI - Staged Representation

CMMI models are designed to describe discrete levels
of process improvement. In the staged representation,
maturity levels provide a recommended order for
approaching process improvement in stages.
CMMI Model Components for Staged Representation

CSQE 2002

CMMI - Staged Representation (Cont.)

Maturity Levels
The maturity level of an organization provides a way to

predict the future performance of an organization within
a given discipline or set of disciplines.
Maturity Level 1: Initial - Processes are usually ad hoc

and chaotic. The organization typically does not provide
a stable environment.
Maturity Level 2: Managed - Requirements are managed

and processes are planned, performed, measured, and
controlled. Projects are performed and managed
according to their documented plans.
Maturity Level 3: Defined - Processes are well

characterized, understood and are described in
standards, procedures, tools, and methods. The
organizations set of standard processes is established
and improved over time.
CSQE 2002


Maturity Level 4: Quantitatively Managed - Quantitative
objectives for quality and process performance are
established and used as criteria in managing processes.
Detailed measures of process performance are

collected and statistically analyzed. Quality and process
performance measures are incorporated into the
organizations measurement repository to support
future fact-based decision making.
Maturity Level 5: Optimizing - The effects of deployed

process improvements are measured and evaluated
against the quantitative process improvement
objectives. Improvement of processes is inherently part
of everybodys role, resulting in a cycle of continual
improvement.
CSQE 2002


CMMI Process Areas
The CMMI process areas apply to both the continuous

representation and the staged representation.
C Process Management
C Project Management
C Engineering
C Support
CSQE 2002

CMMI Appraisal Methods

SCAMPI
The Standard CMMI Appraisal Method for Process

Improvement (SCAMPIK) was developed as an
appraisal tool for benchmarking and a set of guidelines
for future process improvement appraisals requiring
less rigor and repeatability.
SCAMPI was written to support the conduct of

appraisals that conform to the emerging ISO/IEC 15504
technical report.
Appraisal Requirements for CMMI
The Appraisal Requirements for CMMI (ARC) document

contains a set of criteria for developing, defining, and
using appraisal methods based on CMMI products for
both the staged and continuous representations.
The ARC provides requirements for multiple types of

appraisal methods with guidelines for determining the
suitability of a particular appraisal method.
CSQE 2002

ISO SPICE (ISO 15504)

A suite of standards for software process assessment
has been developed by ISO under the SPICE project
(Software Process Improvement for Capability
determination).
The intent of the SPICE standards is to harmonize the

numerous efforts around the world to manage the
software process. These efforts include the SEIs work
with the Capability Maturity Model, Bell Canadas
Trillium, ESPRITs Bootstrap, and many others.
The SPICE standards focus on software process issues

but are also concerned with people, technology,
management practices, customer support and quality,
as well as software development and maintenance
practices.
CSQE 2002

ISO SPICE (Continued)

The Standard for Software Process Assessment
consists of nine parts:
Part 1: Concepts and Introductory Guide
Part 2: A Model for Process Management
Part 3: Rating Processes
Part 4: Guide to Conducting Assessment
Part 5: Construction, Selection and Use of Assessor

Instruments and Tools
Part 6: Qualification and Training of Assessors
Part 7: Guide for the Use in Process Improvement
Part 8: Guide for Use in Determining Supplier

Process Capability
Part 9: Vocabulary
CSQE 2002

Trillium Model
The goal of the Trillium model is to provide a means to
initiate and guide a continuous improvement program.
The Trillium model provides key industry practices
which can be used to improve an existing process or
life cycle.
The practices in the Trillium model are derived from a

benchmarking exercise which focused on all practices
that would contribute to an organizations product
development and support capability.
The Trillium model can be used for:
C Capability Evaluation
C Capability Joint-Assessment
C Capability Assessment and Improvement
C Capability Self-Assessment
C Continuous Improvement (CI) Program

CSQE 2002

Trillium Model
Model Foundation
The Trillium model is based on the Software

Engineering Institutes Capability Maturity Model (CMM)
version 1.1. The architecture of the model differs from
the CMM version 1.1. The most significant differences
are:
C Model architecture based on road maps, rather

than key process areas
C A product rather than software perspective
C Wide coverage of capability impacting issues
C Customer focus, technological maturity, and
telecommunications orientation
In addition to the CMM, the model incorporates the

intent of ISO, Bellcore, Malcolm Baldrige, IEEE and IEC
standards.
CSQE 2002

Trillium Scale
1. Unstructured - The development process is ad
hoc. Project frequently cannot meet quality or
schedule targets. (Risk - High)
2. Repeatable and Project Oriented - Individual

project success is achieved through strong
project management planning and control.
(Risk - Medium)
3. Defined and Process Oriented - Processes are

defined and utilized at the organizational level.
Processes are controlled and improved.
(Risk - Low)
4. Managed and Integrated - Mechanisms for

process improvement. Process change
management and defect prevention programs
are integrated into processes. (Risk - Lower)
5. Fully Integrated - Formal methodologies are

extensively used. Organizational repositories
for development history and process are
utilized and effective. (Risk - Lowest)
CSQE 2002

Trillium Model Architecture
TRILLIUM Model
indicates contains
Process
Capability Capability Areas
impact contain
Goals:
C Organization
C Commitments
C Culture
Road Maps
influence
contain
Process
Functions
Techniques
Practices
cover
Activities:
C Implementation
C Deployment
C Institutionalization
The Trillium Model (Release 3.0)

CSQE 2002

Trillium Capability Areas

There are 8 Capability Areas within the Trillium model.
Each Capability Area contains practices at multiple
Trillium levels. For example, the Management area
spans levels 2 to 4 while the Quality Systems area spans
levels 2 to 5.
Trillium Capability Profile

A profile of the Capability Areas is an important
measure of a software development organization since
it illustrates the relative areas of strength and
weakness. This profile shows that organizations
typically achieve some higher level practices without
having completed all the practices at the lower levels.
Trillium Capability Profile

CSQE 2002

Trillium Capability Level

To achieve a Trillium level, an organization must satisfy
a minimum of 90% of the criteria in each of the 8
Capability Areas at that level. Levels 3, 4, and 5 require
the achievement of all lower Trillium levels (i.e. levels
cannot be skipped).
Road maps
Each Capability Area includes one or more road maps.

A road map is a set of related practices that focus on an
organizational area or need, or a specific element within
the product development process. Each road map
represents a significant capability for a software
development organization.
Practices
Within a given road map, the level of practices is based

on their respective degree of maturity. An organization
matures through the road map levels. Lower level
practices must be implemented and sustained for
higher level practices to achieve maximum
effectiveness.
CSQE 2002

Bootstrap Method
Bootstrap is a project completed as part of the
European Strategic Program for Research in Information
Technology, also known as ESPRIT. Its goal was to
develop a method for software process assessment,
quantitative measurement, and improvement.
Bootstrap enhanced and refined the Software

Engineering Institutes process assessment method and
adapted it to the needs of the European software
industry, including nondefensive sectors like banking,
insurance, and administration.
Bootstrap Assessment
The Bootstrap method assesses both the organization

and its projects, to help determine whether the
organization is providing the necessary resources to do
the projects, and that the projects are using the
resources efficiently.
CSQE 2002

Bootstrap Method (Continued)

Bootstrap Assessment (Continued)
Bootstrap Hierarchy of Process Quality Attributes

CSQE 2002

Bootstrap Method (Continued)

Bootstrap Assessment (Continued)
The metric used in Bootstrap is the Bootstrap maturity

level algorithm - an enhanced and refined version of the
SEI algorithm. Like the SEI method, the Bootstrap
algorithm differentiates among five levels of software
processes. However, unlike the SEI method, the
Bootstrap method calculates a maturity level for each
attribute of the process quality model.
Bootstrap questionnaires are used, one for

organizations and one for projects, to identify the
degree to which an organizations or projects quality
attributes satisfy the criterion of a particular maturity
level, from 2 to 5.
Improvements
Once the attribute rating is completed and a quality

profile is obtained, the organization creates an action
plan to examine and interpret the strengths and
weaknesses of existing processes.
CSQE 2002

4. OTHER SOFTWARE STANDARDS
Other Software Standards
TL 9000 Quality System Standards

QuEST Forum
The Quality Excellence for Suppliers of

Telecommunications (QuEST) Forum was founded to
foster continued improvement to the quality and
reliability of Telecommunication products and services.
The Forum establishes and maintains a common set of
quality system requirements and metrics built on
currently used industry standards, including the ISO
9000 standard.
The intent of the TL 9000 standards is to promote

consistency, efficiency and customer satisfaction.
CSQE 2002

TL 9000 (Continued)
TL 9000 is a common set of quality system requirements
and metrics design specifically for the Telcom industry
encompassing ISO 9001 and best practices.
Characteristics of the TL 9000 relationship to other
requirements are:
C TL 9000 includes ISO 9000 as a foundation by

copying in verbatim all audit able requirements
from ISO 9001-1994 (Release 2.5) and ISO 9001-
2000 (Release 3.0).
C Conformance to TL 9000 constitutes

conformance to corresponding ISO 9001
requirements
CSQE 2002

TL 9000 (Continued)
TL 9000 Structure
TL 9000 Model
QuEST Forum Focus Area
International Standard ISO 9001
Common TL 9000 Requirements T
L
Hardware Specific Software Specific Services Specific
Requirements Requirements Requirements
9
Common TL 9000 Metrics 0
0
Hardware Software Services
Specific Specific Specific
0
Metrics Metrics Metrics
CSQE 2002

TL 9000 (Continued)
TL 9000 Quality System Requirements
TL 9000 Quality System Requirements handbook

consists of 4 major sections. It establishes a common
set of quality system requirements for suppliers of
Telecommunications products: hardware, (h/w),
software (s/w), or services.
TL 9000 registration requires fulfillment of the TL 9000

Quality System Requirements and the reporting of
Quality System metric data. There are 37 requirements
common to ISO 9001: 1994, with addition requirements
(16 s/w only, 15 h/w only, 5 services only, 5 h/w and
services, and 5 h/w and s/w).
TL 9000 Metrics are categorized into Common metrics

(C) which apply to software, hardware, and service,
Hardware metrics (H), Software metrics (S) and Services
metrics (V).
CSQE 2002

TL 9000 (Continued)
TL 9000 Release 3.0 Structure
TL 9000 Release 3.0 consists of two handbooks:
C TL 9000 Quality Management System

Requirements Book 1
C TL 9000 Quality Management System Metric

Book 2
This release of TL- 9000 has been reorganized to meet

the philosophy and format of ISO 9001- 2000 and is
therefore Based on:
C Eight Management principles of ISO 9001 - 2000

C Process approach Model
C Uses concepts of Plan-Do-Check-Act
C Copying in verbatim all audit able requirements
from ISO 9001- 2000
C Format of requirements section identical to ISO
9001 - 2000 five major sections
CSQE 2002

C. LEADERSHIP TOOLS AND SKILLS
1. ORGANIZATIONAL LEADERSHIP
Leadership Tools and Skills
Leadership Tools and Skills is presented in the

C Organizational Leadership
C Team Management
C Team Tools
C Facilitation Skills
C Communication Skills
CSQE 2002

Organizational Leadership
Effective improvement programs do not happen
accidentally. Careful planning and implementation are
required to ensure that the proper resources are
available and applied to the right problems.
Key resources may include people trained in problem

solving tools, appropriate equipment, analysis tools,
and capital resources.
Assigning human resources may be the most difficult

element, since highly skilled problem solvers are a
valuable resource.
CSQE 2002

Analyzing Current Situations

Pande suggests that embarking on an improvement
initiative begins with a management decision to
embrace a change that says Theres a better way to run
our organization. The readiness assessment includes
a review of the following areas:
C Assess the outlook and future path of the

business
C Evaluate the current organizational

performance
C Review the capacity for systems change and

improvement
The above assessment will go a long way towards

deciding if current efforts are sufficient or whether it
appropriate to undertake new initiatives.
CSQE 2002

SWOT Analysis
SWOT analysis requires that a comprehensive appraisal
of internal and external situations be undertaken before
suitable strategic options can be determined. Good
strategies are built on the strengths of a company and
on exploiting opportunities. Each company has a
different set of opportunities and threats with differing
strengths and weaknesses. Thus, strategies will be
different for each firm.
Strengths and Weaknesses
A strength is something that the company is good at

doing. The strength can be a skill, expertise, a patent,
key resource, technology, market position, or anything
that provides an advantage.
A weakness is something that the firm lacks or is a

condition that puts it at a disadvantage. Either the
strengths can be overly magnified or the weaknesses
minimized, such that the process is negated.
CSQE 2002

SWOT Analysis (Continued)

Opportunities and Threats
The firm must be able to assess the external

environment in preparation for challenges. SWOT
analysis is designed to make management go beyond
their plant boundaries in the strategic planning effort.
The external environment can include assessment of

economic, sociopolitical, social, technological and
competitive elements. A firms external world will
provide opportunities and threats. The strategy must
match up with:
C Opportunities suited to the firms capabilities

C Defenses against external threats
C Changes to the external environment
CSQE 2002

Implementing and Managing Change

Organizations generally undergo change in four major
areas: strategy, technology, structure and personnel.
C Strategic change occurs when the company

shifts its direction and resources toward new
businesses or markets.
C Technological change occurs when the

company decides that automation or
modernization of key processes are essential
for overall competitiveness.
C Structural changes occur when the company

undergoes a management delayering process
or goes from a functional structure to a product
structure.
C Changing the attitudes and behaviors of

company personnel are often undertaken
through organizational development
techniques.
CSQE 2002

Implementing Change (Cont.)

There is a classical model for the change process. It
has three phases to it: unfreezing, movement, and
refreezing.
C Unfreezing: To the change agent, the first

phase, means unfreezing the existing behavior
patterns and practices of the work group.
C Movement: The next step would be to move the

people or practices to a new arrangement.
C Refreezing: At the proper moment in time (with

the skills, technology, or practices in place), the
process, including the people, are refrozen.
Kotter lists a strategy for dealing with resistance to

change:
C Educate and communicate the change

C Enlist employee participation in the project
C Have support efforts such as training or
counseling
C Have negotiated arrangements for change
C Use stronger measures if required
CSQE 2002

Knowledge Management
Knowledge management is a process that a firm can
use to build their capabilities in maintaining and
improving performance. The aim is to continuously
improve the firm through the sharing of knowledge. The
basic concept is to have the right knowledge, at the
right time, at the right place.
Duffy presents a knowledge management structure

consisting of three layers:
C Process layer: the logic link of data to people

usage
C Data layer: a link across different media

(databases, text data, video, audio)
C User interface: access for people to the

information assets
CSQE 2002

Knowledge Management (Continued)

The above three layers will be intertwined in the
architecture of the knowledge management system
which may include:
C User interface (UI)
C Metamodel & knowledge map
C Repository
C Access tools and knowledge management

enablers
CSQE 2002

Motivation Techniques
Certainly the most challenging management
responsibility is how to both sustain and increase
internal motivation in the work group.
Effective managers must have confidence in both their

subordinates and themselves.
Workers tend to respond to and respect the manager

who knows their capabilities, who is fair and consistent,
and who respects them as individuals.
Frederick Taylor
Frederick Taylor produced a systematic organization of
the shop work broken down by task and instruction.
This significantly raised productivity. Piecework
incentives rewarded the productive and punished the
slackers.
Maximum efficiency became the driving force of the

time. Time and motion studies would reveal where to
cut down on wasted steps in the process. The workers
were encouraged to do more by providing incentives.
CSQE 2002

The Hawthorne Studies

The Hawthorne Studies at Western Electric Company in
1924 revealed two important points:
1. Group behavior has a powerful influence upon

individual members. The work group is a
significant factor, either for or against
productivity, being largely influenced by the
managements ability to effectively lead the
work force.
2. The work group is a social group which fulfills

certain human needs. Up to this time, these
needs were considered to be fulfilled in the
home, church, and organizations away from the
working environment.
One of the primary objectives of the study was to

determine the effect of illumination on productivity. The
Hawthorne effect suggested that people who are
singled out for special attention tend to perform as
anticipated.
CSQE 2002

Abraham Maslow
Abraham Maslows theory of human needs identified
five levels of human needs and they are listed below
from the highest to the lowest:
C Self-actualization needs
C Esteem needs
C Social needs
C Safety needs
C Physiological needs
Maslows theory affirmed the view that individuals are

motivated to lower-order needs until these are relatively
satisfied, and then higher-order needs must be met to
sustain satisfaction.
CSQE 2002

Douglas McGregor
Douglas McGregor introduced new theories, which he
called Theory X and Theory Y. McGregor contended that
traditional management practices were rooted in certain
basic negative assumptions about people (Theory X):
C Are fundamentally lazy

C Avoid responsibility
C Lack integrity
C Are indifferent to organizational needs
C Prefer to be directed by others
C Avoid making decisions
C Are not interested in achievement
By contrast, Theory Y contains the following points:
C The expenditure of physical effort in work is as

natural as play or rest.
C Commitment to objectives is a function of the
associated rewards.
C The average human can learn to accept and
seek responsibility.
C Imagination, ingenuity, and creativity are
widely, not narrowly, distributed.
CSQE 2002

Frederick W. Herzberg
Frederick W. Herzberg proposed that motivation can be
divided into two factors, which have been referred to by
a variety of names:
C Dissatisfiers and Satisfiers, or

C Maintenance factors and Motivators, or
C Hygiene factors and Motivators, or
C Extrinsic factors and Intrinsic factors
Hygiene factors do not provide strong motivation, but

cause dissatisfaction if they are not present. Motivators
or intrinsic factors do provide strong motivation and
satisfaction when they are present.
Hygiene Factors Motivation Factors

Supervision Achievement
Working conditions Recognition
Interpersonal relationships The work itself
Pay and security Responsibility
Company policies Advancement
CSQE 2002

Organizational Empowerment
Organizations find that with empowerment, solutions
are better, decisions have better acceptance, and
performance is increased.
Organizational empowerment generally consists of

steps or stages.
Organizational Empowerment Steps

CSQE 2002

Motivating the Work Force

Kinni suggests that managers, at all levels cannot
directly cause an employee to become motivated. The
best that they can do is follow the following concepts to
create an environment for individuals to motivate
themselves:
1. Know thyself
2. Know your employees
3. Establish a positive attitude
4. Share the goals
5. Monitor progress
6. Develop interesting work
7. Communicate effectively
8. Celebrate success
CSQE 2002

2. TEAM MANAGEMENT
Team Management
A participative style of management is the best
approach to insure employee involvement in the
improvement process.
Team Roles and Responsibilities

Key team roles include the following:
C Champion
C Sponsor
C Facilitator
C Coach
C Leader
C Recorder
C Timekeeper
C Team member
CSQE 2002

2. TEAM MANAGEMENT
Team Dynamics and Team Development

Teams go through four development stages.
Forming
Expectations are unclear. When a team forms, its

members typically start out by exploring the boundaries
of acceptable group behavior.
Storming
Storming consists of conflict and resistance to the

groups task and structure. Conflict often occurs. This
is the most difficult stage for any team to work through.
Norming
A sense of group cohesion develops. Team members

develop norms for resolving conflicts, making
decisions, and completing assignments.
Performing
The group has developed its relationships and purpose.

The team begins to work effectively and cohesively.
CSQE 2002

2. TEAM MANAGEMENT
Team Dynamics (Continued)
Schematic of Team Development Phases

CSQE 2002

2. TEAM MANAGEMENT
Team Life Cycle Characteristics
Team Life Cycle Characteristics
C Build Phase (Forming/Storming)
C Develop Phase (Norming)
C Optimize Phase (Performing)

CSQE 2002

2. TEAM MANAGEMENT
Team Size
A team can consist of members from only one area or
can be made up of a group of representatives from
different parts of the organization. Conventional
wisdom is that teams over 20 people, some think over
15, become too unwieldy and lose the active
participation of all team members. Teams of 4 people or
less may not generate enough ideas.
Team Diversity
Usually team members have diverse skills and
experience. What they share in common is their
involvement in the problem to be addressed. On a team,
each member often brings different experiences, skills,
know-how, and perspectives to the issues.
Diversity in teams strengthens the creative process. A

single person trying to remove a problem or deficiency,
no matter how skilled, has rarely mastered the
intricacies of an entire work process.
Productive teams are sensitive to each others

viewpoints.
CSQE 2002

2. TEAM MANAGEMENT
Dominant or Disruptive Team Members

A team member may be overly dominant or disruptive.
Personality conflicts may exist between team members.
A team member may have too many responsibilities and
cant keep his/her commitments to the team.
If these situations impair the functioning of the team,

the team and the manager should have a series of frank
discussions with the individual centered on whats
expected, whats at stake, or what is happening that
shouldnt be happening.
If the situation doesnt improve, the team member must

be removed.
CSQE 2002

2. TEAM MANAGEMENT
Common Team Problems

C Floundering
C Dominant Participants
C Overbearing Participants
C Negative Nellies
C Opinions as Facts
C Shy Members
C Jump to Solutions
C Attributions
C Put-downs (Discounts & Plops)
C Wanderlust (Tangents & Digressions)
C Feuding
C Risky-Shift
CSQE 2002

3. TEAM TOOLS
Nominal Group Technique

The nominal group technique (NGT) brings people
together to solve problems but limits initial interaction
and encourages independent thinking. The term
nominal is used to describe the limiting of
communications. To conduct a NGT problem solving
meeting:
C A facilitator or moderator leads the discussion
C A group individuals generate ideas
C A problem is presented
C Before any discussion, all members create

ideas individually
C The facilitator requests an idea from each

member until ideas are exhausted, no
discussion is allowed at this point
C Clarification and evaluation of ideas is

permitted
C Voting for the best solution idea

CSQE 2002

3. TEAM TOOLS
Multivoting
Multivoting is a way to select the most popular or
potentially most important items from a previously
generated list. Multivoting consists of the following
steps:
1. Generate and number a list of items
2. Combine similar items if the group agrees
3. If necessary, renumber the list
4. Allow members to choose items that they feel

are most important. Allow the number of
choices equal to one-third of the total items.
5. Members may make their initial choices

silently.
6. To reduce the list, eliminate those items with

the fewest votes.
The items receiving the largest number of votes are

usually worked on or implemented first. The original list
should be saved for future reference and/or action.
CSQE 2002

3. TEAM TOOLS
Brainstorming
Brainstorming is an uninhibited technique for
generating creative ideas when the best solution is not
obvious. Brainstorming does not solve problems or
create a corrective action plan. It is a participative
method to help work teams achieve their goals and
objectives.
C Generate a large number of ideas, dont inhibit

anyone
C Free-wheeling is encouraged
C Dont criticize
C Encourage everyone to participate
C Record all the ideas
C Let ideas incubate
C Select an appropriate meeting place
C The ideal group size is 4-10 people

CSQE 2002

3. TEAM TOOLS
Joint Application Development (JAD)

A team oriented approach to requirements gathering
which is applied during the early stages of analysis and
specification is called Facilitated Application
Specification Technique (FAST).
One of the most popular approaches to FAST is Joint

Application Development (JAD). JAD applies the
following guidelines:
C A meeting is conducted at a neutral site and

attended by both developers and customers
C Rules for preparation and participation are
established
C A facilitator is appointed to control the meeting
C A definition mechanism is used (flip charts)
C The goal is to identify problems, propose
elements of the solution, negotiate different
approaches and specify a preliminary set of
solution requirements in a supportive
environment.
The purpose of JAD is to define the project, design a

solution, and monitor the project until it reaches
completion.
CSQE 2002

3. TEAM TOOLS
JAD (Continued)
The steps for getting a JAD started are:
C Define the project
C Form the JAD group
C The kick off meeting defines goals, roles,

expectations, responsibilities
C JAD meetings - planning, analysis, design

phase
C JAD meetings - development, execute, finish

phase
C Determine if JAD was successful

CSQE 2002

3. TEAM TOOLS
Rapid Application Development (RAD)

Rapid Application Development (RAD) is based on
software development techniques: methodology,
requirements analyses, personnel, construction,
implementation and support. What sets RAD apart is a
structured approach, well trained teams, use of
evolutionary prototypes, and rigid limits on development
time frames. The goals of RAD are: faster, better,
cheaper.
An approach to Rapid Application Development (RAD)

would entail these steps:
1. Scenario-based design and analysis

2. Architecture design and analysis
3. Component specification with maximum reuse
4. Rapid development of remaining modules
5. Frequent testing with end users and systems
personnel
6. Field support tools to allow for evolution
Not explicit in the above approach is mitigation of risks

associated with people and processes. The Capability
Maturity Model (CMM) is not inconsistent with RAD.
CSQE 2002

3. TEAM TOOLS
Rapid Application Development (Cont.)

The RAD path may be adapted to different CASE tools
and development environments. The four stages of
RAD are:
C Requirements Planning
C User Design
C Construction
C Implementation
CSQE 2002

4. FACILITATION SKILLS
Team Facilitation
The team leader and/or facilitator must understand
group dynamics. Facilitators are useful in assisting a
group in the following ways:
C Identifying member training needs

C Avoiding team impasses
C Providing feedback on group effectiveness
C Summarizing points made by the group
C Balancing group member activity
C Clarifying points of view on issues
C Keeping the team on track with the process
C Focusing on progress
C Assessing the change process
C Coaching the leader and participants
The facilitator must avoid:
C Being judgmental of team members

C Taking sides
C Dominating the group discussions
C Solving a problem or giving an answer
C Making suggestions on the task instead of on
the process
CSQE 2002

Conflict Resolution
Conflict can result from mutually exclusive objectives or
views, manifested by emotional responses such as
anger, fear, frustration and elation. Some conflicts are
inevitable in human relationships. When ones actions
may be controlled by the actions of another, there is
opportunity for conflict.
Conflict can lead to productive growth if it is properly

managed and resolved.
A useful strategy for managing conflict is collaboration.

Collaboration takes time and effort, but it addresses the
underlying issues of the situation or conflict.
Collaboration is often more effective than avoidance or
other strategies of conflict resolution, such as
accommodation, domination, or negotiation.
CSQE 2002

Conflict Resolution (Continued)

Each individual uses a number of ways to deal with
conflicts depending upon the circumstances and the
relationships between the people involved. The ways of
dealing with conflicts can be depicted in a two
dimensional model for conflict handling behavior,
adapted from Thomas-Kilmann Conflict Mode
Instrument:
COMPETING COLLABORATING
UNASSERTIVEASSERTIVE
ASSERTIVENESS
COMPROMISING
AVOIDING ACCOMMODATING
UNCOOPERATIVE COOPERATIVE
Conflict Resolution Matrix

CSQE 2002

Conflict Resolution (Continued)

Avoiding is unassertive and uncooperative - the
individual withdraws from the situation.
(You lose, I lose).
Accommodating is unassertive but cooperative

- the individual yields to the wishes of others.
(You win, I lose).
Competing is assertive and uncooperative - the

individual tries to win, even at the expense of
others. (You lose, I win).
Collaborating is assertive but cooperative - the

individual wants things done their way, but is
willing to explore solutions which satisfy the
other persons needs as well. (You win, I win).
Compromising is intermediate in both

assertiveness and cooperativeness - the
individual is willing to partially give in to reach
a middle position, splitting the differences, and
partially satisfying both parties.
(Neither win or lose).
CSQE 2002

Force Field Analysis

A tool often used for problem identification and conflict
resolution is force field analysis. A description of the
process used to perform a force field analysis:
1. A desire to understand the forces acting on a

problem to be resolved
2. Determine the forces favoring the desired goal

(driving forces)
3. Determine the opposing forces to the desired

goal (restraining forces)
4. Determine how to add to the driving forces to

overwhelm the restraining forces, or
5. Remove or weaken the restraining forces, or
6. Do both (strengthen driving forces and weaken

restraining forces)
CSQE 2002

Negotiation Techniques
This treatment of negotiating concentrates on
developing agreements among individuals or groups.
Negotiating is the act of exchanging ideas or changing

relationships to meet a need. As common and as
important as negotiating is in everyday life, most people
learn to negotiate through trial and error.
Negotiating should not be a process of using

overwhelming and irresistible force on the other party.
Some degree of cooperation must be employed in the
negotiating process.
Win-Win Negotiations
C Establish Win-Win Plans
C Develop Win-Win Relationships
C Form Win-Win Agreements
C Perform Win-Win Maintenance

CSQE 2002

Team Performance Factors

Listed below are some illustrative relationship, process,
goal, environmental and role factors that can be used
(or expanded upon) to evaluate team performance.
C Relationship Factors
C Process Factors
C Goal Factors
C Environmental Factors
C Role Factors
CSQE 2002

Team Meeting Structure

1. Develop an agenda
2. Distribute the agenda in advance
3. Start on time
4. Appoint a recorder to record minutes
5. Use visual aids liberally
6. Reinforce
7. Summarize and repeat key points throughout
8. Put unfinished items on next agenda
9. Review assignments and completion dates
10. Finish on time
11. Distribute minutes promptly
12. Critique meeting effectiveness periodically

CSQE 2002

5. COMMUNICATION SKILLS
Communication Skills
For any organization to survive, information must
continually flow in a variety of forms across the
company. Software professionals must be able to
understand this process flow and how to use it.
Presentation Formats
The ability to make public presentations is the number
one predictor of professional success. A good
presentation requires careful planning and preparation.
Written Communication
Good business writing is more about clear thinking than
it is about writing style. An individual must know what
they want to say, the objective, and why it*s important
for the audience to read it. Written communication
methods used in business are:
C Business Letters
C Business Reports
C Memoranda (Memos)
CSQE 2002

Effective Listening
The skills needed to improve listening are relatively
simple to learn and implement. Perhaps the harder task
is developing an active, or more effective, listening
attitude.
People who listen actively find that they experience

f e w e r m i s t akes and few e r i n t e r p e r s o n a l
misunderstandings.
Ineffective listening is one of the most frequent causes

of misunderstandings, mistakes, rework and customers
dissatisfaction. Some of the reasons for ineffective
listening are:
C Listening is hard work
C Competition
C The rush to action
C Speed difference
C Lack of training
CSQE 2002

Effective Listening (Continued)

Benefits of better listening include:
C It improves relationships
C There are fewer misunderstandings
C Better understanding
People typically listen at one of four basic levels of

attentiveness. As one moves through each level, the
potential for understanding, trust, and effective
communication increases.
C The Non-listener
C The Marginal Listener
C The Evaluative Listener
C The Active Listener

CSQE 2002

Interviewing
Interviewing is a useful technique for soliciting
information about a software product, process, or
technique; gathering requirements from customers and
users; or conducting software assessments and audits.
The characteristics of a good interview include the

ability to know what kind of questions to ask, to listen
effectively, and to provide feedback.
Auvine provides some ideas for the art of asking

questions:
C Avoid leading questions: let the group or

individual draw their own conclusion
C Phrase questions in a positive manner
C Prepare questions in advance of the interview

CSQE 2002

Interviewing (Continued)
One of the most fundamental questioning techniques is
to start with broad, open questions and build on the
speaker*s responses by asking narrower, more specific
questions. This is called the funnel technique.
Formulate questions with the desired objective:
C Have a plan
C Keep the questions simple
C Stay focused
C Avoid manipulation
C Ask permission
C Be non-threatening
C Avoid ambiguity
There are two basic types of questions:
C Closed questions are generally simple,

information gathering questions. Response to
a closed questions is usually a yes or no.
C Open questions are generally more stimulating

and require longer, more complex answers.
CSQE 2002

Applying Communication Elements

to Software Documents
Effective software process and procedural
documentation serves as a communication medium
informing all involved stakeholders of their roles and
responsibilities.
Software documentation serves as a communication

tool for those building the product, monitoring the
project and reviewing the product and processes of the
project.
Reviews are very effective communication vehicles to

evaluate the effectiveness of process and procedural
documentation.
Reviews held during software product development

examine product documentation to learn how the
product is being built. Reviews at the final product
evaluation examine how the product was built.
CSQE 2002

D. ETHICAL CONDUCT AND DEVELOPMENT
1. ASQ CODE OF ETHICS
Ethical Conduct and

Professional Development
Ethical Conduct and Professional Development is

presented in the following topic areas:
C ASQ Code of Ethics

C Software Liability and Safety Issues
C Professional Training and Development
CSQE 2002

Professional Codes of Ethics

A system of ethics exists that guides individuals toward
actions that produce the greatest good for all.
Professional ethics takes into account:
C Relations between practicing professionals and

their clients
C Relations between the profession and society
C Relations among professionals
C Relations between employee and employer
C Specialized technical details of the profession
Organizations that represent the membership of a

profession may undertake to set down a code of ethics
specifically for their members.
CSQE 2002

ASQ Code of Ethics

To uphold and advance the honor and dignity of the
profession, and in keeping with high standards of
ethical conduct, I acknowledge that I:
Fundamental Principles
I. Will be honest and impartial; will serve with

devotion my employer, my clients, and the
public.
II. Will strive to increase the competence and
prestige of the profession.
III. Will use my knowledge and skill for the
advancement of human welfare and in
promoting the safety and reliability of products
for public use.
IV. Will earnestly endeavor to aid the work of the
Society.
Relations with the Public
1.1 Will do whatever I can to promote the reliability

and safety of all products that come within my
jurisdiction.
CSQE 2002

ASQ Code of Ethics (Continued)

1.2 Will endeavor to extend public knowledge of
the work of the Society and its members that
relates to the public welfare.
1.3 Will be dignified and modest in explaining my
work and merit.
1.4 Will preface any public statements that I may
issue by clearly indicating on whose behalf
they are made.
Relations with Employers and Clients
2.1 Will act in professional matters as a faithful

agent or trustee for each employer or client.
2.2 Will inform each client or employer of any
business connections, interests, or affiliations
that might influence my judgment or impair the
equitable character of my services.
2.3 Will indicate to my employer or client the
adverse consequences to be expected if my
professional judgement is overruled.
2.4 Will not disclose information concerning the
business affairs or technical processes of any
present or former employer or client without his
or her consent.
CSQE 2002

ASQ Code of Ethics (Continued)

2.5 Will not accept compensation from more than
one party for the same service without the
consent of all parties. If employed, I will
engage in supplementary employment of
consulting practice only with the consent of my
employer.
Relations with Peers
3.1 Will take care that credit for the work of others
is given to those to whom it is due.
3.2 Will endeavor to aid the professional
development and advancement of those in my
employ or under my supervision.
3.3 Will not compete unfairly with others; will
extend my friendship and confidence to all
associates and those with whom I have
business relations.
CSQE 2002

Professional Conduct and Competence

The Association for Computing Machinery (ACM) code,
like the Institute of Electrical and Electronic Engineers
(IEEE) code, includes two specific items that are of
interest to software quality engineers:
C Maintaining Professional Competence Through

Continued Education
C Providing and Accepting Peer Review of

Technical Material
CSQE 2002

Professional Conduct
Terms and Definitions
Conflict of The circumstance of a public

Interest officeholder, business executive, or the
like, whose personal interests might
benefit from his or her official actions or
influence.
Ethical Being in accordance with the rules or
standards for right conduct or practice,
especially the standards of a profession.
Negligence The failure to exercise that degree of
care that, in the circumstances, the law
requires for the protection of other
persons, or those interests of other
persons that may be injuriously affected
by the want of such care.
Recall Also called callback, a summons by a
manufacturer or other agency for the
return of goods or a product already
shipped to market or sold to consumers
but discovered to be defective,
contaminated, unsafe, or the like.
CSQE 2002

Conflict of Interest Issues

for a Software Quality Engineer
Software quality engineers should be aware of
situations in which a conflict of interest might develop.
Examples of potential conflicts of interest include:
C Providing recommendations on the purchase of

a software change management tool while
owning stock in that company.
C Presenting results of an ISO 9001 pre-

assessment to a client with recommendations
to use consulting services provided by your
company.
C Participating in the awarding of a contract to a

private company founded by a university
professor who also supervises your graduate
study program.
C Evaluating supplier progress in completing

corrective actions while your organization acts
as a consultant completing part of the work for
the same supplier.
CSQE 2002

Software Copyright
A software copyright is the legal right to control who
copies your software. It also governs who can
distribute or sell copies or prepare works based on your
software. The U.S. Copyright Law provides the
following protection:
C It is illegal to copy object code or source code

without permission and use it or sell it.
C It is illegal to copy the text of user

documentation that comes with the program
without permission and use it or sell it.
C It is illegal for someone without authorization to

copy part of the code of a copyrighted program.
C It is illegal to duplicate some aspects of the

external characteristics of a program, for
example, screen designs, animation, music,
and sound.
CSQE 2002

2. SOFTWARE LIABILITY AND SAFETY ISSUES
Software Liability and Safety Issues

Managing the risks of large liabilities is an essential part
of operating a software company. Consequences of
software problems can range from the trivial to
catastrophic.
Under state law, negligence involves carelessness or

failure to exercise ordinary care that results in personal
injury or injury to property. Some states have held that
advisors on computer-related matters may be held liable
for negligent advice, a form of liability sometimes called
computer malpractice.
Fraud
Where software has performed very poorly, customers
may assert claims of fraud, in litigation or arbitration.
Fraud is selling something or obtaining an advantage

with a misrepresentation. Half-truths or misleading
statements count as misrepresentations.
CSQE 2002

2. SOFTWARE LIABILITY AND SAFETY ISSUES
Personal Injury
One area of concern is software that can impact human
health and safety. Errors in this type of software can
lead to large personal injury claims. Strategies for
dealing with this type of liability include:
C Software reliability and quality in product

development and testing
C Mechanisms for notifying customers of hazards

and for safety recalls and upgrades
C Product liability insurance that includes

defense of lawsuits
C Allocating the risk of personal injury claims or

liability with your customer
C Including legal counsel in contract negotiations

CSQE 2002

3. PROFESSIONAL TRAINING AND DEVELOPMENT
Professional Training and Development

Software Quality Training
Training records for the SQA organization should be in

accordance with a company wide established training
plan. The plan would provide information on:
C Initial educational requirements for the position

C Evidence of historical training
C Certification requirements
C Planned internal training for new hires
C Planned off site training courses
Quality Auditor Training
Successful completion of an accredited course is

necessary to meet the training requirements for
individual auditor certification. The ISO 9000/9000-3
standards for quality systems and the ISO 10011
guidelines for auditing quality systems provide useful
information in this area.
CSQE 2002

Professional Training and Development

Software Engineering Training
The most significant software quality initiative recently

instituted is an offshoot of the Software Engineering
Institutes Capability Maturity Model (SEI-CMM).
Many companies have initiated internal software

training sponsored by their Software Engineering
Process Group (SEPG). These are composed of
software professionals who are tasked with developing
processes and procedures for the corporations
software development community.
SEI offers courses for the software professional.
Software Quality Certification
Certification is available for:

C Software Quality Engineers (ASQ)
C Software Quality Auditors
C Software Quality Analysts (Quality Assurance
Institute)
C Software Test Engineers
CSQE 2002

Training Needs Analysis

Training needs analysis is usually used to uncover the
gaps between adequate and inadequate job
performance. If it is performed at the beginning stages
of a training effort, proper content direction should be
indicated. The results of a training needs analysis are
twofold:
1. Determine what is being done now

2. Determine what should be done
10 factors for the planning and implementation of an

effective training program are:
1. Determine needs
2. Set objectives
3. Determine subject content
4. Select participants
5. Determine the best schedule
6. Select appropriate facilities
7. Select appropriate instructors
8. Select and preparing audiovisual aids
9. Coordinate the program
10. Evaluate the program
CSQE 2002

Training and Development

Training and development is the process aimed at
improving the skills and expanding the knowledge of
employees. Generally training will be focused on areas
required for the employees current position.
Training provides very specific employee development

intended to close the gap between current and desired
abilities.
Development is the learning of basic principles, which

can be applied to future conditions.
Education teaches general understanding in selected

subject areas.
Training and development should be a continuous

process for all employees regardless of their position or
age.
CSQE 2002

Training and Development (Continued)

A clear definition of the training objectives and needs is
required prior to development of the training materials.
Training methods include:
C On-the-job training
C Coaching
C Apprenticeship
C Modeling
C Off-the-job training
C Management development
C Mentoring
C Management simulation games
Training must be incorporated as part of the strategic

business plan. Ojectives of training include:
C New employee orientation

C Communication of unique skills unique
C Team building, improvement of productivity
C Imparting new skills
C Disseminating new information
C Development of employees
C Retraining
CSQE 2002

Training and Development (Continued)

The quality of training is not determined by the size of
the training department, but rather the content and
delivery of the training materials, and the interest or
desire by the trainee. The training department staffing
requires three types of people:
C Experts in the subject matter to develop

resource material content
C Technical writers, who design and produce the

training materials
C Teachers, instructors, and presenters who

deliver the training
Students on the average take in information as follows:
C 80% by sight
C 11% by hearing
C 9% by other senses
Smith recommends using more than one sense to help

the student learn.
CSQE 2002

QUESTIONS
2.1. Which of the following standards include elements on

continuous improvement of software processes?
I. Bootstrap
II. ISO SPICE
III. ISO 9001:2000
IV. Trillium
a. I and II only c. II, III and IV only

b. I and III only d. I, II, III and IV
2.2. A capability evaluation performed by a combined

customer-supplier team is called a:
a. Software process assessment

b. Capability joint-assessment
c. Capability evaluation
d. Capability assessment
2.6. Which of the following techniques is most appropriate in

resolving conflict?
a. Accommodating everyones interests

b. Negotiating between the parties
c. Addressing the situations issues
d. Avoiding conflict at all cost
Answers 2.1 d, 2.2 b, 2.6 c

CSQE 2002

QUESTIONS
2.11.Benefits of using Joint Application Development (JAD) for

software requirements include all of the following
EXCEPT:
a. Improves design quality

b. Reduces end user involvement
c. Opens communication channels
d. Builds consensus and ownership
2.13.The key process areas that are identified within the SEI-
CMM are critical indicators that a company is:
a. Maturing as a software development organization

b. Meeting all key practices and common features
c. Becoming a benchmark company in the industry
d. Moving from an ad hoc to a managed development
process
2.21.ISO-compliant software quality systems must satisfy

which of the following?
a. ISO 9001/Q9001 and ISO 9000-3

b. ISO 9001/Q9001 or ISO 9000-3
c. ISO 9001/Q9001 only
d. ISO 9000-3 only
Answers 2.11 b, 2.13 d, 2.21 c

CSQE 2002

QUESTIONS
2.25.The SEI CMM levels from lowest to highest maturity are:
a. Initial, defined, repeatable, managed, optimizing

b. Initial, repeatable, managed, defined, optimizing
c. Initial, repeatable, defined, optimizing, managed
d. Initial, repeatable, defined, managed, optimizing
2.28. The Bellcore quality system requirements are:
a. A supplement to ISO 9001/ISO 9000-3

b. A supplement to the Trillium Model
c. Specialized requirements for test software
d. Reliability requirements for space software
2.29.The Bootstrap method is:
a. An ongoing project of the European Stategic Program

for Research in Information Technology (ESPRIT)
b. An adaption of SEI process assessment method for
the European software industry
c. A departure from the ISO 9000-3 guidelines for
software quality assurance
d. A one-dimensional assessment method that focuses
on the technology applied to process quality
attributes
Answers 2.25 d, 2.28 a, 2.29 b

CSQE 2002

QUESTIONS
2.32.A prevention versus detection philosophy requires:
a. Correction of defects upon discovery

b. Prevention effort early in the life cycle
c. Prevention action upon appearance of defects
d. Independent evaluation of programmer errors
2.36.A SWOT analysis will cause the company to look at

opportunities and threats as part of its analysis. The best
area to look for opportunities is in:
a. New technologies
b. External environment
c. Emerging technologies
d. Internal environment
2.40.In order to select a problem to work on from a list of

contenders, which of the following team tools would a
facilitator be LEAST likely to employ?
a. Nominal group technique

b. Groupthink
c. Multivoting
d. Brainstorming
Answers 2.32 b, 2.36 b, 2.40 b

QUALITY COUNCIL OF INDIANA III-1 (128)
CSQE 2002
III. SOFTWARE QUALITY MANAGEMENT
WE HAVE DISCOVERED THIS

INCREDIBLE MANAGEMENT
PRINCIPLE: PEOPLE TALK TO
EACH OTHER
JOHN Mc CONNELL, Sr.

CEO, WORTHINGTON INDUSTRIES
CSQE 2002

A. GOALS AND OBJECTIVES
1. QUALITY GOALS AND OBJECTIVES
Introduction
Software Quality Management is presented in the

C Goals and Objectives

C Methodologies
Goals and Objectives
Goals and Objectives is presented in the following

topic areas:
C Quality Goals and Objectives

C Outsourced Services
C Planning
C Tracking
C Software Quality Management (SQM)
System Documentation
C Customer Requirements
CSQE 2002

Quality Principles
Quality can be addressed by an organization in their
vision or mission statements. In most cases, one or
more quality elements will be included as guiding
principles.
The term principles means a basic foundation of
beliefs, truths, etc., upon which others are based. Often
these principles are formalized into one or more quality
policies. From quality policies a number of strategic
and tactical quality goals may be developed. An
organizations principles may stress some of the
following points:
C Customer satisfaction is a key

C Defects must be prevented
C Manufacturing assumes quality responsibility
C The process must be controlled
C Everyone participates in quality
C Quality is designed into the product
C Total quality is a group activity
CSQE 2002

Quality Policies
A quality policy represents the overall quality intentions
and direction of an organization regarding quality, as
formally expressed by top management. These policies
will be transmitted to all levels of the organization.
Strategic Quality Goals

Quality goals of a strategic nature, become a part of the
strategic business plan. The quality goals are specific,
quantified, and scheduled.
Quality goals may be linked to product performance,

service performance, customer satisfaction, quality
improvement, or quality costs.
Strategic quality goals cut across many departments

and address issues that are applicable organization
wide.
CSQE 2002

Tactical Quality Goals

Tactical quality goals are the many detailed subgoals
that are derived from strategic quality goals.
Departmental tactical goals should concisely state how

the strategic quality goals (and needs) of the
organization will be implemented.
Software Process Quality

Quality related tasks are inextricably associated with all
of the software development related functions of the
project.
Project
Mgmt
Review
& Audit Change
Vendor
Rev Config
Mgmt
Quality
Standards Documentation
Assurance
Contract
Rev Security
Code & Status

Package
Test
& Ship
Quality as the HUB of the Process

CSQE 2002

Typical Software Quality

Assurance Tasks
Software Quality Assurance tasks typically encompass:
C Generation of quality related documentation
C Review of plans, specifications and records
C Audit of processes, procedures and developers
C Monitoring of plan implementation
C Inspection of code and documentation
C Participate in design reviews and code

inspections
C Corrective Action identify tasks and completion
C Assist in approval of tools and techniques
C Testing participating in or witnessing tests

CSQE 2002

Software Quality Concerns

The introduction of improved software methods is often
slow and requires some degree of constant oversight.
Software engineers must be personally convinced that
the methods proposed in their software development
standards are an effective usage of their time.
Through the application of a strong quality initiative, a

structured, disciplined, and measured personal
software process can eventually evolve. A skilled and
respected quality assurance team can provide the
guidance and feedback needed to help engineers
improve their personal performance.
CSQE 2002

2. OUTSOURCED SERVICES
Outsourced Services
Outsourcing means contracting with a vendor for part or
all of the work of a software and data processing
function. Outsourcing of Information Technology)
services can be accomplished in three ways: straight
acquisition, subcontractor services, and external
resources.
Outsourcing has become a major strategic issue and

many organizations will find that outsourcing can be a
strategic and financial disaster.
Categories of Outsourcing
Process Work
Process work repeats, has a short time-frame, is

standardized, is documented, and operates within the
status quo.
Project Work
Project work is unique, has a long time-frame, is non-

standardized, is difficult to document, and changes the
status quo.
CSQE 2002

Types of Outsourcing
There are three broad types of outsourcing
relationships. Each of these has different strategic,
management and economic issues.
C Global or Strategic
C Tactical or Partial
C Subcontracting or Targeted
CSQE 2002

The Case For Outsourcing

The following are the typical justifications used to
support outsourcing:
C Lower costs
C Risk sharing or reduction
C Economies of scale
C Access to greater skill pool
C Greater focus elimination of non-core

activities
C More control
C More professionalism
C Cash infusion
CSQE 2002

The Case Against Outsourcing

Today it is difficult to obtain evidence to support the
argument against outsourcing, however some factors
are:
C Higher costs
C Risk exposure
C Diseconomies of scale
C Limited access to knowledge base
C Loss of intellectual capital
C Loss of control of core activities
C Conflicting agendas
CSQE 2002

3. PLANNING
Quality Planning Initiatives

Project planning must include quality initiatives at the
very onset of the program. Procedures and resources
need to be identified, as well as, what tools and
techniques will be applied to the development process.
There are key objectives of the SQA function which

should be clearly delineated in the SQA plan. These are:
C Problem prevention
C Identifying problem areas through trend
analysis
C Assisting in identifying and alleviating risk
C Assuring reviews and inspections will be
performed correctly
C Assessing the need and developing procedures
as required
CSQE 2002

3. PLANNING
Quality Planning Initiatives (Continued)

Software quality planning is also based on a
hierarchical set of standards, guides and requirements.
ISO Series DOD-MIL-STDS NASA Bellcore IEEE FCC

Stds.
Corporate Policy
Standard Processes
Procedures (Test, CM, QA)
Procedure Artifacts
Internal/External Guides
SEI-CMM, Coding Stds., etc.
The Structure of a Quality Planning System

CSQE 2002

3. PLANNING
Predicting Software Quality

A hierarchical model of quality factors, criteria and
metrics will be used to predict software quality.
Requirements representing the users needs will be
decomposed using relevant standards and guidebooks
as metric assists in defining both the process and
products quality characteristics.
The measures employed in defining these

characteristics will be defined in a formal documents
usually referred to as the Software Quality Plan.
Software Quality Requirements
The achievement of software quality requirements will

be demonstrated using industry accepted measures of
operational quality, which include:
C Reliability
C Pass/fail criteria
C Traceability
C Software baseline control
C Consistency
C Lines of code (LOC)
CSQE 2002

3. PLANNING
Phase Based Software Activities

Software activities are directly related to the life cycle
phase in which the product resides. A description of
major activities follows:
PHASE TITLE
1 Concept
2 Requirements
3 Design
4 Code/Unit Test
5 Integration Test
6 System Test
7 Field Verification Office (FVO)
Phase Based Software Activities

CSQE 2002

3. PLANNING
Phase Descriptions
A Software Quality Assurance Plan (SQAP) has been
defined by IEEE and DOD. The required topics for each
major phase is:
C Product Introduction
C Documentation Relevant to the Product
C Interrelationships of Supporting Organizations
C Software Programming Practices and

Procedures
C Inspection Procedure
C Software Configuration Management Plan
C Software Library Release Procedures
C Documentation Procedures
CSQE 2002

3. PLANNING
ISO 9000-3 Software Requirements

Planning Requirements
4.2.3 Quality Planning - Software Quality Planning
4.3.2 Contract review - Customer - related concerns
4.4.1 Design Control - General
4.4.2 Design and Development Planning

CSQE 2002

4. TRACKING
ISO 9000-3 Software Requirements (Cont.)

Tracking Requirements
Numerous tracking requirements are detailed in

Planning Requirements.
4.4.9 Design changes
4.5.1 Document and Data Control - Configuration

management
4.8 Product Identification and Traceability
4.13 Control of Nonconforming product
4.16 Control of Quality Records

CSQE 2002

5. SQM SYSTEM DOCUMENTATION
SQM System Documentation

A starting point for discussing Software Quality
Management System (SQMS) documentation is to look
at what constitutes Quality Management System (QMS)
documentation. The SQMS may be the QMS or may be
a part of an overarching QMS.
As a minimum the, QMS documentation must include an

appropriate combination of the following documents:
C Quality policy and quality objectives

C Quality manual that describes the QMS
C Documented procedures
C Other system documentation
Software Quality Management (SQM) is an SEI-CMM

Level 4 activity. Software Quality Assurance (SQA) is
introduced as Level 2 characteristic and involves
process and product reviews and audits.
CSQE 2002

5. SQM SYSTEM DOCUMENTATION
SQM Plan Contents

1. General
1.1. Purpose
1.2. Reference documents
1.3. Management
2. Documentation
2.1. Identification and filing of project documents
2.2. Project document standards
3. Standards, practices, conventions & metrics

3.1. Standards
3.2. Practices & conventions
3.3. Metrics
3.4. Project debrief
4. Reviews & audits

5. Test (tests not covered in SVVP)
6. Problem reporting & corrective action
7. Tools, techniques & methodologies
8. Code control
9. Media control
10. Supplier control
11. Records collection, maintenance & retention
12. Training
13. Risk management
CSQE 2002

6. CUSTOMER REQUIREMENTS
Customer Requirements
IEEE Standard 610.12-1990 defines requirements as:
C A condition or capability needed by a user (or

customer) to solve a problem or achieve an
objective
C A condition or capability that must be met or

possessed by a system or system component to
satisfy a contract, standard, specification, or
other formally imposed documents
IEEE Standard 610.12-1990 defines requirements

analysis as:
C The process of studying user needs to arrive at

a definition of system, hardware, or software
requirements
C The process of studying and refining system,

hardware, or software requirements
CSQE 2002

Software Requirements Specification

IEEE Standard 610.12-1990 defines software
requirements specification (SRS) as documentation of
the essential requirements (functions, performance,
design constraints, and attributes) of the software and
its external interfaces.
A well-written SRS will have the following attributes:
C Correct
C Unambiguous
C Complete
C Verifiable
C Consistent
C Understandable by customer
C Modifiable
C Traced
C Traceable
C Design independent
C Annotated
C Concise
C Organized
CSQE 2002

Problem Analysis Techniques

Object-Oriented Problem Analysis
Most object-oriented approaches stress the definition

and refinement of objects in the real world and classes
of objects in the real world. From a requirements
perspective, an object is any real world entity, important
to the discussion of the requirements, with a crisply
defined boundary.
Objects possess attributes, and objects inherit the

attributes of the classes of the real world, its entities,
their attributes, and their relationship. The primary
motivation for object orientation is that as a system
evolves, its functions tend to change, but its objects
remain unchanged.
Object-oriented techniques:
C Object-Oriented Analysis (OOA)

C Jackson System Development
C Entity Relationship Modeling
C Business Modeling
CSQE 2002

Function-Oriented Problem Analysis
In function-oriented problem analysis, a hierarchy of

functions (also called processes, transforms,
transactions, bubbles, and activities) are created.
At the root of the hierarchy is the most abstract (most

general) function, e.g., provide voice communications.
At the leaf of the hierarchy are the least abstract (most
detailed) functions, e.g., generate a dial tone.
Function-oriented problem analysis techniques:
C Data-flow diagrams (used in most techniques)

C Data dictionaries (used in most techniques)
C Structured requirements definition (SRD)
C Structured analysis and design technique
(SADT)
C Modern structured analysis
C Structured analysis and system specification
(SASS)
C Decision tables and decision trees
C Activity diagrams
CSQE 2002

Problem Analysis Techniques (Cont.)

State-Oriented Problem Analysis
It is often helpful to think of the major states or modes of

operations of the problem domain. Some state-oriented
problem analysis techniques include:
C State charts
C Finite state machines
Behavioral Analysis
Behavioral requirements define precisely what inputs the
software expects, what outputs will be generated by the
software, and the details of relationships that exist
between those inputs and outputs.
Nonbehavioral Analysis
Nonbehavioral requirements define the attributes of the
system as it performs its job. They include a complete
description of the systems requirement levels.
CSQE 2002

Requirements Elicitation Techniques

Several techniques for eliciting requirements:
C Interviewing and questionnaires
C Requirements workshops
C Joint application development (JAD)
C Rapid application development (RAD)
C Brainstorming and idea reduction
C Storyboarding
C Interactive
C Applying use cases
C Role playing
C Prototyping
C Throwaway
C Evolutionary
CSQE 2002

B. METHODOLOGIES
1. REVIEW, INSPECTION AND TESTING
Methodologies
Methodologies is presented in the following topic

areas:
C Review, Inspection, and Testing
C Change Management Methods
C Cost of Quality (COQ)
C Quality Data Tracking
C Problem Reporting and Corrective Action

Procedures
C Quality Improvement Processes

CSQE 2002

B. METHODOLOGIES
Reviews
Software management and technical reviews are
conducted to ensure that appropriate progress is being
made and may evaluate conformance to specification.
Reviews may be on-going as part of a managed quality
initiative or may be directed towards a specific project.
Inspection
The objective of inspections are to locate defects early
in the development cycle. Inspections also compare the
product with applicable standards and specifications.
Testing
Testing is performed to assess functionality, to assess
performance, to locate incorrect or missing functions, to
detect logic errors, etc. As a result of testing, defects
may be corrected, software programs may be modified
and the results may be used for future planning.
CSQE 2002

B. METHODOLOGIES
Usage of Tools in the SQA Function

General purpose SQA tools cover a specific area of the
software development process. These areas include:
C Requirements Tracer
C Database Analyzer
C Complexity Analyzer
C Logic Analyzer
C Reliability Model
C Simulators
C Standards Analyzer
C Data-flow Analyzer
C Interface Analyzer
C Test Generator
CSQE 2002

B. METHODOLOGIES
2. CHANGE MANAGEMENT METHODS
Change Management Methods

Most software engineering involves complex tasks done
under a tight schedule. Process and technology change
has the following tasks:
C Identify the key problems

C Establish priorities
C Define action plans
C Get technical management agreement
C Assign qualified people
C Provide training and guidance
C Launch implementation
C Verify quality of implementation
C Track and report progress
C .Fix inevitable problems
To make major process improvements, software

organizations need an overall plan, some dedicated
process people, clear goals and management
commitment to these goals.
CSQE 2002

B. METHODOLOGIES
Change Management (Continued)

Champions, Sponsors, and Change Agents
Champions are those who initiate the change process.
Someone in a position of authority needs to sponsor the

process change.
Change agents will lead change planning and

implementation.
The Software Engineering Process Group (SEPG) has

the task of initiating and sustaining process change. A
key SEPG role is to ensure that these changes are
effectively implemented and institutionalized.
The SEPG fills the change agent role by providing the

skilled resources to make change happen. When
management approves the action plans, the SEPG leads
in launching the required efforts, providing leadership in
getting them staffed, and supporting the work with
information, training and skills.
CSQE 2002

B. METHODOLOGIES
Change Management (Cont.)

Resistance to Change
Change must be handled with care or it will generate

resistance. Software process change must start with an
unfreezing step. Unfreezing utilizes roles of the
process champion, sponsors and change agents to get
changes moving.
Three key elements of making changes are planning,

implementation, and communication.
To ensure that change is institutionalized, a refreezing

step has the basic objective of retaining improvements
in the general practices of the organization. Education
and training should be part of every software refreezing
plan.
CSQE 2002

B. METHODOLOGIES
Software Process Assessment

Process assessments help software organizations
improve themselves by identifying their critical problems
and establishing improvement priorities. Assessment
objectives are:
C Learn how the organization works

C Identify major problems
C Enroll leaders in the change process
A software process assessment is not an audit but a

review or appraisal of an organizations software
process. It is performed to advise management and
professionals how they can improve their organization.
Conducting Assessments
Assessments are conducted by a team of professionals

to identify the highest priority areas for improvement and
to provide guidance on how to make improvements.
CSQE 2002

B. METHODOLOGIES
Software Process Maturity Framework

The SEI has defined five levels of software process
maturity:
1. Initial - at the initial level, an organization can be

characterized as having an ad hoc, or possibly
chaotic, processes.
2. Repeatable - the organization has achieved a

stable process with repeatable levels of control.
Basic Software Configuration Management
(SCM) and Software Quality Assurance (SQA)
processes exist.
3. Defined - the software process is defined,

technology can be introduced to improve the
process.
4. Managed - comprehensive software process

measurements, SPC can be applied to analyze
and improve the process.
5. Optimizing - continuous improvement and

optimization of software process, SPC is used
to maximize the use of technology, increase
productivity and improve product quality.
CSQE 2002

B. METHODOLOGIES
Assessment Phases
Assessments are typically conducted in three phases:
1. Preparation
2. On-site assessment period
3. Findings and action recommendations are

presented to local management
Software Environments
A software environment is a full set of facilities that
support the efficient use of an effective process. It must
be convenient to use, support customization, have an
open architecture, and have a comprehensive
conceptual schema that encompass the database,
process data, tool interfacing and environment
evolution.
CSQE 2002

B. METHODOLOGIES
3. COST OF QUALITY (COQ)
Cost of Quality (COQ)

Most companies utilize financial reports which compare
actual with budgeted costs. The difference is called a
variance and, if significant, may prompt management
action.
The Traditional Cost Breakdown

CSQE 2002

B. METHODOLOGIES
Quality Cost Categories

Quality costs consist of all those costs associated with
company efforts devoted to planning the quality system,
efforts to verify that quality is being obtained, and
failures resulting from inadequate systems. Quality cost
categories are:
Prevention costs: The costs of activities specifically

designed to prevent poor quality in products or services.
Appraisal costs: The costs associated with measuring,

evaluating, or auditing products or services to assure
conformance to quality standards and performance
requirements.
Failure costs: The costs resulting from products or

services not conforming to requirements or customer/
user needs-that is, the costs resulting from poor quality.
Internal failure costs: Failure costs which occur

prior to delivery or shipment of the product, or the
furnishing of a service, to the customer.
External failure costs: Failure costs which occur

after shipment of the product, or during or after
furnishing a service, to the customer.
CSQE 2002

B. METHODOLOGIES
Quality Cost Categories (Continued)
The Three Levels of Product Costs

CSQE 2002

B. METHODOLOGIES
Contrast: Big Q and Little Q
Topic Content of little Q Content of big Q

Products Manufactured All products, goods
goods and services, whether
for sale or not
Processes Process directly All process,

related to manufacturing
manufacture of support,
goods business, etc.
Customer Clients who buy the All who are affected,

products external and internal
Industries Manufacturing All industries,

manufacturing,
services,
government, etc.,
whether for profit or
not
Cost of Costs associated All costs that would

Poor with deficient disappear if
Quality manufactured everything were
goods perfect
CSQE 2002

B. METHODOLOGIES
Prevention Costs
Applicant screening Pilot projects
Capability studies Planning
Controlled storage Procedure reviews
Design reviews Procedure writing
Education Prototype test
Maintenance Quality design
Field testing Quality incentives
Fixture design Safety reviews
Forecasting Surveys
Housekeeping Time and motion studies
Job descriptions Training
Market analysis Vendor evaluation
Personnel reviews Vendor surveys
CSQE 2002

B. METHODOLOGIES
Appraisal Costs
Audits Laboratory testing
Document checking Other expense reviews
Drawing checking Personnel testing
Equipment calibration Procedure checking
Final inspection Prototype inspection
In-process inspection Receiving inspection
Inspection and test Shipping inspection
Inspection reporting Test equipment
maintenance
CSQE 2002

B. METHODOLOGIES
Internal Failure Costs

Accidents Late time cards
Accounting error Obsolescence (design
corrections changes)
Design changes Overpayments
Employee turnover Premium freight
Engineering changes Redesign
Equipment downtime Reinspection
Excess interest Repair
expense Retesting
Excess inventory Retyping letters
Excess material Rework
handling Scrap
Excess travel expense Scrap allowances
Failure reviews Sorting
CSQE 2002

B. METHODOLOGIES
External Failure Costs

Note that many of the costs related to internal failure
also appear on this list.
Bad debts Liability

Customer complaint Loss of market share
visits Obsolescence
Customer Overpayments
dissatisfaction Penalties
Engineering change Premium freight
notices Price concessions
Equipment downtime Pricing errors
Excess installation Recalls
costs Redesign
Excess interest Reinspection
expense Repair costs
Excess inventory Restocking costs
Excess material Retesting
handling Returns
Excess travel expense Rework
Failure reviews Scrap
Field service training Sorting
costs Warranty expenses
Liability suits
CSQE 2002

B. METHODOLOGIES
4. QUALITY DATA TRACKING
Quality Data Tracking

Quality Data Tracking Principles
The principles of effective quality data tracking are:
C The quality data gathered is tracked in

accordance with pre-defined objectives and
plans.
C The choice of the quality data to be tracked

should be based on a model of the process
being followed.
C The impact of quality data tracking on the entire

organization must be considered.
C Management support and commitment is key to

the quality data tracking process.
If quality data is tracked without a clear objective, it

could be a waste of time and resources.
CSQE 2002

B. METHODOLOGIES
Quality Data Tracking (Continued)

Quality Data Tracking Principles (Continued)
The four prime reasons for tracking quality data are:
C Understanding
C Evaluation
C Control
C Prediction
CSQE 2002

B. METHODOLOGIES
Quality Data Tracking (Continued)

The Impact of Data Tracking
on the Organization
When people know that data about them is being tracked

and analyzed, their performance will generally change.
The effects of people on the results of data tracking

concerns the impact of peoples attitude on the tracking.
In order to achieve all the objectives, it is important that

the data tracking be automated as much as possible. It
is imperative that data tracking be made a part of the
existing software engineering process.
CSQE 2002

B. METHODOLOGIES
5. PROBLEM REPORTING &CORRECTIVE ACTION
Problem Reporting and

Corrective Action Procedures
The purpose of a problem reporting system or procedure
is to report and collect information on the overall status
of the software defect or process non-conformance or
any other quality system deficiency.
Once this overall status is known, a corrective action

system or procedure is then followed to correct the
software defect or correct or make changes to the
existing process to fix the process non-conformance.
The problem reports could be either common cause

(process related) or special cause (product or user
related.)
A software defect is failure of any software code or data

structure in the software product to meet its
requirements or standards.
A process non-conformance is a failure to follow a pre-

defined, accepted and advertised process in the
development of a software product.
CSQE 2002

B. METHODOLOGIES
Problem Reporting Procedures

A software defect or a process non-conformance or any
other quality system deficiency is a deviation of any
product or process from its stated requirements or
standards.
After a problem report is prepared, the data is entered

into a database management system which is used to
help automate the otherwise laborious effort of tracking
defects and providing management reports.
A software defect reporting and tracking system or

procedure should be able to provide information such as
error and correction status, the number of errors found
per product, and the severity and/or criticality of open
defects.
The data enables the impact of these defects to be

evaluated so that the use of resources may be
prioritized.
CSQE 2002

B. METHODOLOGIES
Corrective Action Procedures

These procedures involve the execution of a process to
correct the detected product or process related problem.
When the decision is made to correct the software

defect, there should be procedures to control the
corrective action process. Such procedures should
include follow-up to ensure that the software defect has
been documented and corrected in the appropriate
version of the software, and to assure that adequate
testing, including regression testing has been done.
The typical corrective action process is:
C Identify and record problem

C Name of person and/or organization responsible
C Date assigned to correct the problem
C Status of corrective action
C Reference and date item is closed
A key to corrective action is to be able to track down the

cause of the problem, identify the root cause, determine
corrective action and take action to prevent recurrence.
CSQE 2002

B. METHODOLOGIES
6. QUALITY IMPROVEMENT PROCESSES
Quality Improvement Processes

Quality improvement processes must be earned through
the process of problem solving. Problem solving can
only be achieved if problem or defect prevention,
detection and removal is pursued on a problem-by-
problem or a defect-by-defect basis. Quality
improvement processes can be categorized into three
broad categories. Defect prevention, defect detection
and defect removal.
Defect Prevention
Defect prevention is highly beneficial because the costs
of finding and repairing defects increases exponentially
the latter they are found in the process. Even if defects
are found early in the process, preventing defects is
generally less expensive than finding and fixing them.
CSQE 2002

B. METHODOLOGIES
Defect Prevention (Continued)

The Software Defect Prevention Process
The basic implementation steps for a defect prevention

process are:
1. Defect reporting
2. Cause analysis
3. Action plan development
4. Action implementation
5. Performance tracking
6. Starting over
CSQE 2002

B. METHODOLOGIES
Defect Detection and Removal

The process of defect detection is usually used in
conjunction with defect removal and corrective action
procedures.
Defect detection starts with finding the problem areas.

These are achieved either by following simple charting
techniques to complicated mathematical models of the
program.
Trend Analysis
Trend analysis is comparing recent entity levels to past

entity levels. These levels could be either defect or cost
or any other software product or process entity.
Pareto Analysis
Pareto analysis involves listing the factors that

contribute to the problem and ranking them according to
the magnitude of the contributions. In most cases or
situations, a relatively small number of causes or
sources will contribute a relatively large percentage of
the total defects.
CSQE 2002

B. METHODOLOGIES
Defect Detection and Removal (Cont.)

Pareto Analysis (Continued)
Pareto Analysis is also known as the 80/20 Rule which

helps separate the Vital Few from the Trivial Many. For
example, 80 percent of defects were found in 20 percent
of the modules. The development team should
concentrate their reviews and testing effort on those
modules.
The outcome of this technique is known as a Pareto

Diagram that presents the most frequently occurring
problem and the rest in decaying order.
ERRORS ON ORDER FORMS
150 100
Number of Errors
80
100
60
40
50
20
0 0
G J MQB D CA OR N L I E H K F P
Order Form Item
Pareto Analysis Example

CSQE 2002

B. METHODOLOGIES
Defect Detection and Removal (Cont.)

Reviews
Management and technical reviews are generally

conducted for management and typically provide
information for management action.
Inspections and walkthroughs, on the other hand, are

peer examinations aimed at defect removal thus
assisting the producers in improving their work.
Testing
More time is typically spent in testing than in any other

phase of software development. There is of course no
question that if a software product has not been tested,
it will not work.
The straightforward approach to testing is to detect

defects and fix them. There are several types of
software tests.
CSQE 2002

QUESTIONS
3.4 According to IEEE STD 610.12-1990 requirements are

considered:
I. A condition or capability needed by a user to solve a
problem or achieve an objective
II. A condition or capability that must be met by a system
to satisfy a contract, standard or specification
III. The process of studying user requirements to arrive
at a definition of system hardware or software
IV. The process of studying and refining system
hardware or software detail
a. I and II only c. I, II and III only

b. I and III only d. I, I, III and IV
3.5. A project was initiated without a review and formal

acceptance of the software quality management plan. The
responsibility for this action lies with:
a. The quality manager c. The customer
b. The project manager d. The project team
3.6. Software function-oriented problem analysis might

logically involve which of the following techniques?
I. Jackson system development
II. Data-flow diagrams
III. Object-oriented analysis
IV. Structured requirements definition
a. I, II and III only c. II, III and IV only

b. II and IV only d. I, II, III and IV
Answers 3.4 a, 3.5 b, 3.6 b

CSQE 2002

QUESTIONS
3.16.If a majority of software problem reports can be traced

back to poor requirements definition, the most likely
problem could be:
a. Developers are not implementing the language
correctly
b. The wrong version of the compiler is being used
c. Systems engineering interpreted the customers
requests incorrectly
d. The customer is foreign and language semantics exist
3.18.If a software organization has a foundation for continuous

improvement and is in a position to use SPC to maximize
the use of technology, increase productivity and to
improve product quality, it is said to be at what SEI - CMM
level?
a. Repeatable c. Managed
b. Defined d. Optimizing
3.19.What would be considered the best way to handle software

change?
a. With management commitment, planning and
education
b. By the appointment of change agents with adequate
authority
c. By the use of assessments to identify high priority
areas
d. With adequate resources and knowledge of current
processes
Answers 3.16 c, 3.18 d, 3.19 a

CSQE 2002

QUESTIONS
3.22.The most significant step in process definition consists of:
a. Defining what comes next

b. Establishing a configuration management team
c. Defining entry and exit criteria
d. Establishing management signature requirements
3.25.Which of the following is NOT part of software quality

management?
a. Assisting in establishing risks and contingency plans

b. Holding weekly status meetings to assess developers
progress
c. Reviewing code and documentation for errors
d. Auditing management plans for compliance to
process
3.26.A high rate of customer problem reports is most likely an

indication that:
I. The initial requirements may not have been fully
understood
II. The customer's test facility requires maintenance
III. In-house testing might lack in-depth test case
scenarios
IV. Marketing has oversold the products real capability
a. I, II, III and IV c. I, II and IV only

b. I and III only d. II, III and IV only
Answers 3.22 c, 3.25 b, 3.26 b

CSQE 2002

QUESTIONS
3.30.What is potentially the most expensive form of quality data

tracking?
a. Data that does not have a defined cost base

b. Data that is collected without a clear objective
c. Data that focuses only on external failures
d. Data that does not consider the human element
3.33.What is potentially the greatest weakness in software

problem reporting?
a. Failure to adequately describe the defect

b. Failure to not indicate the time and location of the
defect
c. Failure to follow-up with appropriate corrective action
d. Failure to identify the programmer who created the
defect
3.35.Software quality assurance provides a means for:
a. Assigning quality responsibilities to all key personnel

b. Assuring state-of-the-art design
c. Attaining minimum fabrication cost
d. Achieving 100% inspection of all product units
Answers 3.30 b, 3.33 c, 3.36 a

CSQE 2002

QUESTIONS
3.37.Software quality records are:
a. Those involving software requirements

b. Those related to software code units
c. Those that support the product quality
d. Those defined in the projects test planning document
3.41.The common practices for subcontractor management are

addressed in what SEI CMM level?
a. Level 2
b. Level 3
c. Level 4
d. Level 5
3.44.What is the most overriding reason that many companies

are outsourcing a portion of the work of software and data
processing functions?
a. Lower costs
b. More control
c. Less bureaucracy
d. Less risk exposure
Answers 3.37 c, 3.41 a, 3.44 a

CSQE 2002

QUESTIONS
3.48.Which of the following standards or guidelines are most

applicable to the development of software quality plan
contents?
I. IEEE STD 610.12-1990

II. IEEE STD 730-1989
III. IEEE STD 830-1998
IV. ISO 9000-3-1997
a. I and II only
b. I and III only
c. II and IV only
d. III and IV only
3.49.IEEE STD 730-1989 would NOT address which of the

following elements in the purpose portion of Software Plan
Content?
a. Project description
b. Code and media control
c. Life cycle model
d. Quality Objectives
Answers 3.48 c, 3.49 b

QUALITY COUNCIL OF INDIANA IV-1 (188)
CSQE 2002
IV. SOFTWARE AUDITS
FINDING THE PROBLEM IS

MORE IMPORTANT THAN THE
SOLUTION.
ALBERT EINSTEIN
CSQE 2002
IV. SOFTWARE AUDITS

1. PROGRAM DEVELOPMENT AND ADMINISTRATION
Introduction
Software audits are described in the following topic
areas:
C Program Development and Administration

C Audit Preparation and Execution
C Audit Reporting and Follow-up
Software Audits
Treatment in this Section on Software Auditing are
summary in nature and, where appropriate for
preparation for the CSQE exam, stress the application
of auditing to the software engineering process.
However, there may be questions on the exam which

assume a background knowledge in the principles and
practices of auditing. Other texts are available that offer
more detailed and rigorous treatment of auditing
principles and practices, including the CQA Primer and
many of the references at the end of this Section.
CSQE 2002
IV. SOFTWARE AUDITS

Audit Program Objectives

The audit authority (or audit manager) should take
responsibility to establish clear objectives for the audit
program and for the specific audits that will be
performed. Audits evaluate the performance of the
company and should result in continuous improvement.
Auditing consists of four* general stages:
C Planning and Preparation

C Performance
C Reporting
C Corrective Action and Follow-up
* Some authorities show the same activities in three

to six stages.
CSQE 2002
IV. SOFTWARE AUDITS

Audit Program Objectives (Continued)

ANSI/ISO/ASQC Q10011-1-1994 is a guideline for
auditing quality systems. It outlines the following audit
objectives and responsibilities:
C To determine conformity with a quality standard
C To determine effectiveness of the implemented

quality system
C To provide an opportunity for improvement
C To meet regulatory requirements
C To permit registration to a particular standard
Management reviews, which include a presentation of

audit results, are used to assure that the audit program
is meeting the desired expectations and that audit
results are effective.
CSQE 2002
IV. SOFTWARE AUDITS

Audit Administration
Top management has the responsibility for establishing
and authorizing the organizational quality auditing
program.
ANSI/ISO/ASQ Q10011-3-1994. is a guideline for

managing audit programs, which details the
requirements for registrars and other auditing
organizations. Key elements are:
C Organization for managing audit programs

C Standards
C Qualification of staff
C Suitability of auditors
C Monitoring auditor performance
C Training
C Commitment of resources
C Audit program planning and scheduling
C Audit reporting
C Corrective action follow-up
C Confidentiality
C Joint audits
C Audit program improvement
C Code of ethics
CSQE 2002
IV. SOFTWARE AUDITS

Audit Responsibilities
The following responsibilities are condensed and
paraphrased from ANSI/ASQC Q10011-1-1994.
Auditors are responsible for:
C Complying with audit requirements

C Documenting audit observations
C Reporting audit results
C Verifying the effectiveness of corrective actions
C Retaining and safeguarding documents
C Maintaining confidentiality of the audit
The client:
C Determines the need, scope, and purpose

C Determines the auditing organization
C Receives the audit report
C Determines follow up action
The auditees responsibilities include:
C Informing employees about the audit

C Cooperating with the auditors
C Providing access to facilities and materials
C Determining and initiating corrective action
CSQE 2002
IV. SOFTWARE AUDITS

Lead Auditors Responsibilities

A lead auditor is the audit team leader who is authorized
by the company or organization to manage the planning
and performance of the audit, including the
management of the audit team. The lead auditor is
ultimately responsible for all phases of the audit.
According to ANSI/ISO/ASQC Q10011-1, the lead auditor

should:
C Define the requirements of each audit

assignment
C Comply with applicable auditing requirements
C Prepare working documents and brief the
auditors
C Review documentation
C Report critical nonconformities
C Report obstacles in performing the audit
C Report audit results
CSQE 2002
IV. SOFTWARE AUDITS

Software Audit Definition

From a general point of view, an audit is defined as an
activity to determine, through investigation, the
adequacy of, and adherence to, established procedures,
instructions, specifications, codes, standards, or other
applicable contractual and licensing requirements, and
their effectiveness of implementation.
Specifically related to Software, the ANSI/IEEE 1028

Standard for Software Reviews and Audits, states that
an audit is an independent evaluation of software
products or processes to ascertain compliance to
standards, specifications, and procedures based on
objective criteria that includes documents that specify:
C The form or content of the product to be

produced
C The process by which the products shall be

produced
C How compliance to standards or guidelines

shall be measured
CSQE 2002
IV. SOFTWARE AUDITS

Auditing Standards
ISO 10011
ANSI/ISO/ASQC Q10011-1, 2, 3 - 1994 Guidelines for

Auditing Quality Systems is an authoritative standard
used as an auditing guideline for the development of the
audit program and the implementation of the audit
activities. The following topics are addressed in ISO
10011:
C Standards - to be used in the audit process
C Auditors - criteria for an auditor and team

composition.
C Monitoring and maintenance of the auditor

performance.
C Code of ethics
CSQE 2002
IV. SOFTWARE AUDITS

Auditing Standards (Continued)

ISO 9000
ISO 9001:2000, Section 8.2.2, identifies the requirement

to perform internal audits, which address the following:
C Requirements for documented plans and

procedures for conducting regular internal
audits of the quality system
C Records of results from internal audits
C Evidence of review of the results
C Identification of planned corrective actions and

their follow up
ISO 9000-3:1997 provides additional guidance for

auditing software in the form of:
C Clarification of terms peculiar to software

development
C Considerations which are specific to software

CSQE 2002
IV. SOFTWARE AUDITS


TickIT
A registration scheme for Information Technology (IT),

called TickIT, has been formulated by IT professionals
in the UK. The objectives of TickIT are as follows:
C To ensure ISO 9000 series is applied

appropriately to software
C To ensure consistency of certification with the

IT industry
C To enable mutual recognition of registration

across the IT industry
Under the TickIT scheme, auditors are required to pass

a rigid set of criteria to become TickIT accredited.
CSQE 2002
IV. SOFTWARE AUDITS


SEI Capability Maturity Model
(CMM) Assessments
The Software Engineering Institutes CMM is becoming

a commonly used de facto standard by which to
evaluate an organizations ability to develop software
within a defined development process. The method for
making this determination is termed an assessment as
opposed to the commonly used term of audit when
dealing with audits.
An assessment evaluates the practice in place and

verifies the organizations ability to meet the intent of
the processes described in the CMMs Key Practice
Areas (KPA). The product of the assessment is a gap
analysis defining those areas needing additional
attention for compliance. The report will provide a plan
from which the organization can advance its current
level of maturity.
There are five distinct levels of achievement, and many

contracts are currently requiring that an organization
demonstrate its ability to be equal to or greater than that
of Level II.
CSQE 2002
IV. SOFTWARE AUDITS

2. AUDIT PREPARATION AND EXECUTION
Audit Types
First Party Audit
The first party audit, also known as an internal audit, is

performed within your own company.
Second Party Audit
The second party audit is performed by a customer on

their supplier.
Second party audits are also referred to as external

audits, where you are the one performing the audit.
Another form of second party audit exists when you are

being audited by your customer, and is usually referred
to as an extrinsic audit.
Third Party Audit
A third party audit exists when an outside source is

used to conduct the audit. The third party is contracted
to conduct the audit on behalf of the party of the first
part (client company), on the supplier. A common type
of third party audit is an ISO 9000 registration audit.
CSQE 2002
IV. SOFTWARE AUDITS

Audit Types (Continued)

Internal Audit
An internal audit is performed within an organization to

measure its own performance, strengths and
weaknesses against its internally established
procedures and systems. An internal audit is
considered to be a first party audit.
External Audit
An external audit is an audit performed by company

directive on an outside source, such as a supplier. An
external audit is considered to be a second party audit.
System Audit
A system audit is also known by several other names,

including management audit, operational audit,
programmatic audit, supplier survey and program audit.
It is characterized by its examination of the bigger
picture of the organization and/or project. The
application and effectiveness of system controls for
effective program management are the domain of the
system audit.
CSQE 2002
IV. SOFTWARE AUDITS

Audit Types (Continued)

Process Audit
The process audit examines an activity to verify that the

inputs, actions and outputs are in accordance with
defined requirements. The boundaries of a process
audit lie within a single defined process, for example a
software inspection.
Product Audit
The product audit is an assessment of the final product

or service under contract and its fitness for use.
Compliance Audit
Compliance audits are designed to provide assurance

that defined or specified activities have been performed
properly. Compliance audits that involve software
include the following three types of audits:
C Regulatory Audit
C Management Audit
C Quality Audit
CSQE 2002
IV. SOFTWARE AUDITS

System Audit Matrix

System audits may go by many names making them
difficult to differentiate:
C Extrinsic
C Vendor Survey
C Pre-award Survey
C System Audit
C Management Survey
C Operation Survey
C Assessment Audit
C Appraisal
C Compliance Review
C Quality Review
C Full Audit (Cradle to Grave)
CSQE 2002
IV. SOFTWARE AUDITS

Audit Purposes
There can be no audit unless documented requirements
have been developed. Likewise, some activity must
have taken place in order to measure the
implementation of those requirements. In order to
provide management with the proper feedback on the
implementation process, audits must follow these five
basic precepts:
1. Auditing is a function of management

2. Auditors are qualified to perform their tasks
3. Measurements are taken against defined
standards
4. Conclusions are based on verifiable facts
5. Audit reports focus on the control system
The purpose of management audits is to combine

compliance and effectiveness evaluations.
Any nonconformance is reported as a finding during the

audit. A Corrective Action Request (CAR) is presented
during the exit meeting and is included in the audit
report. The final aspect of auditing is to perform follow
up verifying closure on all corrective actions resulting
from the audit.
CSQE 2002
IV. SOFTWARE AUDITS

Software Audit Process - Objectives

ANSI/IEEE 1028 states the objective of the software
audit process is to provide an objective compliance
confirmation of products and processes to certify
adherence to standards, guidelines, specifications and
procedures.
In performing the audit, audit personnel evaluate

software elements and the processes for producing
them, against objective criteria, such as contract
requirements, plans, specifications, procedures,
guidelines and standards.
The results are documented and are submitted to, the

management of the auditing organization, the entity
initiating the audit, and to any external organizations
identified in the audit plan.
CSQE 2002
IV. SOFTWARE AUDITS

Software Audit Process - Criteria

The following criteria, related to the audit process, must
be established in order to ensure an effective and
efficient audit (ANSI/IEEE 1028, 1988):
C The purpose and scope of the audit must be

clearly identified
C Objective criteria for assessing compliance

must be established
C The software elements and processes to be

audited and any pertinent histories
C Background information regarding the

organization responsible for the products and
processes being audited
C Entry Criteria
C Exit Criteria
CSQE 2002
IV. SOFTWARE AUDITS

Software Audit Frequency

ANSI/IEEE 1028 identifies that the need for an audit is
established by any of the following events:
C A significant project milestone, which is

identified by the charter of the auditing
organization
C Quality System milestone
C External parties require an audit at a specific

calendar date or project milestone
C An internal organization entity has requested

the audit, establishing a clear and specific need
C A major organizational change, such as new

ownership, or possible financial difficulties
According to ISO 10011-1 audit frequency and timing

may be determined by:
C Law or regulation
C The audit program
C Standards
C Client needs
CSQE 2002
IV. SOFTWARE AUDITS

The Audit Schedule

The audit schedule is an overview plan shown on a
timetable. The audits are spread out over a period of
time (usually one year). This is called an annual audit
schedule.
Time is allowed between audits so that the audit

function has the opportunity to plan, conduct, and
report each audit without over-auditing.
Audit schedules are critical to the overall function of the

audit program. The schedule provides a designated
time for the activity. The formality of the schedule helps
to maintain the intended goals of the audit program and
the support of top management.
CSQE 2002
IV. SOFTWARE AUDITS

Auditing Tools and Procedures

Procedures and tools are required both for audit
preparation and for organizing all of the information
gathered during the course of an audit. The audit team
will also need to examine the selected control areas
identified from the various performance standards
chosen for the audit.
Checklists
The checklist serves as a guide to assure that the full

scope of the audit is adequately covered. It also
provides a place for recording data collected during the
fieldwork.
Objective evidence is a statement based on data or

observation.
CSQE 2002
IV. SOFTWARE AUDITS

Auditing Tool and Procedures (Continued)

Authoritative Documents
Audit checklist questions are prepared based on

authoritative documents which are the standards and
audit guidelines against which adherence and
compliance is compared.
Flowchart Usage
The use of flowcharts is often helpful with either tracing

or sampling audits. If the needed flowcharts exist, they
can be requested and followed for accuracy and
compliance. If they do not exist, the auditor may find it
very helpful to make a rough flowchart during the
tracing process.
Interviewing Techniques
Interviewing is the process of obtaining information

from another person in response to questions. It is the
most important form of data one can collect in an audit.
However, it cannot be regarded as conclusive objective
evidence and must be corroborated by an item,
document, or record that verifies the action.
CSQE 2002
IV. SOFTWARE AUDITS

Auditing Tool and Procedures (Continued)

Data Collection Strategy
An auditor may choose to collect valuable information

using a variety of sampling or tracing techniques.
Sampling
Normally auditing does not involve the same large

sample sizes required for inspection applications.
Samples may be collected on a random or discovery
basis.
Trace Forward
The trace forward technique is the examination from the

beginning of a process, product, service, or order to the
end.
Trace Backward
This is the reverse of the trace forward method,

beginning at the end of a process, product, service, or
order and working back through the process, either to
the beginning or to a designated point.
CSQE 2002
IV. SOFTWARE AUDITS

Phases of the Audit Process

The audit process may be divided into four phases:
C Preparation Phase
C Performance (Execution) Phase
C Reporting Phase
C Corrective Action Follow-up Phase
Preparation Phase
The preparation phase starts when the decision is made
to conduct an audit. The following steps should be
performed:
C Define the purpose and scope of the audit

C Determine the audit team resources
C Identify the authority for the audit
C Identify the performance standards
C Develop a technical understanding of the
processes to be audited
C Contact those to be audited
C Perform an initial evaluation of lower-tier
documents to higher level requirements
C Develop written checklists of the data needed
CSQE 2002
IV. SOFTWARE AUDITS

Performance Phase
The performance phase begins with the on-site opening
meeting and includes the gathering of information and
analysis of that information. This phase is completed
when the exit or closing meeting is held.
The performance phase covers the following activities:
C Meeting with the auditee - Opening meeting
C Understanding the software development

process and controls
C Communicating with audit team members
C Interviewing and briefing auditees
C Identifying observations and findings
C Meeting with the auditee - Exit meeting/draft

report
Findings and observations are presented in a factual

manner. A finding is an audit conclusion that identifies
a condition having a significant adverse effect on the
quality of the activity under review.
CSQE 2002
IV. SOFTWARE AUDITS

3. AUDIT REPORTING AND FOLLOW-UP
Reporting Phase
The reporting phase covers the translation of the audit
teams findings and conclusions into a formal audit
report which is accurate, concise, clear, and timely (i.e.
mailed within one week). The audit report should be
consistent with the results presented in the exit
meeting. The reporting phase covers the following
activities:
C Prioritizing the audit results
C Preparing the audit report
C Listing findings and observations
C Including CARs and response dates
C Expressing a judgment of the extent of

auditees compliance
C Approving the report
C Distributing the report
C Retaining audit records according to the audit

plan requirements
CSQE 2002
IV. SOFTWARE AUDITS

3. AUDIT REPORTING AND FOLLOW-UP
Corrective Action Follow up Phase

The auditee is responsible for providing the lead auditor
with a corrective action plan within a specified time
frame. The plan will outline the actions to be taken to
rectify the non-conformance, the individual assigned
and estimated date of completion. The corrective action
request (CAR) is considered for closure once the
corrective action has been implemented and verified.
The corrective action undertaken must demonstrate that

it was implemented in a timely fashion, effective and be
designed to prevent further reoccurrences. The steps
leading to a formal closure of the corrective action
request might necessitate another site visit to verify
implementation and effectiveness of the corrective
action. This phase covers the following activities:
C Proposed corrective action plan (Auditee)
C Implementation of corrective action plan

(Auditee)
C Follow up audit (Auditor)
C Criteria for closure (Auditor)

CSQE 2002
IV. SOFTWARE AUDITS

QUESTIONS
4.3. The software audit process is considered complete when

which of the following occurs?
a. The audit report is published and distributed

b. The auditors follow-up actions have been performed
c. The final findings are presented to the audited group
d. The recommendation report has been submitted
4.6. The most important form of data to collect during an

audit is:
a. Information in response to questions

b. Documents and records
c. Findings of non-conformance
d. Process improvement ideas
4.12. In the Trace Forward method of auditing, the auditor

begins with the software functional requirements, selects
the function(s) of interest, and follows them through the
design, code, and testing processes. Which of the
following is NOT an advantage of Trace Forward
auditing?
a. It shows the processing flow through the

organization
b. It is the most practical technique for partial audits
c. It provides a useful method for training auditors
d. It permits quick detection of deficiencies at the front
end
Answers 4.3 b, 4.6 a, 4.12 b

CSQE 2002
IV. SOFTWARE AUDITS

QUESTIONS
4.15. Deciding upon follow-up action is the responsibility of

the:
a. Audit Client c. Auditor
b. Auditee d. Lead Auditor
4.18. Guidance on implementing quality systems for software

is provided by:
I. ISO 9001
II. ISO 9000-3
III. ISO 10011
IV. TickIT
a. I, II and III only c. I and IV only

b. II and IV only d. II, III and IV only
4.23. The main objective of the software audit process is:
a. To monitor software projects for management

performance
b. To evaluate software processes for cost control
c. To confirm product and process compliance to
standards
d. To train audit personnel for higher-level assignments
Answers 4.15 a, 4.18 b, 4.23 c

QUALITY COUNCIL OF INDIANA V-1 (218)
CSQE 2002
V. SOFTWARE ENGINEERING PROCESSES
IN THE COMPUTER WORLD,

HARDWARE IS ANYTHING YOU
CAN HIT WITH A HAMMER,
SOFTWARE IS WHAT YOU CAN
ONLY CURSE AT.
SOURCE UNKNOWN
CSQE 2002

A. ENVIRONMENTAL CONDITIONS
Introduction
Software Engineering Processes is presented in the

C Environmental Conditions
C Requirements Management
C Requirements Engineering
C Analysis, Design, and Development Methods and
Tools
C Maintenance Management
Environmental Conditions
Environmental Conditions is presented in the

C Life Cycles
C Systems Architecture
CSQE 2002

1. LIFE CYCLES
Life Cycle and Process Models

The software process is a set of tools, methods, and
practices used to produce a software product.
In general, a software engineering process can be

described using three generic phases: definition,
development, and maintenance.
Definition Phase
The definition phase focuses on what. During

definition, the software developer and customer attempt
to identify what:
C Information is to be processed?
C Functions and performances are desired?
C Interfaces are to be established?
C Design constraints exist?
C Validation criteria are required?
Three specific steps will occur:
1. Customer contact
2. Project planning
3. Requirements analysis
CSQE 2002

1. LIFE CYCLES
Life Cycle and Process Models (Cont.)

Development Phase
The development phase focuses on the how:
C Data structures are to be designed

C Software architectures are to be designed
C Procedural details are to be implemented
C Design will be translated to programming language
C Testing will be performed
Three specific steps will occur:
C Design
C Coding
C Testing
CSQE 2002

1. LIFE CYCLES
Life Cycle and Process Models (Cont.)

Maintenance Phase
The maintenance phase focuses on change that is

associated with:
C Error correction
C Adaptations required as the environment evolves
C Enhancements caused by changing requirements
Changes encountered during the maintenance phase:
1. Correction
2. Adaptation
3. Enhancement (Perfective)
4. Reengineering
CSQE 2002

1. LIFE CYCLES
Related Quality Activities

Quality assurance - Reviews are conducted to ensure
that quality is maintained as each phase is completed.
Configuration management - All information created as

part of the definition, development, and maintenance
phases should be uniquely identified and controlled.
Project monitoring - The software engineering process

defines a set of milestones that must be monitored to
ensure that schedule and cost are under control.
Measurement - The software engineering process and

products can be measured and quality and productivity
determined.
CSQE 2002

1. LIFE CYCLES
The Waterfall Model

The oldest paradigm for software engineering is the
classic waterfall model. This paradigm takes a linear,
sequential view of the software engineering process.
SYSTEM
ENGINEERING
ANALYSIS
DESIGN
CODE
TESTING
MAINTENANCE
The Waterfall Model
Each box encompasses a set of tasks that result in the

production of one or more work products. Time is
tracked from left to right and top to bottom.
CSQE 2002

1. LIFE CYCLES
The Waterfall Model (Continued)

The major features of the waterfall chart are the
following:
C Systems Engineering
C Analysis
C Design
C Coding
C Testing
C Maintenance
Waterfall Model
Advantages/Disadvantages
The waterfall model provides a template into which
methods, for analysis, design, coding, testing, and
maintenance, can be placed.
Real projects rarely follow this sequential flow. It is

often difficult for the customers to explicitly state the
requirements. A working version of the software is not
available until late in the project.
CSQE 2002

1. LIFE CYCLES
Prototyping
Prototyping is a process that enables the developer to
create a model of the software which is built in an
evolutionary manner.
REQUIREMENTS
GATHERING &
ANALYSIS
QUICK DESIGN
REFINING
DESIGN & BUILDING
PROTOTYPE PROTOTYPE
CUSTOMER
EVALUATION
OF PROTOTYPE
CUSTOMER
SATISFIED
FULL SCALE
DEVELOPMENT
The prototype may either be built on the target

environment or on a convenient platform. The prototype
is evaluated by the customer/user and is used to refine
the requirements.
CSQE 2002

1. LIFE CYCLES
Spiral Model
The spiral model is an evolutionary model. The model
defines four major activities:
1. Planning
2. Risk analysis
3. Engineering
4. Customer evaluation
With each iteration around the spiral, progressively

more complete versions of the software are built.
CSQE 2002

1. LIFE CYCLES
Incremental Development Model

An incremental approach is commonly applied as a
technique to break a large programming effort into
smaller, more manageable components. Each
component is developed in isolation from the rest and
integration of the components is postponed until they
are complete.
The Incremental Model

CSQE 2002

1. LIFE CYCLES
V-Model
The software engineering process may be viewed as a
V-Model or Decomposition/Integration Model. Time
progression is from left to right, and the project
components become more and more specific and
granular as a project moves down the left side of the
V.
The components are integrated to form larger and larger

components moving up on the right side of the V.
V Model of Software Engineering Process

CSQE 2002

1. LIFE CYCLES
Object-Oriented Development
The Object-Oriented (OO) development paradigm begins
with requirements gathering. Once objectives for the
software are defined, Object-Oriented Analysis (OOA)
begins. OOA is an activity in which classes and objects
are extracted from the initial statement of the
requirements. Then Object-Oriented Design (OOD) is
used to create a model of each of the program
components known as objects.
The software library is reviewed to determine if any

existing program components can be used. If reusable
components are found, they are used as building blocks
to construct a prototype of the software.
CSQE 2002

1. LIFE CYCLES
Hybrid Development Models

Most commercial software projects use a combination
of elements from more than one life cycle model. This
project management approach allows experienced
managers to tailor a model that best satisfies the project
requirements.
Comparison of Life Cycle Models

The Primer lists a comparison of the relative strengths
and weaknesses of the major software engineering life
cycle models:
C Waterfall
C Prototyping
C Spiral
C Incremental
C V-Model
C OOA/OOD
C Hybrid models
CSQE 2002

2. SYSTEMS ARCHITECTURE
Systems Architecture
A systems architecture is defined in terms of a
collection of components and interactions among those
components.
Client Server
Client-server systems require a network and at least two
machines to operate. The server receives input
requests from the client and manipulates the data by
applying the applications business logic rules.
N-Tier
An N-tier configuration refers to networks employing
multiple servers. Services are provided directly for
users applications and indirectly for one another.
Network servers often spread the workload across

multiple computer systems. Each service runs on a
cluster of systems, called a tier. Each tier can be
changed and scaled independently of the other tiers,
thereby making the site more scalable and supportable
overall.
CSQE 2002

Systems Architecture (Continued)
Credit Internet
Cert
Card Firewall
Server
Server
Database
Server App Web
Proxy
Server Server Server
FTP
Server
DNS E-Mail
Print
Server Server
Server
File
Win
Server
Server
Figure 5.1 N-Tier Configuration
C Business-to-Business (B2B): Web sites create new

opportunities for businesses to make transactions
employing TCP/IP extranet or virtual private network
connection between companies.
C Business-to-Consumer (B2C): Web sites designed

for advertising retail products directly to consumers
and processing their orders over the Internet.
C Business-to-Employee (B2E): Transactions

performed by business employees using web sites.
CSQE 2002

Web (Internet/Intranet/Extranet)
The Internet is a network of physical connections of
fiber optics, copper wire, routers, hubs and switches
that provide an infrastructure for worldwide
communications between computers.
The Internet is not the same as the web, although these

terms are often erroneously used interchangeably. The
World Wide Web (www) is just one of the many
applications that use the infrastructure of the Internet.
An Intranet is a private computer network using the

same technical standards as the Internet. It limits
access to people within the originating organization.
Extranet is an internal network using Internet

technology that accesses two or more organizations
intranets to form a network.
An Internet Virtual Private Network (IVPN) uses the

Internet to connect remote intra-organizational networks
together.
Tunneling refers to the technology that enables one

network to send its data via another networks
connections.
CSQE 2002

B. REQUIREMENTS MANAGEMENT
1. REQUIREMENTS PRIORITIZATION
Requirements Management
Requirements Management is presented in the

C Requirements Prioritization and Evaluation
C Requirements Change Management
C Bi-directional Requirements Traceability

CSQE 2002

Requirements Prioritization
and Evaluation
A requirements specification document contains
background information necessary for understanding
the proposed product and explains and justifies the
need for various product attributes.
A software requirements specification will describe the

ideal desired product.
The project development team organizes its resources

to implement the product requirements using the
prioritizing method described in the software
requirements specification.
Unambiguous
A requirements statement is unambiguous if all of those

involved interpret the statement in the same way.
CSQE 2002

Requirements Prioritization (Continued)

Correctness
A requirements statement is said to be correct if it

describes a condition or attribute that is required of the
final product. In practice this is established through a
requirements review process.
Verifiable
The requirements specification should clearly describe

how each requirement will be verified. Concrete terms
and measurable quantities should be used whenever
possible.
Consistency
The set of requirements should be evaluated to assure

that individual requirements do not conflict with one
another.
Completeness
The requirements are evaluated to assure that all

significant requirements for the product are provided.
CSQE 2002

2. REQUIREMENTS CHANGE MANAGEMENT
Requirements Change Management

Requirements will change as specification, design,
development, and test proceed. This complicates
development considerably, but is unavoidable. Every
project must have procedures for making changes to
requirements as development proceeds.
Procedures for managing changing requirements need

to include the following activities:
C Submitting requirements change requests
C Evaluating requirements change requests
C Reviewing requirements change requests
C Scheduling changes to documents, code, and tests
C Implementing the scheduled changes

CSQE 2002

3. BI-DIRECTIONAL REQUIREMENTS TRACEABILITY
Bi-directional Requirements Traceability

Requirements tracing means providing explicit
information that identifies all the lower level
requirements that derive from a higher level
requirement, and all the design or code elements that
implement each specific requirement.
A requirements trace table is added to the lower level

document that explicitly identifies each higher level
requirement that the lower level requirement supports,
or the design element implements. This is termed a
backwards trace table.
A simple tool can invert this table to provide a forward

trace table that should then be added to the higher level
document.
CSQE 2002

3. BI-DIRECTIONAL REQUIREMENTS TRACEABILITY
Requirements to Design Verification

Where explicit requirements are not given for a design
entity, requirements traceability can still be used if the
requirements that the entity enables are explicitly cited.
In this case the forward trace table would just name the
design entity that enable each requirement. As before,
the review of each entity would verify that all the
enabled higher level requirements are cited, and only
the enabled requirements were cited.
Design to Code Verification

To trace from design to code, requirements must be
explicitly stated at the component level and forward
traced to the component classes and functions that
enable that requirement.
Requirements to Test Verification

Requirements tracing can also be used to assure
thorough and complete testing. Each system test can
cite the customer system requirement covered by that
test.
CSQE 2002

C. REQUIREMENTS ENGINEERING
1. REQUIREMENT TYPES
Requirements Engineering
Requirements Engineering is presented in the

C Requirement Types
C Requirements Elicitation
C Requirements Analysis and Modeling
C System and Softw are Requirements

Specifications
CSQE 2002

Requirement Types
There are many definitions of requirement in software
engineering.
A requirement is a quality or condition that matters to

someone who matters.
IEEE Standard 610.12-1990 defines a requirement as:
A. A condition or capability needed by a user to

solve a problem or achieve an objective.
B. A condition or capability that must be met or

possessed by a system or system component to
satisfy a contract, standard or specification.
C. A documented representation of a condition or

capability as in definition (A) or (B).
IEEE Standard 1220-1998 defines a requirement as:
A statement that identifies a product design

characteristic, which is unambiguous, testable or
measurable, and necessary for product or process
acceptability.
CSQE 2002

Requirement Types (Continued)

Requirement types include the following:
C Functional Requirements
C Quality Requirements
C Performance Requirements
C Regulatory Requirements
C Security Requirements
CSQE 2002

2. REQUIREMENTS ELICITATION
Requirements Elicitation
Software requirements analysis can be divided into five
areas of effort:
1. Problem recognition
2. Evaluation and synthesis
3. Modeling
4. Specification
5. Review
The analysis focuses on the system specification and

the software project plan. The goal of the analyst is the
recognition of the basic problem elements as perceived
by the customer/user. Problem evaluation and solution
synthesis must:
C Evaluate the flow and content of information

C Define and elaborate all software functions
C Understand software behavior in the context of
events that affect the system
C Establish system interface characteristics, and
uncover design constraints
Each of these tasks serves to describe the problem

domain so that the overall approach or solution may be
correctly synthesized.
CSQE 2002

Requirements Gathering Processes

The development process begins by understanding the
organizations business requirements. This leads to
a vision and scope document that describes the
background leading to the decision to develop a new or
modified system or capability and describes the system
to be developed.
Recommended requirements elicitation techniques,

which can be used in combination:
C Interviews
C Document Analysis
C Brainstorming
C Requirements Workshops
C Joint Application Development (JAD)

C Rapid Application Development model (RAD)
C Prototyping
C Use Cases
C Storyboards
C Interfaces Analysis
C Modeling
C Performance and Capacity Analysis
CSQE 2002

Requirements Gathering Practices
1. Write a project vision and scope document.

2. Initiate a project glossary that provides definitions
of words and acronyms.
3. Evolve real requirements via a joint customer/user
and developer effort.
4. Document the rationale for each requirement.
5. Provide training for analysts and customer/user
representatives.
6. Establish requirements change control mechanism.
7. Prioritize the real requirements.
8. Consider an incremental development approach.
9. Use peer reviews and inspections of requirements.
10. Use an automated requirements tool.
11. Use proven requirements gathering techniques.
12. Provide domain/subject matter experts.
13. Provide a mechanism to share information and
best practices among projects.
14. Establish a continuous improvement ethic,
teamwork approach, and a quality culture.
15. Involve customers and users throughout the
development effort.
16. Perform requirements validation and verification.
CSQE 2002

Requirements Elicitation Obstacles

There are a number of obstacles to requirements
elicitation that must be overcome to clearly define
business requirements.
C User Procedures
C Current Capabilities
C Formal Business Rules
C Gold Plating
C Requirements Creep
CSQE 2002

Quality Function Deployment (QFD)

Quality Function Deployment objectives is to see that
the voice of the customer is heard throughout all
stages of product development.
QFD House of Quality

CSQE 2002

3. REQUIREMENTS ANALYSIS AND MODELING
Requirements Analysis and Modeling

Requirements analysis is the first technical analysis
step in the software engineering process. A general
statement of the software scope is refined into a
specification.
The requirements analysis task is a process of

refinement, modeling, and specification derivation. The
software scope is modeled using a combination of
graphical notation, and textual language renditions of
the problem.
The software requirements specification (SRS) is

developed as an outcome of analysis.
CSQE 2002

DFD Data Flow Diagram

DFDs are a graphical notation methods that depict the
information flow, and the transforms that are applied, as
the data moves from the input to the output. The DFD
may be used to represent the system or software during
requirements analysis and any other level of detail.
A producer or consumer of information that resides

outside the bounds of the system to be modeled.
A transformer of information that resides within the

bounds of the system to be modeled.
A data item or collection of data items. The arrowhead

indicates the direction of data flow.
A repository of data that is to be stored for use by one or

more processes. It may be as simple as a buffer or
queue or as sophisticated as a relational data base.
DFD Basic Notation

CSQE 2002

Control Flow Model

One method for control flow modeling is to represent
control flow through the generation of control flow
diagrams (CFD). Data flow is represented using a solid
arrow, while control flow is represented using a dashed
arrow.
CSQE 2002

Data Dictionary
The data dictionary is an organized listing of all data
elements that are pertinent to the system, with precise
rigorous definitions. This is known as the data
modeling component of structured analysis.
Entity Relationship Diagrams

The cornerstone for relationship data modeling is the
entity-relationship (E-R) diagram, whose purpose is to
identify data objects and their relationships. Data
objects are represented by labeled rectangles.
Relationships are indicated with diamonds.
E - R DIAGRAM
DATA OBJECT TABLE
TITLE SECTIONS QUESTIONS EDIT REVIEW ETC.

CSQE 2002

Requirements Analysis
Common Characteristics
Requirements analysis methods have the following
common characteristics:
C Mechanisms for information domain analysis
C Approachs for functional representations
C Definitions of interfaces
C Mechanisms for problem partitioning
C Support for abstraction
C Representations of essential views
Data Structure Oriented Analysis
Data structure oriented analysis methods represent

software requirements by focusing on data structures
rather than data flow.
CSQE 2002

Warnier-Orr Methodology
Data Structure Systems Development (DSSD), also
called the Warnier-Orr methodology, evolved from work
by J.D. Warnier. Warnier developed a notation for
representing an information hierarchy using three
constructs for sequence, selection and repetition.
DSSD Notation
CSQE 2002

Jackson System Development

Jackson System Development (JSD) focuses on models
of the real-world information domain analysis and its
relationship to system and program design. To conduct
JSD, the analyst applies the followings steps:
C Entity action step C Functional step

C Entity structure step C System timing step
C Initial modeling step C Implementation step
Jackson System Development Illustration

CSQE 2002

Object-Oriented Requirements Analysis

When using an object-oriented paradigm, the boundary
between analysis and design is fuzzy because of the
interplay between the two. In analysis, a model is
formed by discovering the classes and objects that form
the problem domain, while in design the abstractions
and mechanisms that provide the behavior of the model
are invented.
Several approaches for specification and analysis for

object-oriented systems are listed below.
C Categorical
C Behavioral
C Domain
C Use-Case
C Textural
C Structured
CSQE 2002

4. SYSTEM AND SOFTWARE REQUIREMENTS
System Requirements
Specification Process
The specification of system requirements is a function
of system engineering, which is a problem solving
activity.
Systems engineering is best described as an iterative

process of top-down synthesis, development, and
operation of a system that satisfies the full range of
system requirements. The desired function and
behavior of the system are uncovered, analyzed, and
allocated to the individual engineering components.
Trade off Criteria

C Project considerations
C Business considerations
C Technical analysis
C Manufacturing evaluation
C Human issues
C Legal considerations
C Environmental interfaces
C Off-the-shelf solutions
CSQE 2002

System Specification Document

The system specification document describes the
function and performance of the computer-based
system, the constraints that affect system development,
and information system inputs and outputs.
Requirements Specification Format

The system requirements specification must describe:
C Functions and capabilities of the system

C Business, organizational, and user requirements
C Safety, security, and ergonomic requirements
C Operational and interface requirements
C Constraints
C Qualification requirements
A suggested outline for system specification:
C Introduction
C Functional and data descriptions
C Subsystem descriptions
C System models
C System validation requirements
CSQE 2002

Software Requirements Specification

The objective of software requirements specification is
to refine the scope of the software requirements and
produce an operational software component that can be
integrated with the other components of the system.
Software requirements specification principles:
C The specification should separate functionality from

implementation
C The desired behavior of the system should

encompass data and functional responses to the
system to environmental stimuli.
C A cognitive model should be developed.
C The specification should describe the system as

perceived by the user.
C The specification should be developed so that it is

amendable to change.
C Specification information should be nested in

increasing levels of detail.
CSQE 2002

D. ANALYSIS, DESIGN, AND DEVELOPMENT
1. SOFTWARE DESIGN METHODS
Analysis, Design, and

Development Methods and Tools
Analysis, Design, and Development Methods and

Tools are presented in the following topic areas:
C Software Design Methods
C Types of Software Reuse
C Clean Room and Other Formal Methods
C Software Development Tools

CSQE 2002

Software Design Methods

The goal in software design is to produce a model or
representation of the entity that will later be built.
Information Domain
Software is built to transform data from one form to
another. Software also processes events. The
information domain contains three different views of the
data and control as each is processed by the computer
program:
C Information Flow represents the manner in which

data and control change as each moves through the
system.
C Information Content represents some form of initial

individual data, and control items, that are
combined to form a larger information domain as a
new record with its associated attributes.
C Information Structure is the representation of the

relationships that exist between different pieces of
information.
CSQE 2002

Models
Models are created during requirements analysis to gain
an understanding of the entity to be built. Models must
be capable of illustrating how the software transforms
information.
Partitioning
During software requirements analysis, the problems
that are to be solved are often too large to be
understood as a whole. The problem is partitioned into
segments that can be more readily understood.
Object-Oriented Design
Object-Oriented Design (OOD) is a method of design
encompassing the process of object-oriented
decomposition and a notation for depicting both logical
and physical as well as static and dynamic models of
the system under design. Major conceptual elements in
the object-oriented model are abstraction,
encapsulation, modularity, and hierarchy and three
minor elements are typing, concurrency, and
persistence.
CSQE 2002

Object-Oriented Design (Continued)

Objects have states, behaviors, and identities. The state
of an object encompasses all of the properties of the
object plus the current values of each of the properties.
Behavior is how an object acts and reacts, in terms of
its state changes and message passing. The identity of
an object is the property that distinguishes it from all
other objects.
Object-Oriented Analysis (OOA) is done to identify the

functional, behavioral, and data requirements for the
system. OOA consists of five major activities that are
done together to describe the problem/solution:
C Finding the Classes and Objects

C Identifying Structures
C Identifying Subjects
C Defining Attributes
C Defining Services
OOD is used to create models of the program and each

of the program components (objects). OOA and OOD
occur interactively until a reasonable design for the
system has been derived.
CSQE 2002

Structured Analysis and Design (SAD)

Structured analysis is a widely used technique that
consists of procedures that allow the analyst to
decompose software functions; a graphical notation
that communicates the relationships of information and
function within software; and project control guidelines
for applying the methodology.
Using data and control flow diagrams, the analysis

partitions the functions that transform the flow.
CSQE 2002

Unified Modeling Language (UML)

UML is a general-purpose visual modeling language that
is designed to specify, visualize, construct and
document the artifacts of an object-oriented software
system.
The language is based on a small number of core

concepts that can be combined and extended so that
expert object modelers can define large and complex
systems across a wide range of domains.
UML is designed to document abstract models at

multiple layers and from several viewpoints. The four
levels of abstraction in UML
C Meta-metamodel
C Metamodel
C Model
C User objects (user data)
Using UML, a single description can cover the business

logic, user activities, system behavior, and the software
architecture.
CSQE 2002

Program Design Steps

Classical software design is a process through which
requirements are translated into a representation of
software.
Preliminary design transforms the requirements into

data and software architectures.
Detailed design refines the architectures into detailed

data structures and algorithmic representations of the
software.
The resulting designs cover the data, architecture,

procedures, and interfaces.
CSQE 2002

2. TYPES OF SOFTWARE REUSE
Types of Software Reuse

Effective software reuse is regarded as a desirable
characteristic of the software development processes.
A software component should be designed and

implemented so that it can be used in many different
programs. In the context of creating software systems,
reuse is viewed as an activity.
Reuse can best be defined as any procedure that

produces a software component or a software system,
by reusing something from a previous system or
development effort.
Advantages of Reuse
C Cost savings
C Reliability
C Efficiency
CSQE 2002

Reengineering
Reengineering is a type of reuse that takes on a system
view and is generally applied in specialized
circumstances. Reengineering is an activity or process
to rebuild an existing system.
Software Reengineering Paradigm

Reengineering of software is a time and cost intensive
activity. Therefore, a sound strategy for reengineering
must be developed before the activity is started. The
following activities define a pragmatic and cyclic
reengineering process model. The software
reengineering paradigm includes:
C Inventory Analysis
C Documentation Restructuring Analysis
C Code Restructuring
C Data Restructuring
CSQE 2002

Reverse Engineering
Reverse engineering can be viewed conceptually as a
reuse method that involves a disassembling or
decomposition process. Reverse engineering is a
process of design recovery from software source code,
usually because the original design has been lost or
obscured over time.
Plug-and-Play
Plug-and-play means that a self-contained software
component can be automatically incorporated into an
existing system. The software component performs any
reconfiguration necessary in real time.
Extensive regression testing and requalification must be

done to verify that the components will play with any
configuration deployed in the field. As future system
applications are developed, the application must be re-
qualified to ensure compatibility with the application
and all configurations of the application.
CSQE 2002

3. CLEAN ROOM AND OTHER FORMAL METHODS
Defect Prevention Methods

Defect prevention reduces to good programming and
management support for qualified software engineers.
A documented standard of good programming practices
is necessary to provide a road map for the development
life cycle.
Using Standards for Defect Prevention
Standards provide a documented approach to achieving

quality products through the adoption of recognized
best industry practices. Standards application should
be consistently applied throughout the project.
Documentation Standards
A standard method applied to the writing of

requirements and design, provides an approach to
Formal Technical Reviews (FTR). Documentation
standards can be viewed as a defect prevention
mechanism.
CSQE 2002

Classical Cleanroom
With the classical cleanroom process, software is
developed using a statistical quality control approach to
the review process. This procedure is defect prevention
rather than defect removal. This is achieved by using
human mathematical verification to prepare software for
system test.
This process provides validated statistical certification

of the softwares quality. The measure of the derived
quality is often stated as the mean time to failure.
Software engineers are organized into the following

three separate engineering activities:
1. A specification team creates a formal specification

for each software increment.
2. A development team designs and codes each

software increment using a formal approach called
the box-structured strategy.
3. A certification team performs all testing and uses

tracks and analyze errors and their causes.
CSQE 2002

Classical Cleanroom (Continued)
Classical Cleanroom Paradigm

CSQE 2002

Modified Cleanroom
The Modified Cleanroom approach usees less formal
methods that are nonetheless quite rigorous and can
deliver code with very low fault densities. The
hallmarks of a modified cleanroom approach are:
1. Explicit, enumerated requirements at the

component and program function level
2. Explicit detailed design
3. Explicit correctness arguments that the detail

design correctly implements the requirements
4. The use of formally conducted software

inspections to peer review the correctness
arguments and verify that the code correctly
implements the detail design
Defect Removal Methods

Defect removal is really defect detection followed by a
repair operation. The practice of reviewing the products
takes place during all phases of the software life cycle.
CSQE 2002

Formal Reviews
Formal technical reviews are the most effective way to
uncover and correct errors. Technical reviews are
applied at various points during software development
and serve to uncover defects that can then be removed.
Technical reviews are needed, simply because large

classes of errors escape the originator more easily than
they might escape others using a different paradigm.
The formal technical review (FTR) is sometimes called

an inspection or walkthrough.
Subjects of Reviews
The subject matter for formal reviews are: requirements

specifications, top-level-design specifications,
intermediate design specifications, detailed design
specifications, code and test specifications.
Planning documents are subject to review but are not

directly germane to the subject of defect removal.
CSQE 2002

4. SOFTWARE DEVELOPMENT TOOLS
Software Development Tools

Software development tools perform more than one
task, often in several categories. The following are
software development tool categories:
C Management Tools
C Modeling Tools
C Design Tools
C Code Analysis and Testing Tools
C Documentation Tools
C Relational Databases
CSQE 2002

E. MAINTENANCE MANAGEMENT
1. MAINTENANCE TYPES
Maintenance Management
Maintenance Management is presented in the

C Maintenance Types
C Operational Maintenance
CSQE 2002

Maintenance Types
This Section describes the characteristics of corrective,
adaptive, and perfective maintenance types and their
benefits and risks.
Characteristics of Software Maintenance

While maintenance of existing software can account for
over 70% of all effort expended by a software
organization, it is likely to be completely ignored in the
planning and costing of a software development project.
Software Engineering
Aspects of Maintenance
The concept of software maintenance has different
implications when contrasted with other engineering
fields. For example software, unlike hardware, cannot
wear out. In most fields of engineering, maintenance
affects only new instances or generations of the
product.
Software maintenance has more to do with fixing errors

and with the evolution of the software.
CSQE 2002

Software Evolution
The term maintenance usually implies keeping
something that can wear out or break in good repair.
Since software does not wear out or break, the term
software evolution has come into use with respect to
describing software maintenance. Software evolution
more accurately describes the types of activities that
occur during the process of maintaining software.
Software Maintenance
Software maintenance is defined as a set of engineering
activities that occur after software has been delivered to
a customer and put into operation.
Software Maintenance Process

Software maintenance follows an orderly sequence:
C A change request or error is documented

C Analysis is performed to determine the impact
C Requirements, design and code are analyzed
C Testing
C Revisions are issued
CSQE 2002

Categories of Software Maintenance

Corrective Maintenance
Corrective maintenance is defined as changes required

to correct defects found during the operation of a
software system or execution of a software application.
Adaptive Maintenance
Adaptive maintenance results in modifications to the

software to accommodate changes to its external or
operating environment.
Perfective Maintenance or Enhancements
Perfective maintenance extends the software beyond its

original functional requirements and are changes made
in response to user or customer requests or to improve
the efficiency and/or documentation.
Often perfective maintenance is referred to as an

enhancement.
CSQE 2002

Maintenance Costs
The following are typical comparisons in the industry in
terms of overall maintenance effort of the defined types
of maintenance:
C Corrective maintenance (17%)

C Adaptive maintenance (18%)
C Perfective maintenance (65%)
Maintenance Risks and Benefits

Perfective maintenance introduces the most risk and
activities typically result in growth and degradation of
the softwares structure. The net result is that software
tends to increase in size and its structure tends to
degrade with time.
Quick Fix Model

The Quick Fix Model involves a simple modification to
the software code with little thought to how it might
impact the overall structure of the system. This
involves analyzing the code and establishing what must
be changed to implement the specified functionality.
CSQE 2002

Iterative Enhancement Model

This maintenance model starts with an analysis of the
existing systems requirement, design, code, test and
analysis documents.
The highest level artifact which is affected by the

purposed change is then modified to reflect the change
and the resultant change is then propagated downward
through the full set of system artifacts.
Maintainability
Maintainability is defined as the ease with which
software can be understood, corrected, adapted, and/or
enhanced.
Factors that Control Maintainability
Carelessness of design, coding, and testing has a

negative impact on the ability to maintain the software.
A poor configuration can have a similar negative impact.
CSQE 2002

Maintenance Side Effects

Modification of software is dangerous. When used in
the context of software maintenance, the term side
effects implies an error or other undesirable behavior
that occurs in other elements of the program as the
result of modification. Three major categories for side
effects are:
C Coding Side Effects
C Data Side Effects
C Documentation Side Effects

CSQE 2002

2. OPERATIONAL MAINTENANCE
Operational Maintenance
Software Maintenance Framework
The software maintenance framework describes some

of the factors that contribute to the maintenance
challenge. The elements of this framework are the:
C User
C Environment
C Maintenance process
C Software product
C Maintenance personnel
A maintenance organization/group must be established,

reporting and evaluation procedures must be described,
and a standard sequence of events must be defined for
each maintenance request.
A record keeping procedure for maintenance activities

must be established, as well as a definition of review
and evaluation criteria.
CSQE 2002

Operational Maintenance (Continued)

Software Maintenance Framework (Continued)
MAINTENANCE
REQUEST
INFORM
REQUESTER
ERROR TYPE IMPROVEMENT
DON'T
EVALUATE
DO
SERIOUSNESS MINOR
ADAPTATION
DO
NO YES
INFO
EVALUATE REQUIRED
MAJOR CATEGORIZE
PRIORITIZE
PRIORITIZE CONSULT
CATEGORIZE
TO TOP WITH
PRIORITIZE
OF LIST REQUESTER
SELECT
TASK FROM
LIST
PLAN &
DESIGN
SOLUTION
COMPLETE NO
YES
END DO OTHER ACTIVITIES
Software Maintenance Flow

CSQE 2002

System Documentation
The system documentation includes all of the
documents describing the implementation of the system
from the requirements specification to final acceptance
test plan. System documents aid product maintenance.
System documentation should be structured, with an

overview leading the reader into more formal and
detailed descriptions of each aspect of the system.
CSQE 2002

Lehmans Laws
Lehmans laws are one of the few examples in software
engineering of theories that have been derived from
observations. The proposed laws were derived from
these measurements.
C Continuing change
C Increasing complexity
C Large program evolution
C Organizational stability
C Conservation of familiarity
CSQE 2002

Program Understanding
Program understanding is fundamental to an effective
change process. Prior to implementing any change, it
is essential to understand the software product as a
whole and the programs affected by the change in
particular.
Program understanding consumes a significant

proportion of maintenance effort and resources.
Examples of descriptive models of how maintenance

engineers go about understanding programs include:
C Top-down
C Bottom-up or chunking
C Opportunistic
CSQE 2002

Maintenance Personnel
Developers tend to think of development as a form of
puzzle solving, and it is reassuring when they
successfully complete a difficult section of code.
Software maintenance, on the other hand, entails very

little new creation and is therefore categorized as dull,
and unexciting detective work. Several steps that can
improve this negative image and motivate the
maintenance staff:
1. Couple software objectives to organizational goals.
2. Couple software maintenance rewards to

organizational performance.
3. Integrate software maintenance personnel into

operational teams.
4. Create a discretionary, preventive maintenance

budget that allows the maintenance team to decide
when to reengineer parts of the software.
5. Involve maintenance staff early in the software

process during standards preparation and reviews.
CSQE 2002

Maintenance Tools
A software maintenance tool is any artifact that
supports a software maintainer in performing a task.
Use of tools can significantly simplify the maintenance
task, increase efficiency and productivity, and support
the evolution of the entire software system. A taxonomy
of maintenance tools includes:
C Program Understanding and Reverse Engineering

C Program slicer
C Static analyzer
C Dynamic analyzer
C Data flow analyzer
C Cross-referencer
C Dependency analyzer
C Transformation tool
C Testing
C Simulator
C Test case generator
C Test path generators
C Other Tasks
C Configuration Management
C Documentation
C Complexity assessment
CSQE 2002

QUESTIONS
5.1. A defined software life cycle process should be:
a. Aligned with the International ISO Standard

b. Built in accordance with the SEIs CMM
c. A reflection of the actual life cycle practice in place
d. Maintained on the LAN
5.5. In order to assure that a software product release

implements a set of features, one would perform:
a. Tracing code to test cases

b. Tracing code to requirements
c. Tracing test cases to requirements
d. Tracing design to test cases
5.10.Which of the following software tools would be most

useful for describing an object oriented programming
solution?
a. Inventory Manager
b. E-R Diagram
c. Test Generator
d. Outline Processor
Answers 5.1 c, 5.5 c, 5.10 b

CSQE 2002

QUESTIONS
5.12.Identify the life cycle model would be most applicable for

a large software development project moving an existing
application to a new platform without changing any
features or user interfaces:
a. Waterfall
b. Prototyping
c. Degenerative
c. Spiral
5.16.When would extensive regression testing and

requalification be necessary for a plug and play
component?
a. If the component were to be reused into another software

system
b. If the cost savings for usage warranted it
c. If the reliability of the component were questionable
d. If the issue of efficiency were of paramount importance
5.20.Which of the following best describes the three phases of

the software engineering process?
a. Code, test and ship

b. Definition, development and maintenance
c. Generate requirements, hire developers and code
d. Specify documentation, code and deliver product
Answers 5.12 a, 5.16 a, 5.20 b

CSQE 2002

QUESTIONS
5.27.Coding can best be described as the:
a. Last part of software development

b. Transference of design requirements into a machine
readable language
c. Establishment of a network design
d. Key step in generating test data
5.28.A Rapid Prototyping approach to software development

allows for:
I. The refining of the design elements through each

iteration
II. Customer evaluation of each incremental development
cycle
III. Quicker organization and completion of the project
IV. Validating the end product in smaller units
a. I, III and IV only c. I, II, III and IV

b. II, III and IV only d. I, II and IV only
5.31.In the Spiral Process the engineering element is

constantly involved in:
a. Reallocating the budget

b. Establishing management commitment
c. Developing the next level of the product
d. Establishing a configuration management plan
Answers 5.27 b, 5.28 d, 5.31 c

CSQE 2002

QUESTIONS
5.33.The KEY feature of Object-Oriented programming is which

of the following?
a. Its ability to portray code graphically

b. Its ability to create reusable components
c. It is easier to learn than ADA
d. It no longer requires establishing a library for generated
code
5.35.The usage of CASE tools in a software development

project:
I. Alleviates the need for configuration management

II. Aids in maintaining the consistency of file naming
conventions
III. Provides the ability to cut & paste the deliverable
documentation
IV. Does the code generation automatically
a. I and IV only c. I, II, III and IV

b. II and III only d. II and IV only
5.39.A formal or technical review, may also be referred to as a:
a. Validation process
b. Inspection or walkthrough
c. Tape verification
d. Reengineering Paradigm
Answers 5.33 b, 5.35 b, 5.39 b

CSQE 2002

QUESTIONS
5.44. Software requirements analysis would NOT involves

which of the following?
a. Problem recognition
b. Evaluation and synthesis
c. Modeling
d. Marketing analysis
5.51. When changes are incorporated into a product because

of requests from the customer/user community, its
termed:
a. Perfective maintenance
b. Corrective maintenance
c. Adaptive maintenance
d. A customer complaint
5.52.A Software Engineering Process Group (SEPG) would

NOT be tasked with:
a. Establishing a documented process

b. Institutionalizing change
c. Encouraging discipline in software practices
d. Monitoring process changes for effectiveness
Answers 5.44 d, 5.51 a, 5.52 c

CSQE 2002

QUESTIONS
5.59.Why is it necessary to capture software interfaces during

requirements elicitation?
a. To identify internal software interfaces at the component

level
b Interfaces are a design constraint and should not be
included in requirements
c. Interfaces are created by requirements analysts, user
information is not needed
d. To identify external interfaces early clarifies the product
scope
5.62.The belief that as an evolving program changes, its

structure tends to become more complex is known as:
a. McCabes cyclomatic complexity

b Lehmans law
c. Weibull distribution
d. Reverse engineering
5.65.The Unified Modeling Language (UML) would NOT be used

to describe:
a. Modeling languages
b Object-Oriented program behaviors
c. Data structures and transformations
d. Graphical User Interfaces
Answers 5.59 d, 5.62 b, 5.65 d

QUALITY COUNCIL OF INDIANA VI-1 (296)
CSQE 2002
VI. PROGRAM AND PROJECT MANAGEMENT
BEGIN AT THE BEGINNING ...

AND GO ON TILL YOU COME
TO THE END; THEN STOP.
LEWIS CARROLL
CHARLES LUTWIDGE DODGSON
CSQE 2002

INTRODUCTION
Program and Project Management
Program and Project Management is presented in

C Planning
C Tracking and Controlling
C Risk Management
CSQE 2002

A. PLANNING
1. PROJECT PLANNING ELEMENTS
Planning
Planning is presented in the following topic areas:
C Project Planning Elements

C Goal-setting and Deployment
C Project Planning Tools
C Cost and Value Data
CSQE 2002

A. PLANNING
Software Project Planning Elements

Software Project Definition
Software projects are initiated through many sources,

including:
C Contractual obligations
C Requests for quote or proposal
C Market demands through feasibility studies
C Internally initiated research and development
efforts
C Customer requests
C Business needs
C Strategic need for new technologies
CSQE 2002

A. PLANNING
Software Project Scope Management
In order to conduct a successful software project one

must understand the scope of work to be done, the
risks to be incurred, the resources required, the tasks to
be accomplished, the project goals and milestones to
be tracked, the effort to be expended (cost) and the
scheduled to be followed.
Basic software project management processes, to

address these issues, are set out in the Level 2 key
process areas of the CMM. In particular, the CMM
identifies the core elements that must be defined in the
process of software project scope management.
The scope of the project must be clearly defined before

a software project starts.
Software Project Estimation - Forecasting
The next critical step in software project planning is the

software estimation process. Estimation is one of the
major activities defined in the CMM - as a level 2 key
activity and is used to forecast the resources, cost and
schedule development of a software project.
CSQE 2002

A. PLANNING
Software Project Planning (Continued)

Software Project Forecasting (Continued)
The Constructive Cost Model or COCOMO Model may be

used for software estimation. The system is
decomposed and then lines of code (LOC) are estimated
for each major decomposition element to provide effort
and schedule forecast. The software project planning
cycle is a series of iterative negotiations that define:
C The initial baselined requirements
C The development of a work breakdown

structure (WBS) which defines the work
C Defined and secured commitments to support

the tasks and constraints
C Product element sizing estimates
C The resource requirements
C A schedule which is compatible with the

proposed delivery requirement
CSQE 2002

A. PLANNING
Software Project Planning

Definition Phase
The estimates are compared with the software projects
delivery requirement and a final commitment is made.
Initial
Consensus
&
Commitment
Reconciled
Requirements Allocate
Requirements
WBS Estimate S/W

Sizing
SLOC
Evaluate
Project
Resources
Man Months
Develop
& Dollars
Initial
Schedule
Realign Schedule
Estimates
Resolve
Schedule
Conflicts
Project GO
Develop S/W
Product
Product
Estimates Compare to
Baseline
Estimates
Development Planning Cycle

CSQE 2002

A. PLANNING
Initiation of Customer Project Goals

A software project that has a specific customer should
initiate a series of conferences with the customer to
identify the objectives, and project scope and thereby
establish the goals of the project.
Contract Review Goals

For projects which have a contractual commitment, the
software project planning process includes a
comprehensive contract review to affirm the
corporations performance capability. This step is an
ISO certification requirement.
A satisfactory contract review process will constitute

the basis for the development of subsequent program
plans addressing the methodology and processes for
implementation. A record of the contract review
process should be retained in the project file. Any
discrepancies should be documented with task
assignment and tracking to closure.
CSQE 2002

A. PLANNING
Software Management Plan (SMP)

Key Topics
The software management plan is generated after

completion of the initial estimates and schedules. This
document contains a textual description of the
interrelationships of the projects various sub-elements
that will be utilized in the realization of the product
build. The following outline is a recommended
structure for a SMP as required by NASA.
1.0 Introduction
2.0 Related Documentation
3.0 Purpose and Description of Software Project
4.0 Resources, Budgets, Schedules and Organization
5.0 Acquisition Activities Plan
6.0 Development Activities Plan

CSQE 2002

A. PLANNING
SMP (Continued)
7.0 Sustaining Engineering and Operations Activities
Plan
8.0 Assurance Plan
9.0 Risk Management Plan
10.0 Configuration Management Plan
11.0 Delivery and Operational Transition Plan
12.0 Abbreviations and Acronyms
13.0 Glossary
14.0 Notes
15.0 Appendices
CSQE 2002

A. PLANNING
2. GOAL-SETTING AND DEPLOYMENT
Goal-setting and Deployment

Goal setting and deployment activities are used for the
entire software project life cycle. Critical project
elements for which goals are established in the project
planning activities, are as follows:
C Size, cost, schedule and task duration
C Critical project resources
C Project product quality
C Software technical activities
C All measurement and metrics data required by

the technical staff, management or customer
CSQE 2002

A. PLANNING
2. GOAL-SETTING AND DEPLOYMENT
Goal Deployment Methods

Deployment methods for goal setting activities take the
form of establishing gating points with reviews. Gating
reviews may be held throughout the framework
established by the life cycle of the project. These gating
points usually map to the major milestones of the
software development process.
Milestone reviews are typically held at completion of the

following:
C Definition of project deliverables
C Finalization of all project design activities
C Completion of all integration test activities
C Completion of all system test activities
C Final evaluation of all project deliverables
C Final customer acceptance

CSQE 2002

A. PLANNING
3. PROJECT PLANNING TOOLS
Estimation
An activity in the planning of software is estimation of
the following:
C Effort required (man-months)
C Resources (machine, manpower, and money)
C Project duration (expressed as milestones)
Estimation techniques share the following attributes:
C The project scope must be thoroughly defined
C Past measures of effort, durations and costs

are used as a basis for the new project
estimate
C The project is broken down to small tasks each

of which are estimated individually
CSQE 2002

A. PLANNING
Estimation (Continued)
In order to get a credible cost estimation, use more than
one software estimating parametric model. The inputs
to these models usually consist of:
C Analogy
C Engineering assessment
C Subject matter experts (SME)
C Parametric modeling
CSQE 2002

A. PLANNING
COCOMO Estimation Model

The basis for many modeling tools is Barry Boehms
COnstructive COst MOdel (COCOMO). There are three
versions of the model:
C Basic - development effort and cost are derived

from program size estimated in lines of code.
C Intermediate - this computation considers

subjective assessments of product, hardware,
personnel and project attributes.
C Advanced - incorporates all of the intermediate

issues and cost drivers of analysis and
software design of the engineering process.
The cost drivers of a project can be further subdivided

into four major categories, each of which have
additional attributes:
1. Product attributes
2. Hardware attributes
3. Personnel attributes
4. Project attributes
CSQE 2002

A. PLANNING
Automated Estimation Tools

The estimation tools listed below provide the planner
with cost and effort, provide what-if scenarios, and
show schedule variances across the life cycle of the
project. Each tool will require some quantitative
estimate of project size, in either lines of code (LOC) or
function points, qualitative characteristics (complexity,
reliability and criticality) and development environment.
The tools and their developers are:
C Before You Leap (BYL) Gordon Group
C Wang Institute Cost Wang Institute

Model (WICOMO)
C DEC Plan Digital Equipment Corp.
C SLIM based on the Rayleigh-

Norden curve and the
Putnam Estimation model
C Checkpoint Software Productivity

Research model based on
the works of C. Jones.
CSQE 2002

A. PLANNING
Network Planning Tools

Common applications of network planning include the
Program Evaluation and Review Technique (PERT), the
Critical Path Method (CPM), and Gantt Charts. The
network rules are:
C Before an activity may begin, all activities

preceding it must be completed.
C Arrows imply logical precedence only. The

length and direction have no meaning.
C Any two events may be directly connected by

only one activity.
C Event numbers must be unique.
C The network must start at a single event, and

end at a single event.
CSQE 2002

A. PLANNING
Program Evaluation and

Review Technique (PERT)
PERT requirements are:
C All project individual tasks must be included in

the network.
C Events and activities are sequenced in the

network to allow determination of the critical
path.
C Time estimates are made for each activity and

stated as three values: optimistic, most likely,
and pessimistic elapsed times.
C The critical path is the sequence of tasks which

requires the greatest expected time.
The slack time, S, for an event is the latest date an event

can occur or be finished without extending the project,
(TL), minus the earliest date an event can occur (TE). For
events on the critical path, TL = TE, and S = 0.
S = T L - TE
CSQE 2002

A. PLANNING
PERT Chart Example

An example of a PERT chart is shown below. Circles
represent the start and end of each task. The numbers
within the circles identify the events. The arrows
represent tasks and the numbers along the arrows are
the task durations in weeks.
PERT Chart Example
Events 1 and 5 on the chart are burst points.

Events 6, 7, and 8 are sink points.
The critical path is 0-1-3-5-7-8-9-10, which is the longest

duration path. Tasks which are late in ending may delay
the project, and can modify the critical path. Projects
not on the critical path may be delayed by an amount
equal to the slack time without delaying the completion
of the project.
CSQE 2002

A. PLANNING
Critical Path Method (CPM)

The critical path method (CPM) is activity oriented while
PERT is event oriented. Unique features of CPM
include:
C The emphasis is on activities

C The time and cost factors for each activity are
considered
C Only activities on the critical path are
contemplated
C Activities with the lowest crash cost (per
incremental time savings) are selected first
C As an activity is crashed, it is possible for a
new critical path to develop
For each activity, there is a normal cost and time

required for completion. To crash an activity, the
duration is reduced, while costs increase. Crash means
to apply more resources to complete the activity in a
shorter time.
CSQE 2002

A. PLANNING
CPM Example
Using information from the PERT chart example, and
adding crash times and costs:
CPM Example
Each activity arrow on the PERT chart example

becomes a circle on the CPM example. The letter
indicates the activity and a number. The number is the
normal activity duration in weeks. The critical path is
indicated by the thicker arrows, A-C-F-I-K-L-M.
CSQE 2002

A. PLANNING
CPM Example (Continued)

If the project is done in a normal manner the time is
calculated by adding the normal durations for events on
the critical path A-C-F-I-K-L-M and is 28 weeks at a cost
of $48,400. If we wish to complete the project in 27
weeks, we must crash an activity on the critical path.
The lowest crash cost per week saved, for an item on

the critical path is activity K at $150/week. The total
project cost increases to $48,550, and the time is
reduced to 27 weeks. If K is crashed, we must
recalculate the critical path.
To shorten the project further, we must crash another

activity on the critical path. This process can be
repeated and the CPM time-cost trade-off determined.
CSQE 2002

A. PLANNING
CPM Time-Cost Trade-off
CPM Time-Cost Trade-off Example
Crashing activities in an attempt to reduce the project

duration below 20 weeks, increases cost without further
reduction in time.
The assumption made in crashing an activity is that it is

independent of other activities.
CSQE 2002

A. PLANNING
Critical Path Analysis

The critical path in a project schedule is the sequence
of tasks which require the greatest expected time. It is
the route along which specific tasks must be
accomplished within the allotted time in order that the
overall project duration not be lengthened.
Scheduling tools will identify the critical path so that the

status can be easily interpreted. The majority of these
tools portray the critical path with a bold red line.
Example of Critical Path Analysis using Software

CSQE 2002

A. PLANNING
Work Breakdown Structure (WBS)

A work breakdown structure (WBS) is a grouping of
project elements that organize and define the total work
scope of the project. Each descending level in the WBS
represents an increasingly detailed definition of the
project work and is developed through a decomposition
process.
Decomposition subdivides project deliverables into

smaller components until the deliverables are defined in
sufficient detail to support development of project
activities, with individuals or groups responsible for
each task.
Decomposition involves the following steps:
C Identify the major deliverables of the project.
C Decide if adequate cost and duration estimates

can be developed for each deliverable.
C Verify correctness of decomposition by

determining if each item can be appropriately
scheduled and budgeted.
CSQE 2002

A. PLANNING
Project Sizing
Size estimation is an input to cost estimates. The
biggest difficulty in using the cost estimation algorithms
available today, is the problem of providing sound
sizing estimates.
The estimated software size may be expressed in lines

of code (LOC) or function points. The estimates
provided are expressed in LOC or KLOC and serve as
the basis for estimating other project deliverables - such
as documentation.
Lines of code - Although commonly used as a software

project metric, LOC lack a standard definition and often
lead to erroneous estimates. In most instances, the
metric is viewed as better than nothing regardless of
the inconsistencies in this approach.
CSQE 2002

A. PLANNING
Definition of Function Points

A function point is a measure of software functionality
based on the counted or estimated number of inputs,
outputs, inquiries, and interfaces (externals) of a
system, plus the estimated number of internal files.
The five items that are counted in computing the

function point measure are:
C External Inputs (weighting factor 4X)
C External Outputs (weighting factor 5X)
C External Inquiries (weighting factor 4X)
C External Interfaces (weighting factor 7X)
C Internal Files (weighting factor 104X)
Note some authors use different weighting factors.

CSQE 2002

A. PLANNING
Albrecht - Function Points

This technique measures the function the software is to
perform, from requirements information, in order to
estimate the effort required to develop the software.
Size Estimation Steps
C Compile the requirements for the system and

computer software configuration items (CSCI)
or software product
C Obtain system data (functions to be performed,

counts of inputs and outputs)
C Get other product detail
C Get the data for as many of the component

functions or parts of the system as possible
C Consult individuals with expert knowledge of

the components or functions
C an underestimate of the size of one component

often cancels the overestimate of the size of
another component.
CSQE 2002

A. PLANNING
Summary of Sizing Recommendations

C Estimate the size of every product (at least
each CSCI) separately
C Test the estimated size for compatibility with

the schedule and estimated development effort
C Make size/development effort tradeoff studies

for every product
C Track code growth during development
C Develop and use an organizational experience

database to aid in size estimation
C Estimate size in several ways
C Make size estimates throughout development
C Base estimates of size on experience retained

in database
C Relate code size to the size of other products

such as the amount of design
CSQE 2002

A. PLANNING
4. COST AND VALUE DATA
Cost and Value Data

Project cost management includes the processes and
cost data required to ensure that the project is
completed within the approved budget. The following
processes are involved in project cost management:
C Resource planning process
C Cost estimating process
C Cost budgeting process
C Cost control process

CSQE 2002

A. PLANNING
4. COST AND VALUE DATA
Earned Value Management (EVM)

Earned value management continuously measures
performance by relating three independent variables:
1. The planned value - the physical work

scheduled to be performed
2. Earned value - which is physical work actually

accomplished
3. The actual cost incurred to accomplish the

earned value
Earned value requires that a project be fully defined at

the outset (complete WBS) and then a bottom-up plan
created. Measurement takes place during the entire
period of the projects performance.
Using earned value, a software project manager can

determine if a project will use up all the projects
resources before project completion, forcing software
features to be dropped to stay within budget.
CSQE 2002

B. TRACKING AND CONTROLLING
1. PHASE TRANSITION CONTROL TECHNIQUES
Tracking and Controlling
Tracking and Controlling is presented in the

C Phase Transition Control Techniques

C Interpreting and Reporting Cost of Quality
(COQ) Data
C Tracking Elements and Methods
C Project Reviews
CSQE 2002

Tracking and Controlling

Activities for tracking and monitoring a software
development process include:
C A documented software development plan
C Management approves all commitments
C Changes to commitments are communicated
C The following are are tracked and corrective

actions are taken:
Project size, cost, and schedule

Critical computer resources
Project product quality
Software technical activities
C Measurement and metrics data, for software

project tracking, are recorded for use
C Regular reviews track status, plans,

performance, and issues against the plan
C At project completion, a review captures

significant events as lessons-learned
CSQE 2002

Phase Transition Control Techniques
Scheduling
The estimation process uses past project data to
develop productivity factors to assist in allocating the
required resources for the development process. Data
points are documented in a project schedule to assist
management in tracking the progress of the project.
Some of the factors that must be considered are:
C The software development environment
C Programmer skill levels
C Language usage and compiler stability
C Adjunct resources - such as test beds
C Schedule commitments to customer
C Software development paradigm in place
C Corporate culture
CSQE 2002

Project Tracking/Schedule Development

The sizing and estimation data generated must be
captured in order to determine project status
throughout the program.
One approach is to use an earned-value method based

on project status. This method assigns a percentage of
completion based on complexity to the various tasks
associated with a milestone.
Once the tasks have been defined to provide a

reasonable level of visibility, they are usually tracked to
completion by use of a software scheduling tool.
CSQE 2002

Budgeting
As projects progress, management is concerned with
the current cost of the program as compared to the
original estimation. Performance is measured by
comparing three quantities:
1. The budgeted cost of work performed (BCWP)

2. Actual cost of work performed (ACWP)
3. Budgeted cost of work scheduled (BCWS)
Earned Value Analysis

CSQE 2002

Gantt Charts
One method of showing project status is a Gantt chart.
Task bars are scaled in length to denote the duration of
the tasks. Their relative positions denote the sequence
of tasks with their respective start and completion
dates.
Milestones are goals that project management has

committed to accomplish within a defined time frame.
Through weekly status reviews and the establishment of
prioritized action items, tasks are rescheduled to reduce
risk and aid in reestablishing new dependencies.
Project - X-Wing
03/24 03/31 04/07 04/14 04/21 04/28 05/05 05/12 05/19

Develop Module XDBY
Code
Inspection
Unit Test
Fix
Retest
Review & Accept
A Milestone Gantt Chart

CSQE 2002

2. INTERPRETING AND REPORTING COQ DATA
Cost of Quality (COQ) Data

Improvement Sequence
C Define company quality goals and objectives
C Estimate capabilities of current processes
C Develop realistic programs and projects

consistent with the company goals
C Determine the resource requirements for

approved programs and projects
C Set up quality cost categories of prevention,

appraisal, and failure to accumulate costs
C Arrange for accounting to collect and present

quality costs
C Insure accurate figures or reasonable estimates

by category
C Analyze the quality cost data for major

improvement candidates
C Utilize the Pareto principle to isolate specific

vital areas for investigation
CSQE 2002

Quality Cost Comparison Bases

Quality costs should be related to two or three different
volume bases:
Labor bases:
C Total direct labor (worked)
C Standard labor (planned)
Manufacturing cost bases:

C Direct labor
C Direct material
C Indirect costs
C Production engineering costs
C Provision for complaints
C Packing and shipping
Sales bases:
C Net sales billed
C Contributed value
Unit bases:
C Quality costs, dollars per unit of production
C Quality costs related to production
CSQE 2002

Typical Quality Cost Report

Quality Cost Report for April, 2002
Dollars ($) Percent of Total
Prevention Costs
Quality Control Administration 5250 2.1
Quality Control Engineering 14600 5.9
Other Quality Planning 1250 0.5
Training 2875 1.2
Total Prevention 23975 9.7
Appraisal Costs
Inspection 55300 22.3
Test 23800 9.6
Vendor Control 1700 0.7
Measurement Control 1950 0.8
Materials Consumed 375 0.2
Product Quality Audits 800 0.3
Total Appraisal 83925 33.8
Internal Failure Costs
Scrap 66500 26.8
Repair, Rework 1900 0.8
Vendor Losses 2500 1.0
Failure Analysis 4000 1.6
Total Internal 74900 30.1
External Failure Costs
Failures - Manufacturing 14500 5.8
Failures - Engineering 7350 3.0
Failures - Sales 4430 1.8
Warranty Charges 31750 12.8
Failure Analysis 7600 3.1
Total External 65630 26.4
Total Quality Costs 248430 100.0
Bases
Direct Labor 94900 8.1
Conversion Cost 476700 40.8
Sales 1169082 100.0
Ratios
Internal Failure Costs to Direct Labor 78.9
Internal Failure Costs to Conversion 15.7
Total Quality Costs to Sales 21.3
CSQE 2002

Advantages of a Quality Cost System

C Provides a single overview of quality
C Aligns quality and company goals
C Provides a problem prioritization system
C Improves the effective use of resources
Limitations of a Quality Cost System

C Measurement does not solve quality problems
C Cost reports do not suggest specific actions
C Costs are susceptible to mismanagement
C Important costs may be omitted
C Inappropriate costs may be included
Quality Cost Pitfalls

C Perfectionism in the numbers
C Other data pitfalls
C Inclusion of non-quality costs
C Implications of reducing quality costs to zero
C Reducing quality costs but increasing total
company costs
C Understatement of quality costs
CSQE 2002

Optimum Quality Costs

The total quality curve is depicted in the theoretical
model. The lowest total cost point is ideal. The location
of this point can be shifted, based on competitive
market conditions and technological advances.
As prevention and appraisal costs increase, the failure

costs will decrease until an optimum point is reached.
After this point, increases in appraisal and prevention
costs will not be offset by the decreased failure costs.
Optimum Quality Cost Model

CSQE 2002

Optimum Quality Costs (Continued)
Hypothetical Quality Costs Trends Over Time

CSQE 2002

3. TRACKING ELEMENTS AND METHODS
Project Tracking Elements

Purpose of Tracking
The purpose of tracking project elements is to establish
controlling processes which:
C Track to closure
C Ensure proper oversight
C Tracking and control project costs
C Project risks
Tracking Methods
Project reviews and reporting should focus on:
C Impact on project costs
C Timeliness of effective staffing
C Monitoring the quality level of the project
C Identifying top risks and contingency plans

CSQE 2002

3. TRACKING ELEMENTS AND METHODS
Cost Metrics
Earned Value
A tool for software project tracking and cost oversight

cost is a methodology for calculating earned value.
Tracking and oversight should answer the following
questions:
C What problems are being caused by the cost

variance?
C What is the impact of cost variance on

performance?
C What is the impact of cost overruns on project

effort?
C What are the corrective actions for cost

overruns and their status?
CSQE 2002

4. PROJECT REVIEWS
Project Reviews
The project review is a formal and documented critique
conducted by a committee of qualified company
personnel. The project review extends over all phases
of software development, from inception to completion.
A project review considers all of the important factors in

the creation of a mature product design. Fundamental
review topics include:
C The adequacy of personnel, time, equipment

and money
C The project effectiveness as determined by
internal and external information
C The effectiveness and reliability of corrective
actions
C The true quality level of the delivered product
and/or service
The tollgate acts as a go/no go decision point at

transition points in the project, such as beta testing and
software release.
CSQE 2002

4. PROJECT REVIEWS
Senior Management Reviews

The Capability Maturity Model (CMM) mandates that
senior management oversee project activities on a
periodic basis. The primary purpose of periodic reviews
by senior management is to provide awareness and
insight into software process activities at an appropriate
level of abstraction and in a timely manner.
Senior management will review the following software

project issues:
C The progress and status of activities to develop

and improve the software process are reviewed
against the project plan.
C Conflicts and issues not resolved at lower

levels are reviewed. Action items are assigned,
reviewed, and tracked to closure (closed-loop
methodology).
CSQE 2002

4. PROJECT REVIEWS
Project Status Reviews

Project management oversight is expected to be at a
more detailed level than that of senior management,
reflecting project managements more active
involvement in the operational aspects of the project.
Examples of performance topics include:
C Status reporting
C Progress reporting
C Forecasting
C Progress status reporting
Closed Loop Methodologies

Project integration management includes processes
required to ensure that various elements of the project
are properly coordinated. Outputs from integrated
change control, which ensure a closed loop
methodology are:
C Project plan updates.

C Corrective action plans
C Lessons learned
CSQE 2002

C. RISK MANAGEMENT
1. RISK MANAGEMENT PLANNING METHODS
Risk Management
Risk Management is presented in the following

topic areas:
C Risk Management Planning Methods

C Risk Probability
C Product Release Decisions
C Software Security, Safety, and Hazard
Analysis Issues
CSQE 2002

C. RISK MANAGEMENT
Risk Management Planning Methods

Software development is one of the most risk prone
management challenges of this decade. Risk factors
can negatively impact the development process and, if
neglected, can lead to project failure. To counteract
these factors, software risk must be actively assessed,
controlled, and reduced on a routine basis.
Risk is defined as the probability of an undesirable

event occurring and the impact of that event occurring.
Failure on a project can be experienced in three ways:
C The product does not meet specified

performance levels
C Actual costs are higher than budgeted
C Delivery of the product is too late
Risks can be segmented into five risk areas:
C Technical risk (performance related)

C Supportability risk (performance related)
C Programmatic risk (environment related)
C Cost risk
C Schedule risk
CSQE 2002

C. RISK MANAGEMENT
Risk Management (Continued)

The management of risk requires taking the specific
actions:
Analyze Plan
Identify Communicate Track
Mitigate Control
Risk Management Continuous Process

CSQE 2002

C. RISK MANAGEMENT
Risk Management (Continued)

For major software developments a risk management
plan is a smart way to guide the risk management
process and to document the results or status of the
risk management process.
The Risk Management Process

CSQE 2002

C. RISK MANAGEMENT
Uncertainties in Quantifying Software Risk

Noise, in the form of uncertainty, is introduced at
several points in the software development process.
Uncertainty in the process is evidenced in many ways.
Some examples are:
C Uncertainty in the product requirements
C Variability in the performance of the assigned

people
C Inaccuracies in the measurement data
C Variation in the judgment of the project

measurement analysts
CSQE 2002

C. RISK MANAGEMENT
2. RISK PROBABILITY
Risk Probability
Risk management is the systematic process of
identifying, analyzing, and responding to project risk
and measuring and managing the probability of risk
occurrence. The purpose of risk management is to
minimize the probability and consequences of adverse
events to project objectives.
Risk Probability Methodology

A risk has three primary components:
C An event (an unwanted change)

C A probability of occurrence of that event
C The impact of that event (amount at stake)
The risk management process includes risk

Identification, risk analysis and a risk response process
in the form of established contingency plans.
Risk Identification
Risk identification involves determining which risks
might affect the project and documenting their
characteristics.
CSQE 2002

C. RISK MANAGEMENT
2. RISK PROBABILITY
Risk Identification (Continued)

The tools and techniques for risk identification include:
C Documentation Reviews
C Information Gathering Techniques
C Interviewing Key Experts
C Assumption Analysis
The risk identification process can take many forms but

brainstorming sessions are most prevalent in the
industry.
Qualitative Risk Analysis

Qualitative risk analysis is the process of assessing the
impact and likelihood of identified risks. Qualitative risk
analysis includes risk prioritization according to the
potential effect on project objectives.
Quantitative risk analysis may employ numerical and

probabilistic analysis.
CSQE 2002

C. RISK MANAGEMENT
2. RISK PROBABILITY
Risk Response - Contingency Plans

Risk response is the process of developing contingency
plans, to reduce threats to the project objectives. Risk
response includes the identification and assignment of
individuals to take responsibility for each critical risk
response in the form of a contingency plan.
Several risk response strategies are possible in

development of contingency plans:
C Avoidance
C Transference
C Mitigation
C Acceptance
Risk can be reduced where a lower risk choice is

available. Selecting the lower risk choice represents a
risk avoidance decision.
Some amount of risk assumption always occurs in

software development projects. It is up to the project
manager to determine the appropriate level of risk that
can safely be assumed in each risk situation.
CSQE 2002

C. RISK MANAGEMENT
3. PRODUCT RELEASE DECISIONS
Product Release Decisions

The best mechanism for dealing with product release
decisions is through an up-front and formalized
specification of product release requirements. A
product release specification document specifies the
requirements for product release.
Product Release Criteria

Examples of product release criteria for release of a
software product are:
C Documented deliverables
C Verified documents (reviewed)
C Testing has been completed at an acceptable
pass rate
C Any prioritized problem/defects are resolved
C Regression testing has been executed
C Release criteria in all planning documents
(SQA plan) has been achieved
C Risks have been identified and mitigated
C All exceptions to plans have been resolved
C Customer is informed of product release status
C A product support plan is in place
C Quality criteria have been met
CSQE 2002

C. RISK MANAGEMENT
3. PRODUCT RELEASE DECISIONS
Bringing a Project Back on Track

There are various ways of bringing a project back on
track when problems occur that affect quality,
scheduling, customer requirements and product
functionality. The most accepted ways are through risk
identification, management analysis, and trade-offs.
Factors Requiring Trade-offs

Factors that require trade-off decisions and analysis
are:
C Schedule C Customer requirements

C Cost C Product functionality
C Quality
The most common situation requiring software product

release trade-offs is schedule compression running
out of time before software development is completed.
Trade-off decisions include:
C Out-sourcing software development

C Phased deliveries of product functionality
C Delivery with known non-critical defects
C Re-negotiating cost or schedule
CSQE 2002

C. RISK MANAGEMENT
4. SECURITY, SAFETY AND HAZARD ANALYSIS
Software Security, Safety, and

Hazard Analysis Issues
Software safety, security and hazard analysis are
software quality assurance activities that should be
performed by the software development and systems
engineering organizations during the requirements
definition, requirements specification and software
design processes.
Activities focus on the identification and assessment of

potential hazards, safety and security issues that may
impact software negatively and cause an entire system
to fail.
CSQE 2002

C. RISK MANAGEMENT
Software Security
Software security consists of the attributes of software
that bear on its ability to prevent unauthorized access,
whether accidental or deliberate, to programs and data.
Common Software Security Faults
Faults that can compromise software security:
C Programmer errors
C Input overflows
C Fault prevention
C Syntax checking flaws
C Using too general a mechanism to accomplish
a function
C Documentation lapses, wrong, missing,
confusing, or faulty
C Lack of security awareness
CSQE 2002

C. RISK MANAGEMENT
Software Security (Continued)

Authentication
To provide security and privacy for a user, systems need

to support a method to determine the identities of two
or more remote parties that wish to exchange
information. This shared process of identification is
called mutual authentication.
Mutual authentication can provide assurance that the

proper users or servers are being contacted by using
one or a combination of the following:
C Claimants knowledge of something (password)

C Claimants possession of something (key)
C Claimant providing evidence of position and
time (selected computer)
C Claimant is verified by a third party
Passwords along with a user-identification code have

been the predominant method of authentication in the
client/server environment.
CSQE 2002

C. RISK MANAGEMENT

Authentication (Continued)
Factors for securing passwords are:
1. Composition the acceptable character

sequence that makes up a password
2. Length the longer the password, or more

characters in the password, the more secure
Additional factors:
1. Lifetime frequent changes in the password
2. Source machine generation or password
3. Ownership passwords not shared by a group
4. Distribution issuing or changing a password

can be a highly sensitive area
5. Storage how password files are protected

CSQE 2002

C. RISK MANAGEMENT

Authentication (Continued)
6. Entry an intruder not being observed, while

physically logging onto the terminal
7. Transmission a password is transmitted for

authentication purposes over a communication
line
8. Authentication Period only authorized

individuals are allowed on an interactive
session via a remote terminal.
No matter how secure a computer system becomes, it

can be broken into, given sufficient resources, time, and
money.
CSQE 2002

C. RISK MANAGEMENT
Software Safety
Software safety is an integral part of the overall system
safety and software development efforts. Software
safety activities take place in every phase of the system
and software development life cycle beginning as early
as the concept phase and on through to operations and
maintenance.
Safety-Critical Software
Safety-critical software is software that:
Monitors the state of hardware components or

exercises direct command and control over the
condition or state of hardware components;
and, if not performed, performed out-of-sequence,

or performed incorrectly could result in improper
control functions or data that results in erroneous
decisions, which could cause a hazard or allow a
hazardous condition to exist.
Firmware contains computer programs and data loaded

in a class of memory that cannot be dynamically
modified by the computer during processing.
CSQE 2002

C. RISK MANAGEMENT
Software Safety (Continued)

Safety-Critical Software (Continued)
IEC 61508 covers all safety-related systems that are

electrotechnical in nature (i.e. electromechanical
systems, solid-state electronic systems and computer-
based systems). The standard is in seven parts.
In every case, the standard applies to the entire

electrical and/or electronic and/or programmable
electronic (E/E/PE) safety-related system. For safety
functions to be effectively specified and implemented,
it is essential to consider the system as a whole.
ISO Standard 1228-1994, establishes a minimum set of

acceptable requirements for the content of a software
safety plan and applies to the software safety plan used
for the development, procurement, maintenance, and
retirement of safety-critical software.
CSQE 2002

C. RISK MANAGEMENT
Hazard Analysis
Critical systems require defensive functions to prevent
unintended functions and to allow operation to continue
despite errors and component failures. From the
system hazard analysis, some software functions may
be identified to prevent hazards or to mitigate the effect
of problems.
Special software functions are often included to detect,

tolerate, override, or recover from failures.
Software Hazard Analysis

A prerequisite for any critical system is an analysis of
the hazards or threats that the system must protect
against. Software hazard analysis is an integral part of
system hazard analysis, and both should be conducted
in order to assure that all hazards have been identified.
Both types of hazard analysis are essential in designing

a system for fail-safe operation.
Analysis techniques are used to assign the severity and

probability of occurrence.
CSQE 2002

C. RISK MANAGEMENT
Software Hazard Analysis (Continued)

HACCP
HACCP (Hazard Analysis and Critical Control Point) is a
pro-active process control system by which food quality
is ensured (and can be applied to software). The
HACCP process consists of:
C Hazard analysis
C Identifying critical control points (CCP)
C Establishing critical limits for each CCP
C Monitoring data to control processes
C Corrective action
C Record keeping
C Verification
Real-time Logic
Real-time logic builds a system model by specifying
events and corresponding actions.
Petri Net Models

Petri net models are used to determine faults that are
most hazardous. Once hazards are identified, software
safety related requirements can be specified.
CSQE 2002

C. RISK MANAGEMENT
Software Hazard Analysis (Continued)

Fault Tree Analysis
Fault tree analysis builds a graphic model of the

sequential and concurrent combination of events that
can lead to a hazardous event. Using a fault tree it is
possible to observe the consequences of a sequence of
interrelated failures that occur in system components.
COMPUTER
FAILS
TO WORK
OR
HARD
.0025 CPU KEYBOARD MONITOR
DRIVE
CALCULATED
.10 .15 .20 .15
AND
A Hypothetical FTA Schematic

DISK DISK
DRIVE A DRIVE B
.05 .05
CSQE 2002

QUESTIONS
6.5. What is the major reason to do a thorough job of

estimating the size of a software development project?
a. To obtain a good estimate of project costs

b. To provide valuable information to management
c. To determine a feel for the complexity of the project
d. To have concrete information to respond to the
market
6.6. The earned value for a software project is considered to

be:
a. The physical work scheduled in money units

b. The physical work accomplished in money units
c. The actual cost incurred
d. The difference between planned and actual costs
6.7. Which of the following is the best description of project

status accounting?
a. Information on the safety and hazard project risks

b. A description of what the project team has
accomplished
c. A prediction of future project status and progress
d. A description of where a project stands, including
cost and budget
Answers 6.5 a, 6.6 b, 6.7 d

CSQE 2002

QUESTIONS
6.15.Software project plan updates should be:
I. Reviewed at the upper management level

II. Maintained under CM control
III. Listed as a management report
IV. Retained as a lessons-learned document
a. I, II, III and IV c. IV only

b. I, II and IV only d. II and III only
6.19.The use of Program Evaluation and Review Technique

(PERT) requires:
a. The critical path to be known in advance

b. Slack times to be added to the critical path
c. Time estimates for each activity in the network
d. Less data than a Gantt chart
6.21.Work breakdown structure means:
a. All potential development activities have been

crashed
b. That either a PERT or CPM chart has been developed
c. The project plan received the support of upper
management
d. The statement of work is divided into a detailed
listing of activities
Answers 6.15 b, 6.19 c, 6.21 d

CSQE 2002

QUESTIONS
6.22.The critical path in a project means that:

I. The project is important to the profits of the
organization
II. Slack times can be used to delay the ending date of
project
III. Events on this path have no slack time
IV. Delays of events on this path delay the ending date of
the project
a. I and IV only c. III and IV only

b. II and IV only d. II, III and IV only
6.23.Advantages of Gantt charts include:

I. The charts are easy to understand
II. The details of activities are easily displayed
III. Each bar represents a single activity
IV. Time estimates of optimistic, most likely, and
pessimistic for each are included
a. I and II only c. I, II and IV only

b. I and III only d. III and IV only
6.30.A software program managers PERT chart should assist

in determining:
a. The time baselines were established

b. Cost of baseline development
c. Task dependencies and timing criteria
d. Description of each software change
Answers 6.22 c, 6.23 b, 6.30 c

CSQE 2002

QUESTIONS
6.34.Providing for requirements traceability assures the

customer that:
a. The requirements were understood

b. Testing will uncover a majority of the defects
c. All items requested have been accounted for and
tracked to completion
d. Marketing has truthfully portrayed the products
capability
6.41.Which of the following would have the lowest priority

when considering software project planning?
a. Reviewing project controls

b. Maintaining contractually predetermined standards
c. Establishing a quality improvement program and
corrective action system
d. Prioritizing schedule commitments
6.42.Some of the hardware cost driver attributes considered in

the COCOMO model are:
I. Run-time performance constraints

II. Memory constraints
III. Required turnaround time
IV. Volatility of the virtual machine environment
a. I and III only c. I, III and IV only

b. I, II and III only d. I, II, III and IV
Answers 6.34 c, 6.41 d, 6.42 d

CSQE 2002

QUESTIONS
6.47.Consider the following software development path:
If path D F were crashed to 4 days and path B F

were crashed to 7 days what would be the minimum
expected project completion time?
a. 20 days c. 22 days
b. 21 days d. 23 days
6.48.Which of the following statements about software project

management is most valid?
a. All projects must start with firm requirements

b. The most difficult problems are technical
c. The most significant improvements begin with upper
management
d. Cost estimates are the most difficult tasks
Answers 6.47 b, 6.48 c

CSQE 2002

QUESTIONS
6.52.The quality review of a contract is aimed at:
a. Revealing legal terminology that could impact

schedules
b. Assuring management that none of the clauses
require quality initiatives
c. Defining the extent of oversight the customer will
employ
d. Identifying and complying with the standards
imposed
6.53.Which of the following component metrics are used to

determine function points, which are used to make project
estimates?
I. External inquiries
II. Algorithms
III. Internal files
IV. External inputs
a. II and II only c. I, III and IV only

b. I and IV only d. I, II, III and IV
6.55.Which of the following reports or techniques would be the

most logical source of quality cost information?
a. Function point estimations

b. Inspection reports
c. Code generation metrics
d. Project planning methods
Answers 6.52 d, 6.53 c, 6.55 b

QUALITY COUNCIL OF INDIANA VII-1 (370)
CSQE 2002
VII. SOFTWARE METRICS AND MEASUREMENT
IF YOU CANT MEASURE IT

YOU CANT MANAGE IT.
Anonymous
CSQE 2002

INTRODUCTION
Introduction
Software Metrics and Measurement is presented in

C Introduction
C Metrics and Measurement Theory
C Process and Product Measurement
C Analytical Techniques
CSQE 2002

INTRODUCTION
Introduction (Continued)
Application of quantitative methods to software
development techniques include:
C Code Complexity Measures - e.g. McCabes

Cyclomatic Complexity Measures and
Halsteads Software Science.
C Software Project Cost Estimation - based on

Lines of Code (LOC), examples include
Putnams SLIM Model and Barry Boehms
COCOMO Model.
C Software Quality Assurance Techniques were

significantly improved.
C Software Development Processes - were

improved by defining the life cycle as finite
sequential phases and emphasis on project
management.
CSQE 2002

A. METRICS AND MEASUREMENT THEORY
Metrics and Measurement Theory
Metrics and Measurement Theory is presented in

C Definitions
C Basic Measurement Theory
C Psychology of Metrics
CSQE 2002

1. DEFINITIONS
Measurement Theory Definitions

C Entity - An object or event. Software resource
entities include all of the inputs used for
software production.
C Attribute - A feature or property of an entity, for

example amount of code inspected, number of
defects found, and duration of the inspection.
C Measurement - The process by which numbers

or symbols are assigned to attributes of entities
to describe them according to defined rules.
C Primitive - Data that is quantitative descriptions

of software. Primitives are directly measurable,
for example errors, failures, faults, times.
C Measure - Quantitative assessment of the

degree to which a software product or process
possesses a given attribute. Measures are
calculated using primitives, for example
include defect density, cyclomatic complexity,
test execution rate, and test pass rate.
CSQE 2002

1. DEFINITIONS
Reliability
Reliability refers to the consistency of a number of
measurements taken of a metric using the same
measurement method on the same subject.
If repeated measurements are highly consistent or even

identical, then there is a high degree of reliability with
the measurement method or the operational definition.
If the variations among repeated measurements are

large, then reliability is low.
IV equals the Index of Variation, the lower the IV the

more reliable the measurements.
CSQE 2002

1. DEFINITIONS
Validity
Validity relates to accuracy, or the truth.
Software validity is proof that the software is trouble

free and that it functions correctly.
Predictive validity refers to the accuracy of the model

estimates. To achieve predictive validity the input data
must be accurate and reliable. Predictive validity also
requires that model estimates and actual outcomes be
compared empirically.
CSQE 2002

1. DEFINITIONS
Measurement Errors
C Systematic measurement error is associated
with validity.
C Random measurement error is associated with
reliability.
Representational Condition
Representational theory of measurement is based on
sets, relations, axioms and functions. The empirical
relation system C is a set which consists of the set of
entities C related by the set of empirical relations R.
C = (C, R )
For example, a relational system consists of the

classification of three types of software failures. The
relation system (C, R ) where R = three unary relations
R1, R2 and R3 for every x0C (x is an element of C) an
empirical relationship C exists.
If R1 is less critical than R2 which is less critical than R3

etc., R1 = 1, R2 = 2, R3 = 3, satisfies the representational
condition.
CSQE 2002

1. DEFINITIONS
Metrics Definitions
Software metrics. An approach to measuring some
attribute of software products or processes.
Defect. A product anomaly, omissions and

imperfections, faults contained in software.
Error. Human action that results in software containing

a fault.
Failure. (1) The termination of the ability of a functional

unit to perform its required function. (2) An event in
which a system or system component does not perform
a required function within specified limits.
Fault (bug). (1) An accidental condition that causes a

functional unit to fail to perform its required function.
(2) A manifestation of an error in software.
KCSI - Thousand code source instructions.
KLOC - Thousand lines of code.
KSLOC - Thousand source lines of code.

CSQE 2002

2. MEASUREMENT THEORY
Measurement Scales
Nominal Scale: This is the lowest level of measurement.
Data is assorted into groups such as categories.
Ordinal Scale (Ranking): This is the second tier. Data

can be classified or arranged into some logical order
but absolute differences are not possible. Includes
Likert surveys.
Interval Scale: This higher order scale allows the

determination of differences, however there is a lack of
an absolute zero starting point. Fahrenheit and
centigrade scales are examples.
Ratio Scale: Almost all integer scales are also ratio

scales. The number of lines of code written or the
difference between defects per product build are ratio
measurements.
CSQE 2002

Measurement Scales (Continued)
Scale Description
Nominal Data consists of names or categories
only. No ordering scheme is possible.
Ordinal Data is arranged in some order but
(Ranking) differences between values cannot be
determined or are meaningless.
Interval Data is arranged in order and
differences can be found. However,
there is no inherent starting point and
ratios are meaningless.
Ratio An extension of the interval level that
includes an inherent zero starting point.
Both differences and ratios are
meaningful.
The Four Measurement Levels

CSQE 2002

Central Limit Theorem

If a random variable X has mean and finite variance
6 approaches a normal distribution
Fx2, as n increases, X
with mean and variance .
The distribution of sample means approaches a normal

distribution, regardless of the shape of the parent
population.
Distributions of Individuals Versus Means

CSQE 2002

The Central Limit Theorem States:
C The sample means (X 6 s) will be more normally

distributed around : than individual readings (Xs).
The distribution of sample means approaches
normal regardless of the shape of the parent
population.
C For normal distributions, the spread in sample

means (X 6 s) is less than Xs with the standard
deviation of X6 s equal to the standard deviation of
the population (individuals) divided by the square
root of the sample size and is referred to as the
standard error of the mean, SX6 :
CSQE 2002

Measures of Central Tendency

6)
The Mean (X-bar, X
The Mode
The mode is the most frequently occurring number in a

data set.
The Median (Midpoint)
The median is the middle value when the data is

arranged in ascending or descending order. For an
even set of data, the median is the average of the middle
two values.
Central Tendency in Normal and Skewed Distributions

CSQE 2002

Measures of Dispersion
Range (R)
The range of a set of data is the difference between the

largest and smallest values.
Variance (F2, S2)
Standard Deviation (F, s)
Coefficient of Variation (COV)

CSQE 2002

3. PSYCHOLOGY OF METRICS
Psychology of Metrics
In setting up a measurement program, one must be
aware of potential human problems and how to
overcome them.
The most common cause of complaint is where

measures are gathered for one specific agreed
objective, and are used for a different non-agreed
objective.
In general, people do not like being measured.

Measurement should not be used to assess the
individual, but to assess the process. If people are
confident about this intent, they will not waste time
manipulating results.
A commonly observed human phenomenon is the

Hawthorne effect, where the act of measurement can
lead to improvements in quality or productivity, because
people perceive that their efforts are somehow
warranting special attention through the act of
observation (in data collection).
CSQE 2002

3. PSYCHOLOGY OF METRICS
Human Errors
Rounding: This involves the discard of some test
measurement accuracy.
Pencil Whipping: This indicates the faking or

falsification of data.
Pressure: On occasion, an individual will yield to a

strong personality or the need for delivery performance.
Flinching: Some Individuals tend to give a reading the

benefit of doubt by rounding to the next decimal inside
the specification.
Inadvertent Errors: These errors are sporadic in nature

and are difficult to avoid.
Technique Errors: These errors may indicate lack of

training, lack of skill, or lack of capacity.
CSQE 2002

B. PROCESS AND PRODUCT MEASUREMENT
1. PROCESS AND PRODUCT METRICS
Process and Product Measurement
Process and Product Measurement is presented in

C Process, Product, and Resource Metrics

C Commonly Used Metrics
C Software Quality Attributes
C Defect Detection Effectiveness Measures
C Program Performance and Process
Effectiveness
CSQE 2002

Goal/Question/Metric Paradigm
One such paradigm for establishing a software project
metrics program is the Goal/Question/Metric (GQM)
paradigm.
This paradigm consists of identifying goals, from these,

questions are formulated in quantifiable terms, and
finally metrics are established to verify them. Any
software measurement activity should be preceded by
the identification of a software engineering goal. This
goal leads to a question, which leads to metrics.
EVALUATE A NEW
PROGRAMMING METHOD
WHAT ARE THE WHAT ARE

WHO IS USING
METHOD THE METHOD
THE METHOD?
ADVANTAGES? FEATURES?
PROPORTION PROPORTION BETTER LESS

LESS
OF OF FUNCTION SOFTWARE
EFFORT
PROGRAMMERS COMPANIES POINTS EXPENSE
GQM Example
CSQE 2002

Goal/Question/Metric Paradigm (Cont.)

To aid in goal definition, the following set of templates
is provided:
C Purpose: The purpose may be to characterize,

evaluate, predict, motivate, etc., the product or
process. This is done in order to: understand,
assess, manage, engineer, improve, etc.
C Perspective: Look at the goal from another

viewpoint. For example, examine cost
effectiveness, defect, changes, product
measures, etc. from managements viewpoint.
C Environment: The environment consists of

process, people, methods, tools, constraints,
etc.
CSQE 2002

Process, Product, and Resource Metrics

Product Measures: Product measures describe the
characteristics of the product, such as size, complexity,
design features, performance and quality levels.
Process Measures: Processes are numerous software

related activities. Process measures often have time
oriented attributes, include defect levels or cost
elements.
Resource Measures: There are widely divergent

measures to track resources. Examples include
training, costs, speed and ergonomic data.
CSQE 2002

Considerations in Designing Measures

Assessment and Prediction Measures
Measures used for assessment help keep track of a

particular software project and assess test coverage,
productivity, and defect removal efficiency.
Predictive attributes or estimation, such as cost and

effort of processes, and reliability and maintainability of
products, are crucial to the success of software
development efforts.
Using Software Models to Design Metrics
A model is an abstract representation of a given object,

and are needed to define measures unambiguously and
to design project measures.
Models are used for assessing productivity of

personnel, verifying the effectiveness of defect
detection and removal activities, and prediction of cost,
resources and the duration of a project.
CSQE 2002

2. COMMONLY USED METRICS
Commonly Used Metrics

Direct metric: A metric applied during development or
during operations that represents a software quality
factor (for example, mean time to software failure for the
factor, reliability).
Software Quality Metrics Methodology

The IEEE Standard for a Software Quality Metrics
Methodology states that establishing a quality metrics
methodology for software is accomplished through a
systematic approach and is comprised of the following
five steps:
C Establish software quality requirements
C Identify software quality metrics
C Implement the software quality metrics
C Analyze the software quality metrics results
C Validate the software quality metrics

CSQE 2002

Software Quality Metrics Examples

Some software quality metrics are listed below:
Size - Lines of Code
Normally used as a key factor in evaluating effort,

functionality and complexity.
DeMarco - Bang
A computation of functionality.
Halstead - Software Science
Measures properties and structure of computer

programs in order to predict attributes of programs:
C Length, volume, difficulty

C Number of errors
C Effort and time required for development
CSQE 2002

Halstead - Software Science
Software Science proposes the first analytical laws for

the development of computer software. Halsteads
theory is derived from his fundamental assumption that
the human brain follows a fairly rigid set of rules in
developing algorithms. The premise is that any
programming task consists of selecting and arranging
a finite number of program tokens, which are basic
syntactic units distinguishable by a compiler.
Software science uses a set of primitive measures as

listed below:
n1 = the number of distinct operators that

appear in a program
n2 = the number of distinct operands that
appear in a program
N1 = the total number of operator occurrences
N2 = the total number of operand occurrences
The overall program length:

L = N1 + N2 = n1 log2 (n1) + n2 log2(n2)
The actual program volume:

V = N log2 (n) = N log2 (n1 + n2)
CSQE 2002

3. SOFTWARE QUALITY ATTRIBUTES
Software Quality Attributes

Software quality attributes or factors relate to those
aspects of quality that interest a particular set of users
and reflects their particular concerns.
Quality can be defined as: The degree to which a

system, component, or process meets customer or user
needs. Software quality factors (SQFs) are attributes
that a specific group of users believe a specific software
system should possess. That is, a SQF is a user-
oriented view of an aspect of software product quality.
One or more sets of SQFs may constitute a subset of

the requirements for a software product. Any such
requirements should be quantifiable. Various SQFs do
not appear to be quantifiable. McCall identified a
number of SQFs. A number of these SQFs are
associated with one or more metrics.
CSQE 2002

Software Quality Attributes (Continued)

SQA Definition Metric
Correctness The extent a program satisfies Faults/ Lines of
its specifications and fulfills code
the users objectives.
Efficiency The amount of computing Actual or
resources required. Allocated
utilization
Flexibility The effort required to modify Average labor
a program. days to change
Integrity Control of access to software Faults/ Lines
or data by unauthorized
persons.
Inter-operability Effort required to couple one Effort to couple
system with another.
Maintainability Effort required to locate and Average labor
fix a defect in an operational days to fix
program.
Portability Effort required to transfer a Effort to
program from one hardware transport or to
configuration and/or develop
environment to another.
Reliability Extent to which a program Faults/Lines of
can be expected to perform code
its intended function.
Definitions and Metrics for Software Quality Attributes

CSQE 2002

Software Quality Attributes (Continued)

SQA Definition Metric
Verifiability Degree to which a system can be Number of
evaluated to determine whether the requirements
products satisfy the conditions. implementable in
the design
Usability Ease with which a user can learn Number of errors
to operate and interpret a system made by users in a
or component. given time period
Reusability Degree to which a software Resources saved
module can be used in more than using existing
one program or system. software assets
Testability Degree to which a system or Number of
component facilitates the requirements with
establishment of test criteria or test criteria.
permits performance of tests.
Expandability Ease of modification of a system or Spare capacity
or component to increase functional available in a time
Extendability capacity. period.
Performance Degree to which a system or Transactions
component accomplishes its processed per
designated functions. second
Robustness Degree to which a system or Time to re-start after
component functions correctly system failure.
with invalid inputs.
Traceability Degree to which a relationship can Requirements
be established between two or included in a design
more development.
CSQE 2002

Testing
Unit test - The developer exercises the code to see if it
has been properly implemented through some low level
tests of its capability.
Integration test - Individually tested modules are verified

that their interaction is complete and supports the
overall objectives of the requirements.
System test - Product elements are exercised in a real-

world environment. When the system is extremely
large, some aspects of the system are simulated
through an approximation of the actual environment.
Regression tests - Verify the code produced is

compatible with earlier versions. This form of stress
testing provides a level of confidence that the system
will function with the newer requirements.
Beta testing - A pre-release version of the software is

provided to the user community and all bugs are
reported.
CSQE 2002

Software Reliability
One definition for reliability is the probability that a
system will not fail for a stated length of time, starting
from some specific time. In actuality, for a period of
time after software is placed in operation, there is often
a surge of defect discovery. Software defects
include errors or faults. Achieving 100% software
reliability requires software to be free of errors or faults.
In the context of modeling, (e.g. Rayleigh Model)

reliability refers to the degree of change in model output
due to change fluctuations in the input data. In
statistical terms, reliability is related to the confidence
interval of the estimate; the narrower the confidence
interval, the more reliable the estimate.
The confidence interval, in turn, is related to the sample

size; larger samples yield narrower confidence intervals.
Therefore, for the Rayleigh Model, which is implemented
on a six phase development process, the chance of
having a satisfactory confidence interval is very slim.
CSQE 2002

The Rayleigh Model

Reliability models can be classified into two categories:
static models and dynamic models.
A static model uses other attributes of the project or

program modules to estimate the number of defects in
the software.
A dynamic model, usually based on statistical

distributions, uses the current development defect
patterns to estimate end-product reliability. The
Rayleigh model is a dynamic software reliability model.
The Rayleigh Model - Development Stage

CSQE 2002

Exponential Model
The exponential distribution in software is best used for
modeling the defect arrival pattern at the back-end of
the process (for example the final formal test phase).
Exponential Distribution
Other Reliability Growth Models

More than one hundred additional models have been
proposed, each having its own assumptions,
applicability, and limitations.
CSQE 2002

Other Reliability Growth Models (Cont.)

C Time Between Failure Models
C Fault Count Models
C Jelinski-Moranda (J-M) Model
C Littlewood (LW) Models
C Goel-Okumoto (G-O) Imperfect Debugging

Model
C Goel-Okumoto (G-O) Nonhomogeneous

Poisson Process Model (NHPP)
C Musa-Okumoto Logarithmic Poisson Execution

Time Model
C The Delayed S and Inflection S Models

CSQE 2002

4. DEFECT DETECTION EFFECTIVENESS
Defect Removal Effectiveness

Defect removal is one of the top expenses in any
software project, and critically affects project
scheduling. Therefore, it is important to measure the
effectiveness of the defect removal processes, such as
inspections and testing.
Fagan defined error detection effectiveness as:
Jones expanded this measurement for removal

efficiency:
CSQE 2002

Defect Removal Effectiveness (Cont.)

Phase Containment
Development Defect Injection Defect Removal

Phase
Requirements Required gathering process Requirements
and development of analysis and review
requirements specifications
High Level Design work High level design
Design inspection
Low Level Design work Low level design
Design inspection
Code Coding Code inspection
Implementation
Unit Test Bad bug fixes Unit testing
Integration Bad bug fixes Integration testing
System Test Bad bug fixes System testing
Defect Injection and Removal Phases
Defect removal effectiveness (also called phase

containment effectiveness) can be calculated as:
Defects removed (at the step) x 100%
Defects existing on step entry + Defects injected during the development step
CSQE 2002

Defect Removal Effectiveness (Cont.)

Cost Effectiveness
Defect removal at earlier development phases is

generally less expensive.
Yield Effectiveness
Defect removal effectiveness and defect removal

models are useful quality planning and management
models. However, they are not predictive models.
Software reliability models are used to estimate the

reliability or the number of latent defects of the software
product when it is available to the customers.
Customer Impact
Organizations need to assess the effectiveness of

defect detection and removal activities and the impact
on customers.
CSQE 2002

5. PROGRAM PERFORMANCE
Program Performance Analysis

Performance analysis determines if the program or
project is meeting intended plans and targets. The goal
of performance analysis is to provide information for
making decisions in time to affect the product outcome.
Performance analysis consists of six steps:
C Compare plan against actuals

C Predict the outcome by extrapolating
C Assess the impact of the problem or risk
C Identify and evaluate alternatives
C Make a decision
C Monitor trends for possible changes
CSQE 2002

Program Performance Indicators

Indicators Based on Trends
Trend-based performance indicators are used when the

expected or planned value changes regularly over time.
Performance analysis of a trend-based indicator
determines if the actual project performance
corresponds to the plan.
Trend-based indicators may detect problems via visual

clues, including:
C Unusual individual results
C Constant variance of actual performance from

the planned baseline
C Increasing variance between actual

performance and the planned baseline
CSQE 2002

Program Performance Indicators (Cont.)

Indicators Based on Thresholds or Targets
Performance indicators, based on thresholds or targets,

are used when the expected value remains relatively
constant over time. Performance analysis determines
whether the actual project performance meets or
exceeds its established bounds.
There are three approaches for setting thresholds:
C Arbitrary
C Management reserve
C Statistical limits
Thresholds are signals to alert management to

investigate and take possible action. Falling outside a
threshold indicates that work will not finish on schedule
unless requirements or processes change.
CSQE 2002

Process Effectiveness
ISO 9000:2000 defines effectiveness as the extent to
which planned activities are realized and planned
results achieved.
If a process is a set of interrelated or interacting

activities, which transforms input into outputs, then
process effectiveness is the extent to which processes
are realized (occur) and outputs achieve expectations.
Measuring process results is essential to assess the

contribution of the process to business and project
goals.
Process Capability
The SEI-CMM describes five maturity levels with each

maturity level indicating a level of process capability.
The software process capability of an organization

provides another means of predicting the most likely
outcomes from the next software organizational project.
CSQE 2002

C. ANALYTICAL TECHNIQUES
1. DATA INTEGRITY
Analytical Techniques is presented in the following

topic areas:
C Data Integrity
C Quality Tools
C Sampling Theory and Techniques

CSQE 2002

1. DATA INTEGRITY
Correct Data
Data serves as the basis for action. Basili and Weiss
proposed a data collection methodology that could be
applicable anywhere. Their scheme consists of six
steps with considerable feedback and iteration
occurring at several phases:
1. Establish the goal of the data collection

2. Develop a list of questions of interest
3. Establish data categories
4. Design and test data collection forms
5. Collect and validate the data
6. Analyze the data
Data Quality
A fundamental principle of data quality is that data
should be right the first time, which means that the
responsibility for getting the data right is at the point at
which it is recorded.
CSQE 2002

1. DATA INTEGRITY
Software Data Errors

Software errors are found to fit into seven general types:
1. Calculation Error
2. Blank Field
3. Transfer of Data Between Projects Incorrect
4. Entry Error
5. Transfer of Data Within Project Incorrect
6. Impossible Values
7. Process Sequence not Followed

CSQE 2002

1. DATA INTEGRITY
Data Management
It is important to look at data integrity from a data
management perspective.
Perhaps the greatest threat is the staggering complexity

of modern computer systems and the vast number of
things that happen inside them. One way to
counterbalance this threat is with active data
management based on the following principle data
must be actively preserved or it will (passively) decay.
Data can be preserved by active data management that

includes:
C Regular backups
C Offsite backups
C Testing of backups
C Virus protection
C Good security
C Care in the manipulation of files
C Data integrity monitoring
CSQE 2002

1. DATA INTEGRITY
Example of a Data Quality Issue

Most databases contain multiple variations of the same
city, company, customer, or address, including multiple
abbreviations and typographical errors. In the example
below, the city of Washington Heights is represented
by six variations:
1. Wash Hgts. 4. Washngtn Heights

2. Washington Hts 5. Wasshington Hights
3. Washington Hts. 6. Washington Heights
If the database is queried for all Customers from

Washington Heights the results, returned from the
database, will be substantially inaccurate.
Example of a Data Standardization Error

C President C CEO
C Owner C C.E.O.
C Chief Executive Officer C President/Owner
For a query Job title = President only the customers

where President is in the title field will be selected
and the results will not be accurate.
CSQE 2002

2. QUALITY TOOLS
Quality Analysis Tools

The basic statistical tools for quality control have
become an integral part of the quality control literature,
and have been known as Ishikawas seven basic tools.
CSQE 2002

2. QUALITY TOOLS
Quality Analysis Tools (Continued)

Check Sheets
Check sheets or checklists can be used to ensure all

tasks are complete, and that for each task, the important
factors or quality characteristic are covered.
Pareto Diagram
Pareto analysis helps by identifying focus areas that

cause most of the problems. Defects tend to cluster
into a minor number of components.
A = logic errors
B = initialization of
parameters
C = complexity
D = data definition
problems
E = wrong syntax
F = scope of
variables
G = interface
specification
problems
CSQE 2002

2. QUALITY TOOLS

Histogram
Histograms may also be used for software projects and

quality management. A severity histogram can show
more about the product than simply a defect rate.
Graphs
Graphs serve as real-time statements of quality and

workload. Pie charts and bar charts can graphically
display defect data, programming man hours,
component failure, etc.
CSQE 2002

2. QUALITY TOOLS

Scatter Diagram
The relationship between a complexity index (such as

McCabes) and defect level is shown in the Figure.
CSQE 2002

2. QUALITY TOOLS
Control Charts
The control chart is a powerful tool for achieving

statistical process control (SPC). Control charts are
useful for software quality improvement when they are
used not in terms of formal statistical process control
and process capability, rather, they are used as a tool
for improving consistency and stability.
Various metrics from the software development process

can be control charted, for instance, inspection defects
per KLOC, testing defects per KLOC, and phase
effectiveness.
Chart of Component Failures/KLOC

CSQE 2002

2. QUALITY TOOLS

Cause-And-Effect Diagram
The cause-and-effect diagram is a type of fishbone

diagram used in quality improvement efforts for guiding
brain storming efforts. In reading the fishbone, the
effects (root causes) are listed on top, while the
causes, are identified on arrows which radiate out from
the spine.
Poor Insufficient Poor
Planning Preparation Overview
Lack of Incomplete Graphs not

resources documents available
No traceability Lack of
Insufficient time
Prep time
No background
Wrong participants Poor standards
provided
Ineffective
Inspection
All defects
No follow up
not recorded
No process for
No root defect certification
Not trained
cause analysis
Defects not Defect opened

No checklist > 120 days
Categorized
Moderator Inspection
Follow-up
meeting
CSQE 2002

2. QUALITY TOOLS
Flow Charts
A flow chart or process map can depict the sequence of
product, containers, paperwork, operator actions or
administrative procedures. A flow chart is often the
starting point for process improvement.
Common Flow Chart Symbols

CSQE 2002

2. QUALITY TOOLS
Flow Chart Example

CSQE 2002

2. QUALITY TOOLS
Problem-Solving Process
Problem solving is important for process improvement.
Tools, when applied correctly, help groups work

through problems quickly and productively.
A logical problem-solving process:
C Clearly state the problem - in the customers

terms
C Generate a list of root causes
C Prioritize root causes
C Develop solutions to address the largest

causes
C Prioritize solutions
Deploy the best solutions and measure the gains.

CSQE 2002

2. QUALITY TOOLS
Root Cause Analysis

A technique used for establishing root causes, called
repeated questioning, relies on repeatedly asking a
series of simple questions about a problem that has
been uncovered. The questions are simple and direct:
C When was the problem found?
C Where could the problem have been found?
C Where should the problem have been found?
When permanent corrective action is proposed,

management must determine if:
C The root cause analysis has identified the full

extent of the problem
C The corrective action is satisfactory to

eliminate or prevent recurrence
C The corrective action is realistic and

maintainable
CSQE 2002

2. QUALITY TOOLS
Plan - Do - Check - Act

The Shewhart Cycle Plan/Do/Check/Act (PDCA) is often
called the Deming Cycle Plan/Do/Study/Act (PDSA)
because W. Edwards Deming proposed it for process
improvement in Japan. Although he gave credit to
Walter Shewhart, Demings name became permanently
attached to the improvement technique.
The Deming improvement cycle is a continuous

improvement loop which is normally used by a team.
The objective is to improve the input and the output of
any stage.
CSQE 2002

2. QUALITY TOOLS
Management and Planning Tools

The 7 new tools as written by Japanese authors Mizuno
Shigeru; and Asaka Tetsuichi and Ozeki Kazuo are:
1. Relations diagram
2. Affinity diagram (KJ method)
3. Systematic diagram
4. Matrix diagram
5. Matrix data analysis
6. Process decision program chart (PDPC)
7. Arrow diagram
Goal/QPC modified the 7 new QC tools to a similar set

of 7 management and planning tools. They are
identified by the corresponding Japanese sequence
number:
2. Affinity diagram (KJ method)

3. Tree diagram
6. Process decision program chart (PDPC)
5. Matrix diagram
1. Interrelationship digraph (I.D.)
4. Prioritization matrices
7. Activity network diagram
CSQE 2002

2. QUALITY TOOLS
Affinity Diagrams
The affinity diagram method can be used for problem
solving and is similar to the mind mapping technique.
One generates ideas that link up to other ideas to form
patterns of thoughts.
The affinity diagram uses an organized technique to

gather facts and ideas to form developed patterns of
thought. It can be widely used in the planning stages of
a problem to organize the ideas and information.
The affinity diagram can also be referred to as the KJ

method developed by Dr. Kawakita Jiro.
The steps can be organized as follows:
C Define the problem under consideration.

C Have 3" x 5" cards for use.
C Enter ideas, data, facts, opinions on the cards.
C Place the cards on a wall.
C Arrange the notes into similar thought patterns.
C Develop a main category for each group of
cards which becomes the affinity card.
C Borders can be drawn around groups.
CSQE 2002

2. QUALITY TOOLS
Example Affinity Diagram
GET CSQE PRIMER

WATCH VIDEO PRESENTATION
REVIEW OTHER
BOOKS TAKE CSQE
SEMINARS
GET OTHER ATTEND CSQE REFRESHER
TEXTBOOKS
CALL ASQ TO OBTAIN STUDY IN GROUPS HAVE A TUTOR

BODY OF KNOWLEDGE
CONSIDER A CSQE
TRAINING COURSE
HAVE A Q & A SOURCE

TEACH CSQE SUBJECTS
DEVELOP PRACTICAL
EXPERIENCE
STUDY INTENSIVELY MOTIVATE SELF GET BONUS
START EARLY 1-2 YEARS LISTEN TO SUCCESSFUL

PASSED CSQEs
STUDY 1 SUBJECT
AT A TIME FOR
3 - 4 WEEKS BE AROUND OTHERS
DEVELOP WHO ARE POSITIVE
STUDY CSQE TESTS PRIDE
PUMP YOURSELF UP
MAKE YOUR OWN
CSQE EXAMS
CSQE 2002

2. QUALITY TOOLS
Interrelationship Digraphs (I.D.)

The idea is to have a process of creative problem
solving that will eventually indicate some key causes.
In fact, the final solution to the problem will be
determined when the team has analyzed the graph for
the key causes.
The interrelationship digraph can also be referred to as

a relations diagram.
The interrelationship digraph can be created by:
C Develop about 50 items that pertain to the problem.

C Use a pattern of placing closely related items or for
a random display.
C Draw relationship arrows from causes to effects.
C Several revisions can be made.
C After revision an analysis can be attempted.
C Items that need to be worked on right away should
possibly be the ones with the greatest number of
arrows.
CSQE 2002

2. QUALITY TOOLS
Example Interrelationship Digraph

BONUS GET TAKE ASQ HAVE
CSQE CSQE A
FOR
PRIMER WORKSHOP TUTOR
CSQE
ATTEND HOW TO PASS THE STUDY IN

CSQE
GROUPS
REFRESHER CSQE EXAM
PEERS
HAVE
CSQE
CALL
ASQ HAVE A
CALL-IN
MOTIVATION SOURCE
OF SELF
STUDY
JOB CSQE TESTS TAKE
EVALUATION
UNIVERSITY
NEEDS
CSQE COURSES
PROMOTION STUDY
REQUIRES INTENSIVELY
CSQE
CSQE 2002

2. QUALITY TOOLS
Tree Diagrams
The tree diagram is a systematic method to outline all
the details needed to complete a given objective. The
tree diagram can also be referred to as a systematic
diagram. The organization is by levels of importance
(i.e., why - how, goals - means).
C Determine the overall objective or goal and

place it on the far left side of the board.
C Determine the second level of means that

would achieve the goal, or how can you
achieve the why card to the left.
C For each level of the tree, the same line of

questioning is used, until a final level is
achieved. The final level occurs when you have
determined that all details necessary to solve
the overall objective.
C Go over the diagram to confirm that each

means will lead to a successful objective. If so,
your tree is complete.
CSQE 2002

2. QUALITY TOOLS
Example Tree Diagram

CSQE 2002

2. QUALITY TOOLS
Prioritization Matrices
To use the prioritization matrices, the key issues and
concerns are identified and alternatives generated. The
prioritization matrix is a system for decision making.
M. Brassard provides a description of 3 types of

prioritization matrices that can be developed for use:
C The full analytical criteria method
C The consensus criteria method
C The combination I.D./matrix method
The criteria are prioritized, weighted, and applied

against the options generated. A decision based on
numerical values generally can be obtained.
CSQE 2002

2. QUALITY TOOLS
Prioritization Matrices (Continued)

The Criteria Composite Ranking (4 People)
Total
A. Work Experience 0.05 + 0.10 + 0.10 + 0.20 = 0.45
B. Have Tutor 0.10 + 0.20 + 0.30 + 0.10 = 0.70
C. Study In Group 0.15 + 0.10 + 0.05 + 0.20 = 0.50
D. Attend CSQE Refresher 0.25 + 0.20 + 0.20 + 0.30 = 0.95
E. Study Old Tests 0.15 + 0.15 + 0.25 + 0.10 = 0.65
F. High Motivation 0.30 + 0.25 + 0.10 + 0.10 = 0.75
1.00 1.00 1.00 1.00 = 4.00
Completed Rank Order Scores
Criteria 0.45 0.70 0.50 0.95 0.65 0.75

Work Have Study Attend Study High Total
Factors Experience Tutor Group Refresher Tests Motivation
Knowledge and
1(0.45) 2(0.70) 3(0.50) 2(0.95) 1(0.65) 3(0.75) 8.2
Ethics
Quality
2(0.45) 3(0.70) 1(0.50) 3(0.95) 3(0.65) 4(0.75) 11.3
Management
Audits 4(0.45) 1(0.70) 2(0.50) 1(0.95) 2(0.65) 2(0.75) 7.3
Engineering
3(0.45) 4(0.70) 4(0.50) 4(0.95) 5(0.65) 5(0.75) 17.0
Processes
*Program
7(0.45) 7(0.70) 8(0.50) 7(0.95) 6(0.65) 8(0.75) 28.6
Management
*Software
6(0.45) 8(0.70) 7(0.50) 8(0.95) 8(0.65) 7(0.75) 29.9
Metrics
*Verification &
5(0.45) 6(0.70) 5(0.50) 6(0.95) 7(0.65) 6(0.75) 23.7
Validation
Configuration
8(0.45) 5(0.70) 6(0.50) 5(0.95) 4(0.65) 1(0.75) 18.2
Management
CSQE 2002

2. QUALITY TOOLS
Matrix Diagrams
Matrix diagrams show the strength of relationships in a
grid of rows and columns. Basic matrices types:
C L-type...elements on the Y-axis and X-axis

C T-type...2 elements on the Y-axis, a set on the X-axis.
C X-type...2 elements on both the Y-axis and X-axis
C Y-type...2 L-type matrices joined at the Y axis to
produce a matrix design in 3 planes
C C-type (3-d matrix)...2 L-type matrices joined at the Y-
axis, but with only 1 set of relationships indicated
X-Type Matrix
Cause 1
Cause 2
Cause 3
Cause 4
Mach 1 Mach 2 Mach 3 Dept 1 Dept 2 Dept 3 Dept 4 Dept 5
Problem 1
Problem 2
Problem 3
Problem 4
Problem 5
Problem 6
Strong Relationship (3) Relationship (2) Possible (1)
CSQE 2002

2. QUALITY TOOLS
Process Decision Program Charts (PDPC)

The process decision program chart (PDPC) method is
used to chart the course of events that will take us from
a start point to our final complex goal. The various
events are charted and contingencies are planned for.
Of course, some contingencies can not be planned for.
This method is similar to contingency planning.
PDPC uses include new, unique, or complex problems

and the opportunity to create contingencies for
problems.
There are several ways to construct a process decision

program chart (PDPC):
One graphic method is a forward sequence of steps to

show progress toour goal.
Another graphic method can be developed with the goal

as the starting point.
An outline format method can be used to indicate the

levels of the problem. The major steps would make up
the outline.
CSQE 2002

2. QUALITY TOOLS
Process Decision Program Charts (Cont.)

Last Solutions
Major 2nd Last Level to
Categories Level Level "What- ifs" "What- ifs"
RESULT CONTINGENCY
A2 A4
RA4
A3 A5 RA5
A1 CONTINGENCY
START
GOAL
B2 B4 RB4
CONTINGENCY
B1
B3 B5 RB5
CONTINGENCY
CSQE 2002

2. QUALITY TOOLS
Activity Network Diagrams

The arrow diagram is the original Japanese name for
this tool. The activity network diagram describes a
methodology that includes program evaluation and
review techniques (PERT), critical path method (CPM),
node/activity on node diagrams (AON), precedence
diagrams (PDM), and other network diagrams.
Activities, milestones, and critical times must be

developed and then drawn onto a chart. The chart will
then provide a tool to help monitor, schedule, modify,
and review the project.
Event, node: The junction point of an activity
Job, activity: The activity or task
Dummy node: A note inserted to combine the relative

timing of parallel operations
Critical path: The path with the longest time
Slack time (SL): The difference between the latest time

and the earliest finish time
CSQE 2002

2. QUALITY TOOLS
Activity Network Diagrams (Continued)

CSQE 2002

3. SAMPLING THEORY AND TECHNIQUES
Sampling Theory and Techniques

It is important to select a sampling plan appropriate for
the purpose of the use of the data. There are no
standards as to which plan is to be used for data
collection and analysis, therefore the analyst makes a
decision based upon experience and the specific needs.
Advantages of sampling include:
C The number of required testers is relatively small

C There is less disruption of business activities
C Results can be obtained in a short time
C Systemic processes can be evaluated by looking at
a small number of areas
C Opportunities for improvement can be identified
Disadvantages of sampling include:
C The risk that deficiencies may be overlooked

C The risk that deficiencies are exceptional cases
C Corrective action improvement efforts may be
misdirected
C Less organization-wide information is obtained
C No sampling plan can guarantee identification of
deficiencies
CSQE 2002

Sampling Terms
Consumers risk (beta risk) $: The probability of
accepting a bad lot.
Producers risk (alpha risk) ": The probability of

rejecting a good lot.
Sampling errors: One rarely knows whether the product

is good or bad. One only knows what the samples
indicate:
QUALITY
GOOD BAD
CALLED 1-" $
THE PRODUCERS TYPE II ERROR
GOOD
DECISION CONFIDENCE
MADE
" 1- $
CALLED TYPE I ERROR CONSUMERS
BAD CONFIDENCE
CSQE 2002

Acceptance Sampling
Acceptance sampling is typically used for product
testing and acceptance.
Attributes plans - A sample is taken from a lot with each

unit classified as acceptable or defective. The number
of defects is then compared to the acceptance number,
in order to make an accept or reject decision for the lot.
ANSI/ASQC Z1.4 (modernized version of MIL-STD-105E)
is based on high Pas (85-99%) and consists of tables
listing sample size code letters and describing
acceptance and rejection numbers. OC curves
applicable to single, double or multiple plans are
provided.
Variables plans - A sample is taken and one or more

quality characteristic measurements are made on each
unit. The sample size and critical value are based on
the desired sampling risk. These measurements are
then summarized into sample average or standard
deviation which are compared with a critical value
defined in the plan. A decision is then made to accept
or reject the product. An example is ANSI/ASQC Z1.9
(modernized version of MIL-STD-414).
CSQE 2002

Auditing Sampling
Sampling for audit purposes is based on small sample
sizes and may not be representative. There is no
standard sampling plan for audits, but the sample must
be random and must include all audit scope elements.
Random Sampling
Random sampling gives every part an equal chance of
being selected. The sample must be representative of
the product and the sampling sequence must random,
which can be determined from a random number table.
Sequential Sampling
Sequential sampling plans are similar to multiple
sampling plans except that sequential sampling can
theoretically continue indefinitely.
Stratified Sampling
Stratified sampling is to attempt to select random
samples from each group or process that is different
from other similar groups or processes.
CSQE 2002

Required Sample Size

When using statistical inference, the ideal procedure is
to determine the " and $ error desired and then to
calculate the sample size necessary to obtain the
desired decision confidence. The sample size (n)
needed for hypothesis testing depends on:
C The desired type I (") and type II ($) risk

C The minimum value to be detected between the
population means (: - :0)
C The variation in the characteristic being measured
(S or F)
Determine whether a code modification will change the

run time by as much as 4 seconds. The standard
deviation is 20 seconds, 95% confidence level (Z=1.96),
what sample size would confirm the significance of a
mean shift greater than 4 seconds? For variable data
(normal distribution) is:
Requires sample size is 96 modified code run time

values.
CSQE 2002

Required Sample Size (Continued)

The previous formula also works for Poisson data using
6 for F.
c
For binomial data, use the following formula:
Where:
Z = The appropriate Z value

6 = Proportion rate
p
)p = The desired proportion interval
n = Sample size
CSQE 2002

QUESTIONS
7.5. What action should management take when a line chart

shows a steady variance of project performance from the
plan?
a. Adjust the plan accordingly

b. Do nothing
c. Accept the deviation as allowable within the scope of the
plan
d. Take immediate action to correct
7.9. Using the data from the table below, what is the phase
containment of the coding phase?
Phase # of Defects # of Defects
Introduced Found & Removed
Requirements 12 9
Design 25 16
Code 47 42
a. 89.4% b. 86.4% c. 75.0% d. 71.2%
7.10.Software reliability is normally defined in terms of:
a. The probability of failure free operation

b. The defect density of the software product
c. The operational profile of the system
d. Mean time to repair a defect
Answers 7.5 a, 7.9 d, 7.10 a

CSQE 2002

QUESTIONS
7.12.Which of the following is NOT relevant for measuring

effectiveness of inspections?
a. Number of defects per inspector

b. Major defects found and compared to other evaluation
methods
c. How management reports and tracks inspection results
d. The number of defects found in later phases of
development
7.13.Consider the following program:
SUBROUTINE SORT (X,N)

DIMENSION X(N)
IF (N.LT.2) RETURN
DO 20 I = 2, N
DO 10 J = 1, I
IF (X(I).GE.X(J)) GO TO 10
SAVE = X(I)
X(I) = X(J)
X(J) = SAVE
10 CONTINUE
20 CONTINUE
RETURN
END
What is the n2 Halsteads software science for the above

program?
a. 5 b. 6 c. 7 d. 8
Answers 7.12 c, 7.13 c

CSQE 2002

QUESTIONS
7.19.The essential steps of the Goal/Question/Metric paradigm

are:
a. Identify goals, determine metrics, make measurements

b. Identify measurable goals, formulate questions, establish
metrics
c. Identify goals, formulate questions from goals, establish
metrics to verify goals
d. Identify goals, formulate questions to develop metrics,
implement metrics to verify goals
7.21.The element of defect prevention is a key characteristic of

SEI CMM version 1.1 at which of the following levels?
a. Level 3 c. Level 5
b. Level 4 d. Level 2
7.25.The defect rate during formal software system testing (i.e.

software testing after the code is integrated) is usually:
a. Not correlated with the defect rate experienced in the

field
b. Negatively correlated with the defect rate experienced
in the field
c. Positively correlated with the defect rate experienced in
the field
d. Much less than the rate experienced in the field
Answers 7.19 c, 7.21 c, 7.25 c

CSQE 2002

QUESTIONS
7.34.From the table below, what is the approximate defect

removal effectiveness at the design phase?
Phase # of defects
Requirements 6
Design 8
Coding 10
Inspection 15
Testing 31
a. 23% c. 9%
b. 13% d. 15%
7.36.The reliability model that models defect patterns of the

entire development process is known as:
a. Fault count model

b. Littlewood model
c. Rayleigh model
d. Delayed S and Inflection S model
7.40.The main purpose of software quality metrics is to:
a. Measure the effectiveness of the software quality

organization
b. Measure the success of the testing effort
c. Provide a qualitative assessment of the development
process
d. Assess whether software quality requirements are met
Answers 7.34 d, 7.36 c, 7.40 d

CSQE 2002

QUESTIONS
7.45.A measure used to compute the program length, by

counting operators and operands, is known as:
a. Cyclomatic complexity
b. COCOMO
c. Function counts
d. Software science
7.46.In establishing a new software metric for the development

team, which of the following condition(s) should be
avoided?
I. Have a clear objective

II. Use the results to assess individuals
III. Use the results to assess the team
IV. Understanding the effects of software science
a. I and II only c. II and III only

b. II only d. II and IV only
7.51.The basic scale types for software measurement are:
a. Nominal, ordinal, interval, absolute

b. Nominal, ordinal, ratio, absolute
c. Nominal, ordinal, typical, ratio
d. Nominal, ordinal, interval, ratio
Answers 7.45 d, 7.46 d, 7.51 d

CSQE 2002

QUESTIONS
7.53.Defect assessment and reliability models are:
a. Used to evaluate prototype software before starting final

development
b. Used to compare software development projects
c. Examples of models used in software measurement
d. Too complicated to use in most software applications
7.56.The psychology of metrics deals with:
a. Human issues related to the definition and use of metrics

b. Individual responsibility for product defects
c. Performance of teams engaged in process improvement
d. Unexplained improvements in quality and productivity
7.59.Random selection of a sample:
a. Theoretically means that each item has an equal chance

to be selected in the sample
b. Assures that the sample average will equal the population
average
c. Means that a table of random numbers was used to dictate
the selection
d. Is a meaningless theoretical requirement
Answers 7.53 c, 7.56 a, 7.59 a

QUALITY COUNCIL OF INDIANA VIII-1 (452)
CSQE 2002
VIII. SOFTWARE VERIFICATION AND VALIDATION (V&V)
YOU CANT INSPECT

QUALITY INTO A PRODUCT.
SOURCE UNKNOWN
CSQE 2002

A. THEORY
Introduction
Software Verification and Validation is presented in

C Theory
C Reviews and Inspections
C Test Planning and Design
C Test Execution and Evaluation

CSQE 2002

A. THEORY
Theory
Software Verification and Validation (V&V) is a
disciplined approach to evaluating and assessing the
software product during each phase of the software life
cycle. The V&V effort helps ensure that quality is built
into the software during its development and that the
software meets the user requirements.
Software V&V employs review, analysis, and testing

techniques to determine whether a software system and
its intermediate products comply with requirements.
These requirements include functional capabilities and
quality attributes.
The objectives of a V&V effort are to find defects as

early as possible in the life cycle and to determine if
required functions and attributes are built into the
software system.
C Verification that the products of each phase
C Validation that the completed end product

complies with established software and system
requirements.
CSQE 2002

A. THEORY
1. V&V PLANNING PROCEDURES AND TASKS
V&V Planning Procedures and Tasks

Planning of V&V tasks should be part of the overall
project planning activity for the entire software project,
whether the project involves new development,
enhancements, modifications, or maintenance.
The following procedures and tasks are provided as a

guide for use in V&V planning or in preparing Software
V&V Plans (SVVP).
C Identify the V&V Scope
C Establish Specific Objectives from the Scope of

the Project
C Analyze the Project Inputs
C Select Techniques and Tools

CSQE 2002

A. THEORY
V&V Planning Procedures (Continued)

Select Techniques and Tools (Continued)
Some common methods for verification and validation

include:
C Static Analysis
C Structural (Structured) Analysis
C Mathematical Proof
C Simulation
C Testing
CSQE 2002

A. THEORY
V&V Planning Procedures (Continued)

Methods Used in Performing V&V Tasks
Various methods can be used in performing the V&V

tasks. A V&V task may use more than one method (i.e.
code inspections and algorithm analysis for source
code evaluation).
Methods for Iterating V&V

Tasks after Modification
Determine the extent to which V&V tasks should be

iterated as a result of proposed or completed
modifications and enhancements. These planned
iterations are documented in the SVVP and are done
when modifications are completed.
Iterations Based upon Proposed

or Completed Modifications
Proposed modifications are analyzed to determine the

effect of each planned change to software products.
When changes are made, V&V tasks are iterated by
repeating previous tasks or initiating new tasks.
CSQE 2002

A. THEORY
2. V&V PROGRAM
V&V Program Management and Review

The SVVP provides the basis for managing and
reviewing a V&V program. Each topic in the SVVP
includes the objectives and criteria for evaluating task
results.
Managing a V&V Program

Management of a V&V program occurs on an ongoing
basis during the period when the SVVP is performed.
V&V Management Tasks
C Technical accomplishments (Task outputs)

C Resource utilization
C Program status etc.
C Future planning
C Risk management
C Impact analysis of proposed changes
CSQE 2002

A. THEORY
2. V&V PROGRAM
V&V Program Management (Continued)

V&V Management Performance Methods
C Periodic reviews of on-going efforts

C Reviews of V&V tasks and phase
C Evaluate V&V results to determine when to
proceed to the next phase
Reviewing a V&V Program

Review of a V&V program comprises all of the general
responsibilities of management for monitoring,
controlling, and reporting of the V&V effort.
CSQE 2002

A. THEORY
3. EVALUATING PRODUCTS AND PROCESSES
Evaluating Software
Products and Processes
Software work products must be fit for use and
evaluated to assure that:
C The product conforms to its specifications

C The product is correct
C The product is complete, clear, and consistent
C Appropriate alternatives have been considered
C The product complies with standards
C The product meets specified quality attributes
Evaluations identify problems and help to determine the

progress of software development by recommending
that the project continue on to the next stage or correct
specific items first.
CSQE 2002

A. THEORY
3. EVALUATING PRODUCTS AND PROCESSES
Evaluating Software Products

and Processes (Continued)
V&V Evaluation Tasks
C Traceability Analysis
C Documentation
C Source Code
C Plans
C Test Results
C Audit Results
CSQE 2002

A. THEORY
4. INTERFACES
Identify V&V Interfaces

Identify various interfaces used with hardware, user,
operator, and software applications.
Requirements Interface Analysis
A requirements interface analysis is performed to

assure external interfaces to the software and internal
interfaces between software functions are completely
and correctly specified.
Design Interface Analysis
A design interface analysis is performed to assure the

software design interfaces are completely and correctly
specified.
Source Code Interface Analysis
A source code interface analysis is performed to assess

the information and control flow between and within
components. The results of analysis describe
component interface inconsistencies and errors.
CSQE 2002

B. REVIEWS AND INSPECTIONS
1. TYPES
Review and Inspection Types

Management and technical reviews are generally
conducted and provide information for management
action. Inspections and walkthroughs are peer
examinations aimed at assisting the producers in
improving their work.
Software inspections are a powerful way to improve the

quality and productivity of the software process by
assisting programmers in recognizing and fixing their
own errors early in the software engineering process.
C Desk Checking - an informal examination of a

software element by someone other than the
author.
C Walkthroughs - evaluate a software element

and may point out efficiency and readability
problems.
C Inspections - detect and identify software

element defects. Verifies that the software
element satisfies its specifications.
Types of reviews are shown in the table in the Primer.

CSQE 2002

2. ITEMS
Review and Inspection Items

C Project Items
C Project Change Items
C Software Tool Items
C Software Process Items

CSQE 2002

3. PROCESSES
Review Processes
Objectives
Reviews provide management with evidence that:
C The software elements conform to

specifications.
C The development of the software elements is

being done according to plans and guidelines.
C Changes to the software elements are properly

implemented.
Entry Criteria
A software review may not be conducted until:
C A statement of objectives for the review is

established
C The responsible individual indicate readiness
C Software elements are sufficiently complete

CSQE 2002

3. PROCESSES
Review Processes (Continued)

Exit Criteria
A software review is complete when:
C All issues identified in the review statement of

objectives have been addressed
C The results of the review have been issued
Techniques and Methods

Conducting effective reviews is based on the following:
C Team members have identified their issues

before the review starts.
C The review focus is on evaluating conformance

to specifications and ensuring change integrity.
C Conducted by technical leadership and peer

developers.
C Review data, such as deviations and

deficiencies, may be entered into a database.
CSQE 2002

3. PROCESSES
Review Processes (Continued)

Participant Roles
Roles required for the review process include the
following:
C Leader
C Recorder
C Team Member
Management is responsible for acting on the review

team recommendations in a timely manner.
CSQE 2002

3. PROCESSES
Inspection Processes
Objectives
C Find defects at the earliest possible point.
C Verify the software elements satisfy the

specifications and conform to standards.
C Identify deviations.
C Collect engineering data on the software

elements and the inspection process.
Entry Criteria
Inspections are planned and documented according to
the project document and can be triggered by:
C Software element availability
C Project plan compliance
C SQAP or SVVP schedule compliance
C Scheduled reinspection
C At the request of management

CSQE 2002

3. PROCESSES
Inspection Processes (Continued)

Exit Criteria
A single exit criterion applied to all inspections is that
all of the defects, that have been detected, are resolved.
Each project should develop its own criteria to meet the
needs of its specific products and development
environment.
Techniques and Methods

The techniques and methods used for conducting
effective inspections are based on the following
principles:
C The inspection is a formal structured process
C Checklists for each inspection type are tailored

to specific project needs
C Inspectors are prepared
C The focus is on identifying problems and not

on resolving them
C An inspection is conducted by technical people

CSQE 2002

3. PROCESSES
Inspection Processes (Continued)

Participant Roles
All team members, including those with special roles,
are inspectors. Special roles required for the inspection
process include the following:
C Moderator
C Reader
C Recorder (Scribe)
C Inspector (Reviewer)
C Author
CSQE 2002

4. DATA COLLECTION AND SUMMARIES
Data Collection
Software inspections provide data for analysis of the
quality of the software elements, the effectiveness of the
development procedures, and the efficiency of the
inspection process. To enable these analyses, defects
identified during an inspection meeting are categorized
by defect type, class, and severity.
C Defect Type - The defect type identifies

software element attributes.
C Defect Class - The defect class is used to

characterize evidence of nonconformance.
C Defect Severity - Major or Minor

CSQE 2002

Analysis of Data
Analysis of inspection data includes the inspection
metrics:
C Preparation Rate
C Inspection Rate
C Defect Rate
C Defect Density
Phase Containment
The phase containment effectiveness measure (defect
removal effectiveness) is calculated using the following
formula.
Defects removed (at the step) x 100%
Defects existing on step entry + Defects injected during the development step
CSQE 2002

Reporting
Reports should be developed that present the results
from each inspection meeting. A summary of analysis
for each inspection meeting should be included in the
inspection meeting report.
Summary Reports
Analysis results from all inspection meetings, over a
period of time, can be included in a summary report for
the project or organization.
This information can be used to determine the average

inspection rate, average defect rate, and average
effectiveness rate for each software element within the
project, as well as for a large number of software
elements within the organization.
CSQE 2002

C. TEST PLANNING AND DESIGN
1. TYPES OF TESTS
Types of Tests
C Functional - evaluating the feature sets and
capabilities of the system components, the
functions that are delivered, and the security of
the overall system. The program is treated as
a black box.
C Performance - designed to test the run-time

performance of software within the context of
an integrated system.
C Regression - involves conducting all or some

of the previous tests to ensure that new errors
have not been introduced.
There are four types of software maintenance

that will cause one to run regression tests:
C Corrective
C Adaptive
C Perfective
C Preventive
CSQE 2002

1. TYPES OF TESTS
Types of Tests (Continued)

Environmental Load
Load tests study the behavior of the program at
boundary conditions of volume, stress, and storage.
Volume tests study the largest tasks the program can

deal with.
Stress tests look at peak bursts of activity.
Storage tests observe program handling of low memory

or disk.
Load tests cause the system to compete for resources

to check for vulnerabilities in the software or the system
under test.
CSQE 2002

1. TYPES OF TESTS

C Worst Case - designed to test for error handling
when boundary or extreme values are used.
C Perfective - to check that the added capabilities

function as expected.
C Exploratory - purposeful wandering with a

general mission, but without a prescribed
route.
C Random-Input Testing - selecting at random

some subset of all possible input values.
C Certification - done by a third party, a contract

states the level of testing involved and any
standards that must be met.
C Stress - conducted to evaluate a system or

component at or beyond the limits of its
specified requirements.
C Usability - assessed by considering human

factors, overall aesthetics, consistency, and
documentation.
CSQE 2002

1. TYPES OF TESTS

C Real-Time Response - the timing of the data,
parallelism of the tasks, and asynchronous or
intertask synchronization errors.
Test Levels
C Unit Testing - done to show that the smallest
testable piece of software does or does not
satisfy its functional specification, and/or that
its implemented structure does or does not
match the intended design structure.
C Component Testing - done to show that the

component does not satisfy its functional
specification, and/or that its implemented
structure does not match the intended design
structure.
C Integration Testing - hardware and software

components are aggregated to create larger
components. Testing done to show that, even
though components were individually
satisfactory, the combination of components
are incorrect or inconsistent.
CSQE 2002

1. TYPES OF TESTS
Test Levels (Continued)

C System Testing - concerns issues and
behaviors that can only be exposed by testing
the entire integrated system.
C Field Testing - occurs in the customer

environment prior to the system being placed
in operation. Sometimes referred to as a
verification test, a laboratory trial, or a field
trial, and can include any of the following:
C Acceptance Testing
C Qualification Testing
C Operational Testing
C Beta testing - a pre-release version is provided

to the user community with a stipulation that all
bugs are to be reported.
CSQE 2002

2. TEST TOOLS
Test Tools
C Fundamental Tools - include computers, word
processors, compilers, spreadsheets, system
utilities, file viewers, data converters, screen
capture utilities, diagnostic routines, defect
trackers, configuration managers, project
management tools, coverage certifiers, flow
g r a p h g e n e r a t o r s , a n a l yz e r s , a n d
instrumentation facilities.
C Test Execution Automation - automate the

setup, and running, or analysis of tests that
have been either automatically or manually
generated. Include tests managers, data
comparators, capture/replay, stubs and drivers,
and test environments.
C Test Design Automation - include structural

test generators, data-flow test generators,
functional generators, and random test
generators.
CSQE 2002

2. TEST TOOLS
Test Tools (Continued)

C Automated Testing
C Static Analyzers
C Code Auditors
C Assertion Processors
C Test File Generators
C Test Data Generators
C Test Verifiers
C Test Harnesses
C Output Comparators
CSQE 2002

3. TEST STRATEGIES
Test Strategies
Top-Down
The top-down testing strategy starts with the top, or
initial, module in the program. When selecting the next
module to be incrementally tested, at least one of the
modules superordinate (calling) modules must have
been tested previously.
To test the top module, stub modules representing the

next lower level modules must be written. The
production of stubs is very important to this test
strategy. After the testing the top module, one of the
stubs is replaced by an actual module, and the stubs
required by that module are added.
Bottom-Up
The bottom-up test strategy starts with the terminal
modules in the program. Modules may need driver
modules that contain wired-in test inputs, calls for the
module, and displays for the outputs.
CSQE 2002

3. TEST STRATEGIES
Test Strategies (Continued)

Black Box
Black box testing methods focus on the functional
requirements of the software and are based on the
external characteristics of the program being tested.
That is, black box testing enables the software engineer
to derive sets of input conditions that will fully exercise
all functional requirements for a program.
White Box
White box testing is a test case design method that
uses knowledge of the programs procedural design
control structure to derive test cases. Using white box
testing methods, the software engineer can derive test
cases that:
(1) guarantee that all independent paths within a
module have been exercised at least once,
(2) exercise all logical conditions on their true and
false sides,
(3) execute all loops in their boundaries and within
their operational bounds, and
(4) exercise internal data structures to ensure their
validity.
CSQE 2002

3. TEST STRATEGIES
Test Strategies (Continued)

Simulation
A simulation is a model that behaves or operates like a
given system when provided a set of controlled inputs.
Simulation tools can provide the ability to test the
behavior of some software before the software is built.
I/O First
I/O First testing is the use of every valid input condition
as a test case. Additional input conditions are also
used as test cases to demonstrate every valid output.
Beta Testing
Beta testing uncovers errors that only the end user
seems able to find. The beta test is conducted at one or
more customer sites by end users of the software.
Software is sometimes tested at the developers site by

a customer in a controlled environment and are called
alpha tests.
CSQE 2002

4. TEST DESIGN
Test Design
Fault Insertion
This method involves the seeding (injecting) of errors
into a program, testing the program for a while, and
then counting the number of seeded versus original
errors detected. The total number of original errors in
the program can then be estimated.
Fault-Error Handling
This method involves generating each condition for
which a program has error handling code to verify that
the program recognizes the condition and properly
deals with it.
Equivalence Class Partitioning

Equivalence partitioning is a black box testing method
that divides the input domain of a program into classes
of data from which test cases can be derived. An ideal
test case uncovers a class of errors that might
otherwise require many cases to be executed.
CSQE 2002

4. TEST DESIGN
Test Design (Continued)

Equivalence Class Partitioning (Continued)
Two types of equivalence classes are identified: valid
equivalence classes that represent valid inputs to the
program, and invalid equivalence classes that represent
all other possible states of the condition (i.e. erroneous
input values).
Boundary Value Analysis

A greater number of errors tend to occur at the
boundaries of the input domain than in the center.
Boundary value analysis leads to a selection of test
cases that exercise bounding values.
Because boundary conditions may be subtle and

difficult to identify, boundary-value analysis may not be
a simple task. This method does not test combinations
of input conditions.
CSQE 2002

4. TEST DESIGN

Cause-Effect Graphing
Cause-effect graphing is a test case design technique
that provides a concise representation of logical
conditions and corresponding actions. It aids in
selecting, in a systematic way, a high-yield set of test
cases.
Cause-Effect Graphing Symbols

CSQE 2002

4. TEST DESIGN

Error Guessing
Some people seem to be naturally adept at program
testing and seem to have a knack for smelling out
errors. The basic idea is to enumerate a list of possible
errors or error-prone situations and then write test
cases based on the list.
Test Cases
Test cases are developed using black-box methods and
then supplementary test cases are developed as
necessary using white-box methods. An excellent test
case satisfies the following criteria:
C It has a reasonable probability of catching an

error.
C It is not redundant.
C Its the one most likely to find an error.
C It is neither too simple nor too complex.
C It makes program failures obvious.

CSQE 2002

5. TEST COVERAGE OF SPECIFICATIONS
Test Coverage of Specifications

Functions
Most functional test techniques are useful in testing
functional bugs. They are also useful in testing for
requirements and specification bugs to the extent that
the requirements can be expressed in terms of a model,
on which the technique is based.
States
State testing strategies are based on the use of finite-
state machine models for software structure, software
behavior, or specifications of software behavior. States
are represented by nodes in the state graph. When the
model is subjected to inputs, the state changes, or is
said to have made a transition. Transitions are denoted
by links that join the states.
CSQE 2002

5. TEST COVERAGE OF SPECIFICATIONS
Test Coverage of Specifications (Cont.)

Data and Time Domains
Domain testing can be based on specifications and/or
implementation information. Domain testing is usually
applied to one input variable or to simple combinations
of two variables, based on specifications. The domain-
testing strategy is simple, yet potentially tedious.
Localization
The translation and export of software is becoming
more common. Localization is the process of adapting
software for a new place. It might involve a change of
language, but it doesnt have to. The culture must be
considered as well. The localizer is the person who
executes or manages the translation and other
adaptations of the program to the new market.
Internationalization
Adapting software for the international marketplace
requires consideration of several localization strategies.
CSQE 2002

6. TEST ENVIRONMENTS
Test Environments
Test Libraries
Test libraries are used to manage various test
resources. Test cases can be controlled, so changes in
test cases can be tracked in a controlled manner.
Stubs/Drivers
A stub is used to simulate a called subroutine. When a
submodule is not available, a stub can be used to
simulate the called submodule. A driver is the opposite
of a stub. It simulates the calling module and perhaps
the entire submodule environment.
Harnesses
Harnesses are automated test tools that support
processing of tests in a test environment, feeding input
data, and simulating subsidiary modules by stubs.
Simulators As Test Environments

A simulation is a model that behaves or operates like a
given system when provided a set of controlled inputs.
CSQE 2002

7. SUPPLIER COMPONENTS AND PRODUCTS
Supplier Components and Products

Commercial-Off-The-Shelf Products (COTS)
Risks C Defined by market-driven need
C Supplier does not modify for use
C Suppliers problem resolution process
may not be adequate or timely
Benefits C Commercially available

C Stable
C Known capabilities and limitations
C Demonstrated fitness for use
Modified-Off-The-Shelf Products (MOTS)

Risks C May not have all required capabilities
C Supplier may not be willing to modify
C May not meet requirements
C Documentation of modified portion
may not be accurate or complete
C Resolutions may not be adequate
Benefits C Commercially available

C Somewhat stable
C Has known capabilities and limitations
C Has demonstrated fitness for use
CSQE 2002

Supplier Components (Continued)

Fully Developed Software
Components and Products
Risks C May not meet specified requirements
C Requires testing to demonstrate
fitness for use
C Documentation may not be accurate
C Problem resolution process may not
be adequate or timely
Benefits C Unique for a specific application

C Supplier produced a one-of-a-kind
product
C Has potential for future modification
C Documentation is product specific
CSQE 2002

Testing Supplier
Components and Products
Prior to evaluating and testing supplier software,
establish acceptance criteria to assure that the
software, when demonstrated, will meet the needs of
the organization. These may include:
C Satisfying product requirements

C Using the software within another system
C Obtaining user documentation
C Receiving adequate technical support
Commercial-Off-The-Shelf (COTS)
Test cases should be developed to demonstrate the
software meets the functionality required,
documentation is accurate and complete, and the
suppliers problem resolution process is adequate and
timely.
Modified-Off-The-Shelf (MOTS)
Test cases should be developed to demonstrate the
software meets the functionality required, specified
requirements, documentation is accurate and complete,
and the problem resolution process is adequate.
CSQE 2002

8. TEST PLANS
Test Plans
A test plan describes the scope, resources, schedule of
intended testing activities, test items, features to be
tested, testing tasks, who will do each task, and any
risks requiring contingency planning.
System Test Plans

Evaluate the adequacy of system test plans to
demonstrate compliance with operational requirements,
performance at interfaces, adequacy of documentation,
tracing from each test to the requirements.
Acceptance Test Plans

Evaluate the adequacy of acceptance test plans to
demonstrate meeting requirements objectives,
adequacy of documentation, correct packaging, and
tracing from each test to requirements.
Validation Test Plans

Evaluate the adequacy of validation test plans to
demonstrate compliance with established software and
system requirements.
CSQE 2002

D. TEST EXECUTION AND EVALUATION
1. TEST IMPLEMENTATION
Test Implementation
Scheduling
A test manager is responsible for part of an overall

project schedule. Four key scheduling objectives:
C Provide predictability to the project manager.
C Identify opportunities to pull in or protect the

project schedule.
C Be fair to your staff.
C Maximize productivity.
Tips toward achieving good estimates.
C Measure performance and productivity.
C Identify and estimate every task.
C Classify the project.

C Identify tasks as fixed versus recurring.
CSQE 2002

Test Implementation (Continued)

Freezing
Freezing the software prevents further changes from

being made. In some organizations, the freeze date and
the final software are the same milestone. The design
keeps changing until the code is frozen for release to
manufacturing.
Dependencies
The testing effort is dependent on the availability of staff

resources, tools and techniques, and test bed
availability. These dependencies are identified during
test planning. Dependencies may also exist in obtaining
special hardware and documentation.
CSQE 2002


V-Model
The V - model can be used as both a software

engineering process and as a test implementation
platform.
The software testing element may be envisioned by

moving up the right hand side of the V model.
CSQE 2002


Error Repair Models
Error repair and removal involves debugging and other

strategies for identifying where the error occurs in the
code, the process necessary to identify what in the code
causes the error, and removing it. Debugging software
is basically still an art rather than a science, and there
are very few strategies that are precise and well defined.
Acceptance Testing
Acceptance testing is typically conducted when coding

is finished and the software is submitted to the testing
group for further testing.
Be clear about the test criteria so the developers can

run the tests themselves and know they pass before
submitting the code for formal testing.
CSQE 2002


Resources
Staff resources are needed to plan the test, design test

cases, write procedures, and execute the tests. Test
results must be analyzed and reports written.
Automated testing tools are used to improve the testing
effort and to help reduce the amount of work involved in
analyzing test results.
Analysis of Test Results
Test results are analyzed to assess the progress of

testing, whether the testing effort is on schedule, the
number and type of anomalies detected, and the
technical quality of the software.
The results are used in determining when testing is

complete and whether the software is ready to continue
to the next phase of development.
CSQE 2002

2. TEST DOCUMENTATION
Test Documentation
Defect Recording
A problem report is used to record defects occurring

during testing.
Defect Tracking
Whenever a defect has occurred, it means that someone

or something has failed. It is as important to find and
correct the cause of the defect as it is to fix the defect
itself.
Test Report Completion Metrics
The test coverage metric is a measure of the

completeness of the testing process.
TC(%) = (CAP) X (PROG)
CAP = (implemented capabilities)/ (required capabilities)
PROG = (program primitives tested) /

(total program primitives)
CSQE 2002

Test Documentation (Continued)

Trouble Reports
Trouble reports allow communication between the user

of the software and the development organization.
When a software problem occurs, the user records the
problem on a form and conveys it to the development
organization.
Trouble reports may also be used for internal purposes.
Any trouble report should contain sufficient information

for the person attempting to correct the problem to
recreate the failure and determine when the product
executes correctly.
The status of trouble reports should be recorded in the

trouble report log. A trouble report log is a cumulative
record of trouble reports and their resolution.
The trouble report log contains the report number, title,

dates, priority, summary, description, status and
disposition.
CSQE 2002


Input/Output Specifications
A test case specification document specifies each input

required to execute a test case, it identifies all
appropriate databases, files, terminal messages,
memory resident areas, and values passed by the
operating system, including all required relationships
between inputs.
It also specifies all of the outputs and features required

of the test items, with the exact value for each required
output or feature.
C Test Plans provide an overview of the testing effort

for the product.
C Logs are a chronological record of the test

executions and events that happened during
testing.
CSQE 2002


C Test Design Specifications are used to specify how
a feature or group of features will be tested.
C Test Cases are identified by a test design

specification. A test case may be referenced by
several test design specifications, which should
include enough information to permit reuse.
C Test Procedure Specifications describe the steps

for executing a set of test cases and analyzing their
results.
C Test Reports summarize the testing done and

evaluates the results.
CSQE 2002

3. TEST REVIEWS
Test Reviews
Methods for Reviewing Testing Efforts
Management should conduct periodic reviews of on-

going testing efforts, accomplishments, and the use of
resources.
Management should also review testing tasks and

phase results for technical quality; evaluate test results
from each phase and anomaly resolution to determine
when to proceed to the next phase; and define changes
to testing tasks to improve the testing effort.
The reviews may focus on:
C Technical accomplishments
C Future planning
C Risk management
C Resource utilization
CSQE 2002

4. CODE COVERAGE METRICS
Code Coverage Metrics

Branch-to-Branch
The branch testing criteria or strategy is the execution

of enough tests to assure that every branch alternative
has been exercised at least once. For structured
software, branch testing and branch coverage, strictly
includes statement coverage. Branch and statement
coverage are the minimum testing requirement.
Condition
Test cases are developed to exercise or cover the logic

of the program. The test results are used to evaluate
the degree to which the conditions, decisions, and
statements have been exercised.
For programs containing only one condition per

decision, a minimum test criterion is that there be a
sufficient number of test cases to evoke all outcomes of
each decision at least once and invoke each point of
entry at least once. For multiple conditions, test cases
evoke all possible combinations of condition outcomes
and all points of entry to the program.
CSQE 2002

Code Coverage Metrics (Continued)

Domain Coverage
The domain testing method is based on construction of

test data sets for selected paths in a program, where a
path corresponds to some possible flow of control. The
method attempts to uncover errors in a particular path
domain.
Domain coverage is the proportion of possible predicate

values at or near boundaries for a given path or set of
paths in a program.
McCabe Cyclomatic Complexity
The McCabe cyclomatic complexity metric is designed

to indicate a programs testability, understandability and
maintainability. This is accomplished by measuring the
control flow structure, in order to predict the difficulty of
understanding, and extent to which the program is likely
to contain defects.
CSQE 2002


McCabe Cyclomatic Complexity (Continued)
1
R-1
R-2
R-5
4
R-3
3
R-4
5
R-6
6
7
V(G) = E - N + 2 = 11 - 7 + 2 = 6
E = the number of flow graph edges

N = the number of flow graph nodes
CSQE 2002


Boundary
Boundary code coverage uses values that lie in or on

the boundary region (such as endpoints) and
maximum/minimum values (such as field lengths).
Path
A path through a program is a sequence of instructions

or statements that starts at an entry to the program and
goes to an exit. A path testing criteria requires
execution of all possible control flow paths through the
program, which is generally impossible to achieve.
CSQE 2002


Individual Predicate
A predicate is implemented as a process whose

outcome is a truth-functional value. Predicates are
based on relational operators, such as >, >=, =, <=, <, <>
and others such as ... is a member of ... is a subset of,
A .OR. B or A .AND. B and more complicated Boolean
expressions.
Predicate coverage has been achieved if all possible

combinations of truth values corresponding to the
selected path have been explored under some test.
Data
Data-flow testing is the name given to a family of test

strategies based on selecting paths through the
programs control flow in order to explore sequences of
events related to the status of data objects.
CSQE 2002

5. CUSTOMER DELIVERABLES
Customer Deliverables
Testing the Accuracy of
Customer Deliverables
Write system test cases to assure that accuracy and

clarity are demonstrated for the following:
C The software product matches the description of

the software package
C The procedures contained in the user

documentation work without error
C The examples illustrated in the documentation and

training materials work without error
C The capabilities described in the marketing and

training materials exist in the product
Testing of packaged and web-based products poses

new challenges because of hardware, software, and
architectural considerations.
CSQE 2002

6. SEVERITY OF ANOMALIES
Evaluating the Severity of Anomalies

When an anomaly is reported, review the description of
the anomaly to determine:
C Extent to which software operations are impacted
C Any deviation in operation of the software from

expectations
C If information is sufficient to recreate the anomaly
Analyze the effect of the anomaly on the software and

system, using the organizations procedures that define
the severity levels or categories (critical, major,
moderate, minor). Assign a severity level to the
anomaly.
Perform a root cause analysis and evaluate alternate

solutions to resolving the anomaly.
Select a solution and describe its implementation in a

change proposal.
Submit the proposed change for management review

and approval, based on the organizations procedures.
CSQE 2002

QUESTIONS
8.2. Testing a software product or system to determine if

modifications have introduced faults or defects is called:
a. System testing
b. Bottom-up testing
c. Integrated testing
d. Regression testing
8.3. Consider the following figure:
The cyclomatic complexity for

the program is:
a. 4
b. 5
c. 8
d. 10
8.9. The seeding of errors into a program and counting the

number of seeded versus identified defects allows the test
engineer:
a. To estimate the number of defects in the program

b. To estimate the number of remaining defects
c. To estimate the number of errors left in the program
d. To estimate the number of seeded and unseeded errors
Answers 8.2 d, 8.3 a, 8.9 a

CSQE 2002

QUESTIONS
8.14.The customer documentation gets validated during:
a. Inspection
b. System test
c. Functional test
d. Integration test
8.16.Software that is commercially available, and whose fitness

for use has been demonstrated by a broad spectrum of
users, is called:
a. Fully developed
b. MOTS
c. Shrink-wrapped
d. COTS
8.19.The extent to which V&V tasks should include regression

testing would be based upon:
a. Evaluation of audit results

b. Impacts on software operation
c. Iterations due to modifications
d. Assessment of proposed changes
Answers 8.14 b, 8.16 d, 8.19 c

CSQE 2002

QUESTIONS
8.25.An author of a software users manual, during a software

inspection, includes all of the following roles EXCEPT:
a. Meeting the entry criteria requirements

b. Reworking the manual to meet exit criteria
c. Answering questions that come up during the inspection
d. Leading the inspection team through the manual
8.31.Black box testing should be performed in conjunction

with which of the following other test strategies?
a. White box testing

b. Beta testing
c. Top-down testing
d. Bottom-up testing
8.33.Testing of code based on selecting paths through the

programs control flow is known as:
a. Branch-to-branch testing
b. Localization
c. Data-flow testing
d. Path testing
Answers 8.25 d, 8.31 a, 8.33 c

CSQE 2002

QUESTIONS
8.36.The computer program that you have been asked to test

is considered highly complex and was developed by a
group that you feel is at an SEI maturity level 1. The table
of estimated effort in weeks provided below was
developed based on past experience.
Program Anticipated Reliability
Complexity High Med Low
Low 1 2 6
Medium 3 8 12
High 4 16 22
From the table estimate the number of weeks that testing will
take: a. 22 b. 12 c. 9 d. 4
8.37.An important activity that must be accomplished prior to

the testing phase is:
a. Freezing the software

b. Having user manuals complete
c. Removing defects
d. Installing the software in its operating environment
8.39.Peer reviews, inspections, and walkthroughs are:
a. Milestone events in software projects

b. Methods of establishing software requirements
c. Types of inspections performed by peers to improve
quality
d. Alternatives to software element testing
Answers 8.36 a, 8.37 a, 8.39 c

CSQE 2002

QUESTIONS
8.45.Bottom-up testing requires which of the following

conditions?
a. All software must exist at the sub-module level

b. Testing must begin at a terminal module
c. All terminal modules must be tested before testing any
higher modules
d. Special drivers are required only for critical functions of
missing modules
8.49.White-box testing is distinguished from black-box testing

by:
a. Use of the control structure to derive test cases

b. Demonstrating that program requirements have been met
c. Letting software engineers test from the bottom up
d. Finding more incorrect or missing function errors
8.50.Test cases should satisfy which of the following criteria?
a. Combine as many tests in one test case as possible

b. Collect all available data on the function being tested
c. Have at least a 50% chance of catching an error
d. Make program failures obvious and likely to be detected
Answers 8.45 b, 8.49 a, 8.50 d

CSQE 2002

QUESTIONS
8.52.Test coverage of code is accomplished by:
I. Path testing until all control flow paths have been

exercised
II. Branch testing until all branches have been exercised
III. Predicate testing until all combinations of truth values
have been explored
IV. Data-flow testing until all sequences of events related
to data objects have been explored
a. I and II only c. I, II and IV only

b. I, II and III only d. I, II, III and IV
8.54.Test documentation typically includes which of the

following?
a. Test plans, test procedures, test cases, test reports

b. Test plans, test procedures, test cases, test data
c. Test plans, test procedures, test sequences, test data
d. Test plans, test schedules, test sequences, test data
8.58.The testing unit, that simulates the calling module and

perhaps the entire environment under which a subroutine
is to be tested, is called a:
a. Driver
b. Stub
c. Test library
d. Harness
Answers 8.52 b, 8.54 a, 8.58 a

QUALITY COUNCIL OF INDIANA IX-1 (518)
CSQE 2002
IX. SOFTWARE CONFIGURATION MANAGEMENT
NOTHING ENDURES BUT

CHANGE.
HERACLITUS
540 - 480 BC
CSQE 2002

INTRODUCTION/RATIONALE
Software Configuration Management is presented

in the following topic areas:
C Introduction/Rationale
C SCM Definitions
C Configuration Infrastructure
C Configuration Identification
C Configuration Control
C Configuration Status Accounting
C Configuration Audits
C Release and Distribution Issues
CSQE 2002

Introduction
Software configuration management (SCM) is a
discipline for managing the evolution of computer
products during the initial stages of development and
through to maintenance and final product retirement.
SCM provides the following insights and benefits:
C Product attributes are defined
C Product configuration is documented
C Products are correlated with their requirements
C Proposed changes are identified and evaluated

for impact prior to making change decisions
C Change activity is managed using a defined

process
C Configuration information is organized for

retrieval
C Actual product configuration is verified against

the required attributes
CSQE 2002

SCM Rationale
If each software configuration item (SCI) simply
spawned other SCIs, little confusion would result.
Change is inevitable when computer software is built.

The most frustrating software problems are often
caused by poor software configuration management
(SCM).
Software configuration management (SCM) helps

reduce problems by controlling and coordinating the
work products of many different software engineers on
a common project.
Without SCM coordination results in problems:
C Simultaneous update
C Double maintenance
C Shared code & work products
C Common code
C Versions
The key is to have a control system that addresses the

issues for all software configurations. SCM is a set of
activities that has been developed to manage change
through the software engineering process.
CSQE 2002

SCM DEFINITIONS
SCM Definitions
Baseline A specification or product that has

been formally reviewed and agreed
upon, that thereafter serves as the
basis for future development, and that
can be changed only through formal
control procedures.
Configuration A group responsible for evaluating and
Control approving or disapproving proposed
Board changes to configuration items, and
for ensuring implementation of
approved changes.
Configuration An element of configuration
Identification management, consisting of selecting
the configuration items for a system
and recording their functional and
physical characteristics in technical
documentation.
Configuration An aggregation of hardware, software,
Item or both, that is designated for
configuration management and is
treated as a single entity in the
configuration management process.
CSQE 2002

SCM DEFINITIONS
SCM Definitions (Continued)
Configuration Consists of recording and reporting of

Status information that is needed to manage
Accounting a configuration effectively.
Patch An interim software fix pending a final
design fix in source code.
Release The formal notification and
distribution of an approved version.
Software A controlled collection of software
Library and related documentation designed
to aid in software development, use,
or maintenance.
Software Life The period of time that begins when a
Cycle software product is conceived and
ends when the software is no longer
available for use.
CSQE 2002

A. CONFIGURATION INFRASTRUCTURE
1. CONFIGURATION MANAGEMENT
Configuration Infrastructure
Configuration Infrastructure is presented in the

C Configuration Management
C Library/Repository Processes
C Defect Tracking and Library Tools
CSQE 2002

Configuration Management
SCM is divided into the following major functional areas
as illustrated below:
Top Level Configuration Management Model

CSQE 2002

Configuration Management (Continued)

SCM Group
The SEI CMM requires a SCM group which has the

leadership role and is responsible for planning,
coordination, and implementing SCM during all phases
of a project life cycle.
SCM Responsibilities
The SEI CMM identifies the following as responsibilities

of the SCM group:
C Develop the SCM plan and standards

C Manage the projects software baseline library
C Identify software products under SCM
C Manage project member access to the library
C Update software baselines
C Create test and delivery products
C Record product baseline and configuration
change actions
C Produce and distribute configuration status
reports
CSQE 2002


SCM Responsibilities (Continued)
Watts Humphrey addresses SCM responsibilities in

three specific areas:
C Configuration manager
C Module ownership
C Change control board (CCB)

CSQE 2002


SCM Responsibilities (Continued)
The IEEE Guide to Software Configuration Management

identifies the following basic SCM responsibilities:
C Configuration identification
C Configuration control
C Status accounting
C Co-chair audits and reviews
C Co-chair configuration control board (CCB)
C Establish and maintain engineering baselines
C Implement and maintain SCM plan
SCM Planning
The planning activities involved in the management of

a controlled item requires the development of an
internal document which specifically describes the
tasks involved in each phase of product development.
This document, termed the Configuration Management
Plan is a requirement of the recently released ISO
document for telecommunications termed TL 9000 and
a requirement of the SEI-CMM.
CSQE 2002

SCM and Productivity

F.P. Brooks, Jr. in The Mythical Man-Month observed
that it is unreasonable to measure in man-months the
amount of work necessary to build software. A project
that takes one person 12 months is a 12 man-month
project. However, this 12 man-month project is not
likely to be completed by two people in 6 months, or
worse yet by 4 people in 3 months.
Brooks attributed the fall-off in productivity to be the

problem of communication between staff members. The
more people involved, the more time is spent on
communication among the staff.
The number of communication channels increases

faster than the number of people involved. A large team
spends proportionally more time communicating than a
small team.
CSQE 2002

2. LIBRARY/REPOSITORY PROCESSES
Library/Repository Processes
Library control procedures are used to identify how
software configuration items are placed in formal
libraries for the project. These procedures describe
how:
C Controlled software libraries are identified

C Software configuration Items of identified
baselines are physically placed under control in
the appropriate library
Typically, a software library, used to control a project,

considers the following issues in its library procedures:
C Format C General requirements

C Location C Receiving and inspection
requirements
C Documentation C Access control
CSQE 2002

2. LIBRARY/REPOSITORY PROCESSES
Library/Repository (Continued)
Fundamentally three kinds should be considered,
dynamic, controlled, and static.
Types of Libraries
The dynamic library, or developers/programmers

library, is a library used for holding newly created or
modified software entities.
The controlled library, or master library, is a library used

for managing the current baseline(s) and for controlling
changes made to them.
The static library, or software repository, is a library

used to archive various baselines released by general
use.
CSQE 2002

3. DEFECT TRACKING AND LIBRARY TOOLS
Defect Tracking and Library Tools
Concepts in Software
Configuration Management Tools
The Software Engineering Institute (SEI) has conducted
extensive research in software configuration
management (SCM) technology. They discovered that
there is no consistent SCM terminology and the
functionality provided by SCM systems was not
consistent. The SEI has identified 15 concepts that
enable people to discuss automated SCM support.
These concepts are listed below.
C Repository C Subsystem
C Distributed Component C Object Pool
C Context Management C Attribution
C Contract C Consistency
Maintenance
C Change Request C Work Space
C Life Cycle Model C Transparent View
C Change Set C Transaction
C System Modeling
CSQE 2002

Defect Tracking (Continued)
SCM Tools (Continued)

The SEI has identified four SCM models that incorporate
various combinations of the above concepts.
C The Check-out/Check-in Model offers a version

for the management of individual system
components.
C The Composition Model focuses on improving

the construction of system configurations
through selection of alternative versions.
C The Long Transaction Model emphasizes the

evolution of systems as a series of
configuration versions and the coordination of
concurrent team activity.
C The Change Set Model promotes a view of

configuration management focused on logical
changes.
CSQE 2002

SCM Tools (Continued)

UNIX Utilities
Many state-of-the-art SCM tools have their roots in the

development of the UNIX utilities:
C Source Code Control System (SCCS)

C Revision Control System (RCS)
C Make
SCCS and RCS serve as the repository in the commonly

implemented Check-out/Check-in SCM model. SCCS
implements the forward delta scheme that involves
storing the initial baseline and the differences between
each baseline. RCS uses reverse delta storage that
involves storing the latest version along with the
differences between each previous baseline.
The UNIX utility, Make, was developed to automate the

build process. Make builds an executable image from
source and object code, and stores the dependencies
between modules and the rules to build the system. It
identifies changed modules and dependencies and
rebuilds only those modules.
CSQE 2002

SCM Tool Features

State-of-the-art SCM tools may have all, or some
combination, of the following features:
C Version Control C Reporting/Query

C Configuration Support C Tool Integration
C Process Support C Build Support
C Change Control C Release
Management
C Team Support C Customization
Support
C Library/Repository C Graphical User
Support Interfaces (GUIs)
C Security/Protection C Distributed
Development
C Web Support
CSQE 2002

Version Control
Version control is a basic requirement for an SCM tool.
It ensures repeatability - the ability to reproduce any
version of the software at any given time. Version
control involves controlling the different versions of
software, uniquely identifying versions and
configurations, and providing version change history to
ensure traceability.
Configuration Support
A configuration is a collection of components that fulfill
a particular purpose. Tools provide mechanisms that
enable the user to correctly identify and model the
software system, as well as, track relationships between
items in the configuration. To establish traceability
between components, tools allow users to establish
links between components.
CSQE 2002

Library/Repository
The library or repository captures SCM information and
stores versions of items. Tools provide mechanisms
that create an audit trail of all SCM activities and
database transactions.
SCM Tools and Process Support

Several tools include a predefined life cycle model and
provide mechanisms to ensure the proper steps are
completed at each stage of the life cycle.
Tools commonly implement process controls by

allowing the user to define Triggers. Triggers execute
user-defined scripts prior to, or in response to, certain
events, (e.g., when a component is checked out from
the repository, an e-mail message is sent to notify other
team members).
CSQE 2002

Reporting/Query
Most tools have the capability to generate standard
reports and allow the user to develop customized
reports. Typical SCM reports include:
C Dependency Report
C Impact Report
C Build Report
C Change Status Report
C Difference Report
C History Report
C Access Control Report
C Conflict Detection Report
Standard Query Language (SQL), or proprietary

database query languages, are used to obtain database
information. The query capability supports
configuration audits and the collection of metrics.
CSQE 2002

Security/Protection
Most tools allow the user to define varying levels of
access for each project. Mechanisms are also
implemented to provide protection from accidental or
intentional data corruption and loss. Several tools
provide backup, archive, and restore capabilities.
With the introduction of CASE, the traceability problem

is complicated because there is an increased number of
items that need to be traced. Many CASE tools prohibit
SCM tools from accessing or controlling data. Several
distinct approaches have been developed:
C Integrated Project Support Environment (IPSE)

such as one based on the international
standard, the Portable Common Tool
Environment (PCTE)
C A tool coalition set from CASE tool vendors

where the vendors do all the source-code level
integration
C A meta-tool that enables customers to rapidly

develop their own highly customized set of
tools
CSQE 2002

Build Support
SCM tools need to track each item and information
about each item that comprise the build. Several tools
implement their own build facility while others interface
with Make.
C Cross development builds - building on a

remote or different target machine
C Parallel builds - running multiple build

processes on one machine at the same time
C Distributed builds - running multiple builds on

different machines at the same time
Graphical User Interfaces

State-of-the-art tools provide GUIs that have the
capability to display the version history of the system,
allow visual merging, and display on-line reports and
forms. Older tools are currently being redesigned to
support GUIs.
CSQE 2002

B. CONFIGURATION IDENTIFICATION
1. CONFIGURATION ITEMS
Configuration Identification
Configuration Identification is presented in the

C Configuration Items
C Baselines
C Configuration Identification Methods
C Software Builds
CSQE 2002

Configuration Items
Effective configuration identification is a pre-requisite
for the other configuration management activities.
Configuration Identification Process Activity Model

CSQE 2002

Configuration Items (Continued)

Configuration identification involves the following:
C Identifying the structure of the software system

C Uniquely identifying individual components
C Making them accessible in some form
The goals of configuration identification are:
C To create the ability to identify the system

components throughout the life cycle
C To provide traceability between the software
and related SCIs.
Identification activities include the following:
C Selecting items to be placed under SCM control

C Developing the software hierarchy
C Creating an identification scheme that reflects
the software hierarchy
C Uniquely identifying the various revisions of the
software product
C Defining relationships and interfaces between
the various software products
CSQE 2002

Managerial Factors that Guide

Software Product Partitioning
Software configuration management is essentially a
discipline applying technical and administrative
direction and surveillance for managing the evolution of
computer program products during all stages of
development and maintenance. SCM spans all areas of
the software life cycle.
SCM affects the entire software development life cycle

because at some point in the project the product must
be partitioned into SCIs. If this is done too soon in the
project, the changes become very expensive and the
return on investment may be minimal. If partitioning
comes too late in the life cycle, the result may be a
project in which change cannot be managed and the
product is poorly defined and very difficult to maintain.
CSQE 2002

Technical Factors that

Guide Product Partitioning
The following Figure presents a typical breakdown of
software into its distinct parts and presents a
numbering scheme uniquely identifying each
component.
Product Partitioning and Numbering

CSQE 2002

What is a Software Configuration?

The output of the software engineering process is
information that may be divided into three broad
categories:
C Computer programs
C Documents that describe the computer
programs
C Data structures
The items that constitute all information produced as

part of the software engineering process are collectively
called a software configuration.
Software Configuration Item

The term software configuration item (SCI) refers to the
information that is created as part of the software
engineering process. The term configuration item (CI)
is sometimes used interchangeably with the term SCI.
From a formal standpoint, the IEEE defines a CI as a

collection of hardware or software elements treated as
a unit for purposes of configuration management.
CSQE 2002

2. BASELINES
Baselines
The term baseline is an SCM concept that encompasses
the following:
C How SCIs are related to each other

C How SCIs are related to project milestones
The term baseline represents the assignment of a name

to each group of SCIs that are related to each other. A
baseline is a relative arrangement of software parts as
embodied in a software product.
In the context of the software engineering project, a

baseline can be defined as a milestone in the
development of the software that is marked by the
delivery of one or more SCIs and the approval of these
SCIs that is obtained through formal technical review.
It is common for a series of developmental baselines of

increasing complexity to be established during the
various internal and external reviews to determine
progress and technical suitability.
CSQE 2002

2. BASELINES
Versions
From a formal definition standpoint, a baseline is a
specification or product that has been formally reviewed
and agreed upon, that thereafter serves as the basis for
further development, and that can be changed only
through formal change procedures.
When an item is baselined, it becomes frozen. The term

frozen means that the item can only be changed by
creating a new version.
Version Control
Version control combines procedures and tools to
manage different versions of the SCIs that are created
during the software engineering process. It ensures
repeatability and the ability to reproduce any version of
the software at any given time.
The term version is used here to indicate that the SCI

has a defined set of functional capabilities. As
functional capabilities are changed, the SCI is given a
different version identifier.
CSQE 2002

2. BASELINES
Version Control (Continued)

Bug fixing means making changes to a program (or
module or object) that corrects only errors in the design
logic, but does not affect documented functional
capabilities since none of the requirements have
changed. The configuration scheme must provide for a
clear identification of revisions and versions, regardless
if the change is a bug fix or major SCI redesign.
Patches are modifications to an object program made

by replacing a portion of the existing machine code with
modified machine code. Therefore, the object program
is changed without recompiling the source program.
The version control system must be sophisticated
enough to ensure traceability between all SCIs related
to the patch.
Version Control Automation

A variety of automated approaches to version control
(e.g. Code Change Control (CCC), Revision Control
System (RCS), Software Configuration Control System
(SCCS), Change Management System (CMS)) have been
developed to aid with configuration identification tasks.
CSQE 2002

2. BASELINES
Naming Conventions
Another prime function of configuration identification
methods is to:
C Devise the identification numbering or labeling

structure for tracking SCIs
C Set identification schemes for these various
views of the system
C Devise a method for relating the different views
of the same product
With respect to naming configuration items, SCM

planning should describe the methods for naming
controlled items for purposes of storage, retrieval,
tracking, reproduction and distribution. Activities may
include version marking, labeling of documentation and
executable software, serialization and altered item
marking for executable code or data embedded on a
microchip, and identification of physical packaging.
CSQE 2002

3. CONFIGURATION IDENTIFICATION METHODS
Configuration Identification Methods

The SEI CMM requires software items under SCM
control to be properly identified:
C Configuration items must be selected based on

documented criteria.
C Configuration items must be assigned unique

identifiers.
C Characteristics of each configuration item must

be specified.
C Software baselines to which configuration

items belong must be specified.
C Points in development at which configuration

items are placed under SCM must be specified.
C Individuals responsible for configuration items

must be identified.
Software items under SCM are referred to as software

configuration items (SCIs).
CSQE 2002

Configuration Identification Elements

The National Consensus Standard for Configuration
Management defines CM identification very broadly:
C Product Information
C Product Structure
C Product Identifiers
C Document Identification
C Baselines
C Product Identification Recovery
C Interface Control
CSQE 2002

Configuration Identification Steps

Fletcher Buckley describes the process of software
identification:
C Identify and define the categories to be

managed
C Identify and define the software production
processes in each category
C Identify the types of software to be managed in
each process in each category
Software Program Identifiers

Software file identifiers are external identifiers. Typical
identifiers at the computer program level include:
C Name or acronym based on function

C Number that identifies the department, project
number, and product number
C The full record for the program contains all
above elements
CSQE 2002

Code File Identifiers

Code file identifiers are internal identifiers. Typical
identifiers have several parts including file name, file
type, and version number. Source, object, executable,
and image code for the same function would have the
same file name but different file type and version
number. Each version of each type of file would have a
unique identification.
Identification of Objects
Objects have distinct features that identify them: a
name, a description, a list of resources, and a
realization. The name is a unique character string.
The description is a list of data items that identify the
SCI type, project, and version. Resources are entities
that are used or become part of the object. The
realization is a pointer to the content of a basic object
and is not used for an aggregate object.
CSQE 2002

4. SOFTWARE BUILDS
Software Builds
The Software Build Process
A software build is a process of creating application

binaries for a software release. The primary purpose of
a software build is to enable developed items to be
processed to a usable or intermediate state such as to
compile source code into object modules. The software
build process can also be used to create test
configurations that include debug tools or emulation
interfaces to drive test builds.
There are four main inputs to the software build

process:
C The structure of the product

C The items in the product
C Environment items
C Process definitions
Each item to be processed in the build must be

specified.
CSQE 2002

4. SOFTWARE BUILDS
Software Builds (Continued)

Controlling Builds
There are various methods that can be used to control

builds. Some of them are:
C Build early and build often

C Automation of the build process
C Checking all files before build
The build early and build often version control method

helps development teams identify issues that can arise
from applying the wrong files and integration issues.
Automation of the build process will ensure that the

build process is completely repeatable and consistent
and will reduce the chances of a build with the incorrect
version of an application.
Checking all files before the build is a very common

problem even with experienced development teams.
The problem of oversight cannot be easily addressed
since it is the responsibility of the individual developer
to ensure that his or her file has been checked in.
CSQE 2002

4. SOFTWARE BUILDS
Release Process Issues

Release procedures come into play on a project once
the configuration baseline is established. Baselines are
established at specified points in the project and are
usually tied to project milestones. Configuration
baselines are used as a point of departure, where formal
control of the configuration is established.
Configuration baselines, plus the approved changes to

those baselines, constitute the current approved
configuration. After release of a newly configured
baseline, all changes are controlled.
Release procedures generally establish the mechanism

for the following:
C Identification of the initial baseline
C Changes incorporated into the new release
C Mechanisms for releasing software to all

baselines
Release procedures operate in conjunction with

configuration control boards (CCB).
CSQE 2002

C. CONFIGURATION CONTROL
1. ITEM AND BASELINE CONTROL
Configuration Control
Configuration Control is presented in the following

topic areas:
C Item and Baseline Control

C Proposed Modifications
C Review and Configuration Control Boards
C Concurrent Development
C Traceability
C Version Control
C Configuration Item Interfaces
CSQE 2002

Software Configuration Control

The main thrust of software configuration control is to
control changes to the software configuration items
(SCIs). Software configuration control involves
managing and controlling the changes to the SCIs,
baselines and software releases.
The goal of software configuration change control is to

establish mechanisms that will help in the production of
quality software.
The primary objective of configuration control is to

establish and maintain a systematic change
management process that regulates life-cycle costs,
and:
Allows optimum development latitude
Provides efficient processing and

implementation of configuration changes
Ensures accurate and timely changes to

configuration documentation
Eliminates unnecessary change proliferation

CSQE 2002

Software Configuration Control (Cont.)

The Figure below illustrates a generic change control
process for all SCIs inclusive of documentation and
code:
SOFTWARE SOFTWARE PROBLEMS
CHANGE ENHANCEMENTS
SOFTWARE
DESIGN
ANALYZE AND
BOARD
ASSESS IMPACT
REVIEW
ENGINEER-
ING CHANGE
PROPOSAL
PREPARATION
EVALUATE
ENGINEERING
CHANGE
PROPOSAL
INCORPORATE YES NO ARCHIVE

APPROVE
CHANGE CHANGE
?
SOFTWARE/CONFIGURATION
CONTROL BOARD
VERIFY SUPPLY FEEDBACK

CHANGE TO ORIGINATOR
END
Generic Change Control Process

CSQE 2002

Software Configuration Control (Cont.)

The Figure shows that the change control process is
triggered by a software change request, enhancement
request or software problem fix.
The change is analyzed by a review board to assess the

change.
The results are evaluated and documented as an

Engineering Change Proposal. This is used as input
into the configuration control board (CCB) or software
configuration control board (SCCB). The required
content of the engineering change proposal should be
identified in the SCM plan.
If the scope of the change is system wide, the CCB

reviews the change. If the scope of the change is
restricted to software, then the software configuration
control board (SCCB) makes a final decision to
determine the status and priority of the requested
change.
CSQE 2002

2. PROPOSED MODIFICATIONS
Methods for Assessing

Proposed Modifications
All proposed modifications, enhancements, or additions
should be assessed to determine the effect that each
change will have on the system.
When changes are proposed by users to add

functionality, the impact of the proposal should be
analyzed to determine which requirements, design, and
code would be changed and to what degree. The
proposed changes should be evaluated to:
C Clarify an ambiguous or poorly worded

proposal
C Reveal the true extent of a change
C Help management determine which changes to

implement
C Determine how the change fits into the existing

or desired system
C Help management establish a schedule

CSQE 2002

Formality of the Change Control Process

Development Baseline
The development baseline is another commonly used

baseline which defines the state of the system at a
specific point in time, usually relative to an integration
level. It serves to synchronize the engineering activities
and the associated documentation that occur at the time
the system is ready to be promoted to the integration
level.
The term promoted refers to a release or transition

from the state of informal individual engineering (ad
hoc) level development, to a more formal (controlled)
development baseline.
For example, the programmer can no longer change a

program that has been released and integrated with
other programmers units, without notifying the others
involved and gaining their approval to add his/her
changes to the development baseline by way of the
software configuration control board.
CSQE 2002

Impact Assessing Techniques

Configuration control is the systematic process for
evaluating, coordinating, disposition of proposed
changes, and tracking the implementation of approved
changes to baselined code and documentation. The
change control process ensures that changes are
initiated, classified, evaluated, approved or
disapproved, documented, implemented, tested and
incorporated in the new baseline. This change control
process should be documented in the software
configuration management (SCM) plan.
An orderly change process is necessary to ensure that

only approved changes are implemented into any
baselined documents or software. The change process
steps contained in the SCM Plan:
C Change Initiation
C Classification
C Change Evaluation
C Change Dispositioning
C Change History
C Implementation
C Verification
C Baseline Change Control
CSQE 2002

3. REVIEW AND CONFIGURATION BOARDS
Configuration Control Board

The configuration control board (CCB) [or change
control board (CCB)] is usually at the program, product
or system level and includes all aspects of the product
such as hardware, firmware and software.
The CCB controls issues such as schedule, resources,

function, cycle time and configuration of the system as
a whole, while the SCCB is involved in reviewing issues
related to software and documentation.
Software Configuration Control Board (SCCB)
The more technical issues, that do not relate to

performance, cost, schedule, etc. are often assigned to
the software configuration control board (SCCB) [or
software change control board (SCCB)].
The SCCB discusses issues related to specific

schedules for software functions, common data
structures, software design changes and the like.
CSQE 2002

4. CONCURRENT DEVELOPMENT
Concurrent Development
Concurrent Process Model
Pressman defines a concurrent process model that

includes major technical activities, tasks, and their
associated states. All activities exist concurrently but
reside in different states. Events defined in the model
trigger transitions from state to state for each of the
software engineering activities. The concurrent process
model is a network of activities at the system and
component levels.
Concurrent Development Processes
Concurrent changes are changes made by two or more

developers to the same file at the same time. This may
occur among developers working on the same release
or among developers working on different releases of
the same system.
Concurrent development is characterized by common

use of system elements in parallel. SCM must deal with
concurrent changes and contention among developers
who are working on the same software elements.
CSQE 2002

5. TRACEABILITY
Traceability
In Handbook of Software Quality Assurance, traceability
is defined as the linkage between project events.
Pressman says that the progression of the software

process creates a rapid growth of software
configuration items (SCIs). Hence, a hierarchy of
information is created and, there is rarely a one-to-one
correspondence between SCIs. Change that occurs at
any time and for any reason causes confusion.
Therefore, the primary responsibility of SCM is to
control change.
Pressman describes an object-oriented approach for

controlling and managing SCIs. (documents, programs,
and data). Each object has a set of distinct features that
identify it uniquely: name, description, list of resources,
and realization.
In a software development process, the evolution of

software should be visible, traceable and formally
controlled. Software configuration management
provides it by the integrated application of the four
functions - identification, control, auditing, and status
accounting.
CSQE 2002

5. TRACEABILITY
Traceability (Continued)
Configuration management helps raise the visibility of
software projects and infuses them with traceability
through:
C Identification
C Control
C Auditing
C Status Accounting
The following are some of the methods used for

establishing and maintaining traceability of
configuration items (CI).
C Version marking
C Labeling of documentation
C Labeling of executable software
C Project codes
C Project type designators and nomenclature
C Part/item identification numbers
C Firmware labeling
CSQE 2002

6. VERSION CONTROL
Version Control
Baseline Change Control
In order to minimize the number of versions and

frequency of delivery of software components, changes
to software are usually grouped into releases to the
baselines established for the project in the SCM plan.
Most large software systems are developed in

evolutionary baselines and releases. For example,
when one release is in customer use, another is in test
and a third in development, bug fixes must be
propagated between them.
The SCM plan should identify the change control

procedures for all releases and baselines which are
used to arbitrate these issues.
CSQE 2002

7. CONFIGURATION ITEM INTERFACES
Configuration Item Interfaces

Types of Interfaces

defines the following types of interfaces.
C Organizational - between projects and other

organizations
C Phase - between groups that transfer control
C Software - between software programs to other

software and data
C Hardware - between software programs and

infrastructure
Interface Control
Interface control is a matter of defining interface

characteristics and maintaining configuration control
over the characteristics. The interfaces to be defined
are between high-level CIs which may be two SCIs or
may be one SCI and a hardware item.
CSQE 2002

Configuration Item Interfaces (Cont.)

Organizational Interfaces

Management deals with interfaces between projects and
outside organizations. Product specifications should
include interface definitions for the products supplied
by the outside organizations and should include
performance, functional, and physical attributes.
Phase Interfaces
Phase interfaces involve transfers of products when

they complete a defined phase of a project. Control of
the product passes from one group to another within
the project.
CSQE 2002

Configuration Item Interfaces (Cont.)

Software Interfaces
Software interfaces involve the compatibility of

computing functions and the structure and meanings
assigned to data passing between software programs.
Hardware Interfaces
Hardware interfaces involve the compatibility of

software and hardware functions and instructions.
Software must be able to run on target platforms or

control hardware in which it is embedded.
CSQE 2002

D. CONFIGURATION STATUS ACCOUNTING
1. STATUS REPORTING
Configuration Status Accounting
Configuration Status Accounting is presented in the

C Status Accounting and Reporting

C Changes to Configuration Items and
Baselines
C Documentation Control
CSQE 2002

1. STATUS REPORTING
Status Accounting and Reporting

Overview
The term configuration status reporting (sometimes

called status accounting) is an SCM task that may be
viewed as an accounting system. Many of the concepts
may be used to track the flow of software through its
evolution.
Separate accounts can be established for each software

configuration item (SCI). Individual transactions can
then be tracked through each account as these occur.
The configuration status accounting function, at a
minimum, is essentially reporting the transactions
occurring between SCM-controlled entities.
Formally, configuration status accounting is defined as

an element of configuration management, consisting of
recording and reporting of information that is needed to
manage a configuration effectively.
CSQE 2002

1. STATUS REPORTING
Status Accounting and Reporting (Cont.)

The Figure below is an IDEF0 visual description of the
tasks involved in the status accounting process.
Additional information on the IDEF family of methods
may be found at web site http://www.idef.com.
Status Accounting Tasks

CSQE 2002

1. STATUS REPORTING
Status Accounting and Reporting (Cont.)

Purpose
The purpose of software configuration status

accounting is to maintain continuous records of the
status of all baselined items. This record is not only a
useful management tool, but also provides valuable
insurance against disaster and a mechanism for
disaster recovery. The information required for
comprehensive status accounting includes:
C The time at which baselines were established

C When each software configuration item was
included in the baseline
C When each change was added to the baseline
C A description of each configuration item
C Status of each software related engineering
change
C A description of each software change
C Documentation status for each baseline
C Changes planned for each future baseline
CSQE 2002

1. STATUS REPORTING
Needed Information (Continued)

The Figure describes the evolution of status accounting
over the configuration items (CI) life cycle development
phases.
Status Accounting Evolution
For a detailed SCM record of change activity the

following items will generally be needed:
C An SCI index
C Change logs
C All discrepancy reports
C The interdependencies of the SCIs
C All change requests
CSQE 2002

1. STATUS REPORTING
Status Accounting Planning

Status accounting reports need to be addressed in
detail in the software configuration management plan.
These are:
C Type of information that is needed to be

reported
C Degree of controls on status reporting required

by the customer
C Status reporting standards, both internal and

customer driven
Software Configuration Audit

A software configuration audit should periodically be
performed to ensure that the SCM practices and
procedures are rigorously followed. This audit can be
performed directly by the SCM staff or an independent
software quality engineering function. Procedures for
software configuration audits are defined in the SCM
plan.
CSQE 2002

1. STATUS REPORTING
Software Configuration Audit (Cont.)

Project phase reviews are defined by dividing the
project into appropriate phases depending on
parameters such as project size and risks.
The following phase reviews are key points in the

projects, and are appropriate points for performing SCM
audits.
C Requirements
C Functional
C High-level design
C Design
C Product
C Operational
CSQE 2002

2. CHANGES TO CONFIGURATION ITEMS
Changes to Configuration Items

and Baselines
How Baselines Relate to the Change Control
Baselines are effective mechanisms for allowing the

project team to work together at the same time and still
be able to manage changes during the software
development process. The SCM discipline focuses its
change control activities around the construction and
maintenance of baselines.
Once a baseline is established on a project, the baseline

plus approved changes to the baseline, constitute the
current configurations identification of the group of
SCIs associated with that baseline.
CSQE 2002

Types of Baselines
A baseline is a relative arrangement of software parts as

embodied in a software product. While baselines are
relative there is some common usage of baselines in
various sectors of the software industry as follows:
C Functional Baseline - The Initial approved

functional configuration
C Allocated Baseline - The initial approved

allocated configuration
C Product Baseline - The initial approved or

conditionally approved product configuration
identification.
CSQE 2002

Change Control Process

Change control is a two-stage process as shown in the
Figure. A change is presented to the change control
authority in the first stage and, if approved, is
implemented by developers in the second stage.
Change is recognized
and requested
by user.
Change control
authority
decides.
Change request is Change request is

denied, user is approved, developers
informed. are assigned.
Developers check
out CIs, make
change, check in CIs.
SCM establishes
baseline for testing,
testing is performed.
Changes accepted for

next release, versions
are rebuilt, audited.
Changes included in
new version, new
version distributed.
A Change Control Process

CSQE 2002

Change Control Process (Continued)

A change request is submitted and evaluated. The
results of the evaluation are presented as a change
report to the change control authority. If the change is
approved an engineering change order is generated that
describes the change, the constraints that apply, and
the criteria for review and audit. The engineering
change order is assigned to developers for
implementation.
Control of Software Libraries

Options for controlling changes to software libraries:
C Formal control of configuration items at the

component level
C Informal control of configuration items at the

component level
C Formal control of external computer programs

at the CI levels and internal computer programs
at more discrete component levels
CSQE 2002

Control of Software Libraries (Cont.)

Formal control involves change request approval by the
CCB or equivalent change approval authority. Informal
approval involves change request approval by
development managers.
SCR and SCA Procedures
The software change request (SCR) is used to identify

and coordinate changes to software and documents.
The SCR is approved, disapproved, or tabled by the
CCB. If approved the change is assigned to the
software development organization.
The software change authorization (SCA) is used to

control changes to all documents and software in the
SCM library.
SCR and SCA records are maintained in the status

accounting system.
CSQE 2002

3. DOCUMENTATION CONTROL
Documentation Control
Identification, Control, and Status Accounting
includes documentation identification, control, and
status accounting as part of software configuration
management. Software documents are identified by
SCM, released to SCM, and maintained in a controlled
library by SCM. SCM performs document distribution
and administers the document change control process.
The SCR is initiated by the developer, reviewed by the

development manager, and approved by the CCB. The
SCA is prepared by the developer, approved by the
development manager, and implemented by SCM.
SCM implements changes to documents, performs

status accounting, and provides current versions of
documents to support configuration audits.
The trend today is for documentation and software to be

created and maintained in electronic form. Project
libraries contain specifications, design documents, test
plans and procedures, as well as source code. Some
organizations maintain hard copy masters of product
documents for legal reasons.
CSQE 2002

3. DOCUMENTATION CONTROL
Digital Data

Management defines requirements for configuration
management of digital data. Digital data is information
prepared and maintained by electronic means.
Digital Data Life Cycle
A digital data life cycle model is shown below. The

model identifies data status or promotion levels that are
used to manage access, change, and archiving of
product data.
Data Life Cycle Model

CSQE 2002

E. CONFIGURATION AUDITS
1. FUNCTIONAL CONFIGURATION AUDIT
Configuration Audits
Configuration Audits is presented in the following

topic areas:
C Functional Configuration Audit
C Physical Configuration Audit

CSQE 2002

1. FUNCTIONAL CONFIGURATION AUDIT
Functional Configuration Audit

The functional configuration audit provides an
independent evaluation of configuration items, to
determine whether actual functionality and performance
are consistent with the requirement specifications.
Specifically, this audit is held prior to the software

delivery to verify that all requirements specified in the
software requirements specification have been met.
Activities typically planned and executed as part of a

functional configuration audit include:
C Evaluation of earlier verification and validation

efforts
C Testing of product
C Evaluation of the test approach and results
attained
C Tracing requirements from initial specifications
through systems testing
C Evaluating the consistency between baselined
product elements
CSQE 2002

2. PHYSICAL CONFIGURATION AUDIT
Physical Configuration Audit

The physical configuration audit provides an
independent evaluation of whether components, in the
as-built version of the software, map to the
specifications of the software.
Specifically, this audit is held to verify that the software

and its documentation are internally consistent, and are
ready for delivery. Activities typically planned and
executed as part of the physical configuration audit
include evaluation of:
C Product composition and structure

C Product functionality
C Change control
The team will examine the documented baseline and

identify and verify the configuration of the product in
terms of its subordinate units. The physical existence
of the configuration is then compared to its documented
baseline in regards to standards for markings,
nomenclature and packaging.
CSQE 2002

F. RELEASE AND DISTRIBUTION ISSUES
1. PRODUCT RELEASE ISSUES
Release and Distribution Issues
Release and Distribution Issues is presented in the

C Product Release Process Issues
C Packaging, Production, and Distribution

CSQE 2002

Product Release Process Issues

The single entity, identified as a configuration item (CI)
is the aggregate of hardware, software, processed
materials, and services associated with all of the
discrete elements of a software build.
The configuration management (CM) process must

provide for a high level of assurance that this end item
meets the objectives of defined requirements, and is
delivered as a fully functional and integrated unit, stable
within its specified environment.
CSQE 2002

Required Tasks Prior to Formal Release
The formal release phase, commonly referred to as

General Availability is dependent on:
C System and Integration Testing
C Configuration Audits
C Beta Testing
C Documentation
The focus of these tasks is to uncover issues not

previously found in a test lab environment.
CSQE 2002

Post Release
The final release of the product requires a method of

tracking versions to a database of customers. This
could involve serialization of the product traceable to
the CI build status.
The product is released with a:
C Post Pack Audit
C Version Description Document (VDD) (also

called Release Notes by some groups)
C Hardware/Software Compatibility
C Problem Capture and Tracking
C Patches and Interim Updates

CSQE 2002

End of Life (EOL) Planning
The initial business case defines which features can be

delivered to a specific market. This phase also focuses
on the probable life span of the product and the method
for replacing it in the future.
As the EOL approaches, customers are informed of the

replacement product, the final support time line for the
current product, and the possible hardware/software
compatibility issues that might ensue with the
replacement product.
CSQE 2002

2. PACKAGING, PRODUCTION AND DISTRIBUTION
Packaging, Production and Distribution

A software production process is:
C The input can be source files or other software

that is to be electronically processed to provide
a stated output.
C The software production environment includes

tools that transform the inputs into outputs.
C The output of the production process can be

software on electronic or optical media, hard
copy of documents, or both.
The firmware production process consists of four steps:
C Source files are assembled into object files
C Object files are linked to form absolute files (or

executable files)
C Absolute files are formatted into image files
C Image files are embedded into the firmware part

by the profiling process (or programming
process)
CSQE 2002

Packaging
Packaging software and documentation follows the
same principles that apply to packaging hardware:
C Assuring that the current version of all items

(configuration) are the correct version
C Packaging standards
C Determine if packaging will be performed

internally or externally
C Determine level of in-process and final

inspection
C Source of packaging material
C Define kitting, assembly and storage processes

CSQE 2002

Production
Production can be taken in two contexts. The first is the
release of software developed for a specific application.
The second type of production is the physical creation
of a product that might include a CD-ROM,
documentation, marketing material, and packaging.
To deliver commercial software, a master or golden

CD-ROM is created from built and staged software
objects that have passed system testing. The master
is sent to manufacturing where it is mass-produced,
packaged with documentation, shrink-wrapped, and
shipped to the customer.
In the CD-ROM replication process the program is

stamped (often called the glassmaking process) onto
clear polycarbonate material, to which is added a
metallic, reflective layer (for the CD laser to read), and
then the CD is lacquered with a protective coating.
Software for embedded systems is downloaded directly

to the target devices or is used to create a master CD-
ROM or chip. The master is used to mass-produce
chips that are installed in the final hardware devices.
CSQE 2002

Distribution
Distribution of software and related documentation can

be accomplished in a number of ways. The most
familiar is as a CD-ROM. Others include via a web site
[customer portals or FTP (File Transfer Protocol)] or on
a more local basis via a read-only common server.
CSQE 2002

QUESTIONS
9.5. Which of the following is NOT one of the four major areas
of configuration management ?
a. Configuration identification
b. Configuration audits
c. Tool utilization
d. Status accounting and reporting
9.7. Version control activities account for:
I. Change to documentation
II. Fixes initiated through problem reports
III. Status accounting
a. I and II only c. II and III only

b. I and III only d. I, II and III
9.8. Two of the primary goals of Configuration Identification

are:
I. Only releasing master copies to software developers

II. Providing for component identification throughout
the life cycle
III. Establishing factors for the mythical man month
IV. Providing traceability between software and related
SCIs
a. I and IV only c. I and III only

b. II and III only d. II and IV only
Answers 9.5 c, 9.7 d, 9.8 d

CSQE 2002

QUESTIONS
9.11.The reason that tools must be under configuration

control is:
a. Future modifications to the software must use the
initial tool version in order to assure compatibility of
implemented changes
b. To provide a quick check on tool upgrade status
c. To establish a link with the tool representative
throughout the modification process
d. To provide status reports to upper management
9.12.An accepted definition of a software baseline is:

a. A listing of all the CIs on a deliverable tape
b. A relative arrangement of software parts as embodied
in a software product
c. A version description document
d. An annotated file description
9.21.Which of the following would be typical audit questions of

the CM review process?
I. Have the SCM procedures been followed?
II. Have all related SCIs been updated to reflect the
change?
III. Has a formal technical review been conducted to
assess correctness?
IV. Have additional modifications been made which were
outside of the initial scope?
a. I, II, III and IV c. I, II and IV only

b. I, II and III only d. II, III and IV only
Answers 9.11 a, 9.12 b, 9.21 a

CSQE 2002

QUESTIONS
9.24.The definition of a Configuration Item is:
a. The lowest level of a software program

b. Documentation reflecting the developed product
c. Any hardware or software item treated as a unit
d. Items that are uniquely tracked outside of the CM
process
9.25.The term Configuration Identification implies which of

the following?
I. A defined audit trail of the developed product

II. A scheme for identifying CM items
III.Identification of the version of the product in
development
IV. Documentation that establishes the baseline product
a. II, III and IV only c. I, III and IV only

b. IV only d. III and IV only
9.29.Verifying that all the components in a software release

and its documentation are in agreement is called a:
a. Physical configuration audit

b. User documentation audit
c. Functional configuration audit
d. Customer conducted audit
Answers 9.24 c, 9.25 a, 9.29 a

CSQE 2002

QUESTIONS
9.33.Which of the following is the most accurate statement

about management control of SCI interfaces?
a. It is accomplished by defining and controlling
software to software interfaces
b. It is accomplished by defining and controlling
software to software and software to hardware
interfaces
c. It is accomplished by defining and controlling
interfaces and by coordinating phase transfers
d. It is accomplished by defining and controlling
interfaces and by coordinating phase transfers and
concurrent activities
9.36.The Configuration Control Board (CCB) has overall

responsibility for:
I. Maintaining product life cycle processes
II. Managing the definition of the evolving product
III. Reporting to management any changes in supporting
documentation
a. I only c. II and III only

b. II only d. I, II and III
9.39.Customer change requests should be:

a. Immediately honored, showing a desire to please the
customer
b. Referred to upper management for approval
c. Placed into the next release of the product
d. Assessed by a change review board and assigned a
priority
Answers 9.33 d, 9.36 b, 9.39 d

CSQE 2002 001 Transp S

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CSQE 2002 001 Transp S

Uploaded by

Copyright:

Available Formats

QUALITY COUNCIL OF INDIANA INTRO-1 (1)

2002 by Quality Council of Indiana

All rights reserved

CSQE Primer Contents

II. GENERAL KNOWLEDGE, CONDUCT AND

III. SOFTWARE QUALITY MANAGEMENT . . . . . . . III-1

IV. SOFTWARE AUDITS . . . . . . . . . . . . . . . . . . . . . . IV-1

V. SOFTWARE ENGINEERING PROCESSES . . . . . V-1

VI. PROGRAM & PROJECT MANAGEMENT . . . . . VI-1

RISK PROBABILITY . . . . . . . . . . . . . . . . . VI-53

VII. SOFTWARE METRICS . . . . . . . . . . . . . . . . . . . VII-1

VIII. SOFTWARE VERIFICATION & VALIDATION VIII-1

TEST ENVIRONMENTS . . . . . . . . . . . . . VIII-53

IX. SOFTWARE CONFIGURATION MANAGEMENT IX-1

CSQE Primer Question Contents

II. General Knowledge,

III. Software Quality

IV. Software Audits 6.25% 10* 25 63

VI. Program & Project

VII. Software Metrics 15.0% 24 60 150

VIII. Software Verification

IX. Software Configuration

Professionalizing Quality Education

HENRY DAVID THOREAU

To provide recognized software quality engineer

Eligibility entails a combination of eight years work

The CSQE examination lasts 4 hours and begins at

Contact ASQ at (800) 248-1946 or

CSQE Exam (Continued)

The CSQE student should obtain the bibliography

Humphrey, W. (1990). Managing the Software Process.

ISO 9000-3-1997, Quality Management and Quality

ISO 10006:1997 Quality Management - Guidelines to

ISO 10007:1995 Quality Management - Guidelines for

Kan, S. (1995). Metrics and Models in Software Quality

Kaner, C., Falk, J., & Nguyen, H. (1999). Testing

CSQE Exam (Continued)

Pressman, R.S. (1992). Software Engineering: A

Pressman, R. (1993). A Manager's Guide to Software

Schulmeyer, G. G., & McManus, J. I. (1999). Handbook

Additional references are listed at the end of each

ASQ Certified Software Quality Engineer

I. General Knowledge, Conduct, and Ethics

II. Software Quality Management (30 Questions)

III. Software Engineering Processes (26 Questions)

IV. Program and Project Management (24 Questions)

V. Software Metrics, Measurement, and Analytical

VI. Software Verification and Validation (V&V)

VII. Software Configuration Management

Six Levels of Cognition

II. GENERAL KNOWLEDGE, CONDUCT AND ETHICS

SIR HERBERT SAMUEL

II. GENERAL KNOWLEDGE, CONDUCT AND ETHICS

C Quality Philosophy and Principles

Quality Philosophy and Principles

C Benefits of Software Quality

II. GENERAL KNOWLEDGE, CONDUCT AND ETHICS

Benefits of Software Quality

Software quality: The conformance to explicitly stated

The simplest definition is a generalization given by

It is important to note that both definitions cover

Two other definitions of quality are:

C The degree to which a system, component, or