Software Engineering New1 1062217755 2

1
Software Engineering
Lecture 1
Introduction to Software Engineering
Code of Conduct
Software Engineering is a collaborative activity. You are
encouraged to work together, but ...
Some tasks may require individual work.
Always give credit to your sources and collaborators.
Good professional practice: To make use of the expertise of others
and to build on previous work, with proper attribution.
Unethical and academic plagiarism: To use the efforts of others
without attribution.
Projects
Project teams, about 3 to 5 peoples.
Select your own project, any branch of software engineering
Real project for real client who intends to use the software in
production.
Feasibility study and plan
Presentations:
requirements
design
final
2
Project Selection
Some suggested projects
Recitation section to suggest projects
Contact potential clients:
Gain idea of their expectations
Estimate scope and complexity of the project
Discuss business decisions
Assemble project team
Advertise on the web site
Previous Experience
Biggest program that you have written?
Biggest program that you have worked on?
Biggest project team that you have been part of?
Longest project that you have worked on?
Most people who have used your work?
Longest that your project has been in production?
Course Themes
1. Leadership of large software projects
Software as a product
Clients and their needs
Quality
Requirements and specification
Usability
Evolution
Project management
Personnel management
Economic, legal, and social factors
3
Course Themes
2. Large and very large systems
Software design
Software architecture
Object-oriented design
Dependable systems
Reliability
Verification
Legacy systems
Characteristics of Software Products
General characteristics
Usability
Maintainability
Dependability
Efficiency
Good software products require good programming,
but ...
Programming quality is the means to the end, not the
end itself.
Software as a Product
Software is expensive!!
Every software project has a trade-off between:
Functionality
Resources (cost)
Timeliness
Example:
Accounting Management System
4
Client (a.k.a Customer)
The client provides resources and expects some product
in return.
Client satisfaction is the primary measurement of
success.
Question: Who is the client for Microsoft Excel?
Variety of Software Products
Examples?
-Operation System
-Database Management System
-Embedded System
-Games
-Application Software
-
Categories of Product
Categories of client and software product:
Generic (e.g., Microsoft Excel)
Bespoke (customized) (e.g., IRS internal
system)
Many systems are customized versions of generic
packages (e.g., Cornell's payroll system)
5
Variety of Software Products
Software products are very varied
--> Client requirements are very different
--> There is no standard process for software engineering
--> There is no best language, operating system, platform,
database system, development environment, etc.
A skilled software developer knows about a wide variety of
approaches, methods, tools. The craft of software engineering
is to select appropriate methods for each project and apply them
effectively.
Professional Responsibility
Organizations put trust in software developers:
Competence: Software that does not work effectively can
destroy an organization.
Confidentiality: Software developers and systems
administrators may have access to highly confidential information
(e.g., trade secrets, personal data).
Legal environment: Software exists in a complex legal
environment (e.g., intellectual property, obscenity).
Acceptable use and misuse: Computer abuse can paralyze an
organization (e.g., the Internet worm).
Lecture 2
The Software Process
6
Books
Frederick P. Brooks, Jr. The Mythical Man Month.
Addison-Wesley, 1972.
Ian Sommerville, Software Engineering, 6th edition.
Addison-Wesley, 2000.
Grady Booch, James Rumbach, Ivar Jacobson, The
Unified Modeling Language. Addison-Wesley 1999.
Software Process
Fundamental Assumption:
Good processes lead to good software
Good processes reduce risk
Risk Management
What can go wrong in a software project?
How can the risk be reduced?
7
The Software Process (Simplified)
Requirements
Operation and
Maintenance
Implementation
Design
Feasibility and
Planning
The Waterfall Model
Requirements
Definition
System and
Software design
Programming
and Unit Testing
Integration and
System Testing
Operation and
Maintenance
Requirements Analysis and Definition
The system's services, constraints and goals are established by
consultation with system users. They are then defined in a
manner that is understandable by both users and development
staff.
This phase can be divided into:
Feasibility study (often carried out separately)
Requirements analysis
Requirements definition
Requirements specification
8
System and Software Design
System design: Partition the requirements to hardware or
software systems. Establishes an overall system
architecture
Software design: Represent the software system functions
in a form that can be transformed into one or more
executable programs
Unified Modeling Language (UML)
Programming and Unit Testing
The software design is realized as a set of programs or
program units. (Written specifically, acquired from
elsewhere, or modified.)
Individual components are tested against specifications.
Integration and System Testing
The individual program units are:
integrated and tested as a complete system
tested against the requirements as specified
delivered to the client
9
Operation and Maintenance
Operation: The system is put into practical use.
Maintenance: Errors and problems are identified and
fixed.
Evolution: The system evolves over time as
requirements change, to add new functions or adapt the
technical environment.
Phase out: The system is withdrawn from service.
Discussion of the Waterfall Model
Advantages:
Process visibility
Dependence on individuals
Quality control
Cost control
Disadvantages:
Each stage in the process reveals new
understanding of the previous stages, that
requires the earlier stages to be revised.
Feedback in the Waterfall Model
Requirements
Definition
System and
Software design
Programming
and Unit Testing
Integration and
System Testing
Operation and
Maintenance
10
Iterative Refinement
(Evolutionary Development)
Concept: Initial implementation for user comment, followed
by refinement until system is complete.
Vaporware: user interface mock-up
Throw-away software components
Dummy modules
Rapid prototyping
Successive refinement
Requirements
Design
Implementation
(prototype)
Evaluation
Outline
Description
Concurrent
Activities
Requirements
Design
Implementation
Initial
Version
Intermediate
Versions
Final
Version
11
Iterative Refinement & Software
Process
Outline
Description
Concurrent
Activities
Requirements
Design
Implementation Final
Version
When is iterative refinement appropriate?
Iterative Refinement + Waterfall Model:
Graphics for Basic
Outline Description: Add vector graphics to
Dartmouth Basic.
Phase 1: Extend current language with a preprocessor
and run-time support package. (1976/77)
Phase 2: Write new compiler and run-time system
incorporating graphics elements. (1978/80)
12
Graphics for Basic
Design Issues:
Pictorial subprograms: coordinate systems, window/viewport
User specification of perspective
Design Strategy: (Iterative Refinement)
Write a series of prototypes with various proposed semantics
Evaluate with a set of programming tasks
Graphics for Basic
Phase 1: Implementation (Waterfall)
When the final specification was agreed, the entire
preprocessor and run-time support were recoded.
The system was almost entirely bug-free.
Phase 2: New compiler (Waterfall)
Phase 1 was used as the requirements definition for the
final version.
Observations about Software Processes
Completed projects should look like the Waterfall Model
but ... the development process is always partly evolutionary.
Risk is lowered by:
Prototyping key components
Dividing into phases
Following a visible software process
Making use of reusable components
13
Lecture 3
(a) Feasibility Study
(b) Requirements Definition
Feasibility Study
Before beginning a project, a short, low-cost study to identify
Client
Scope
Potential benefits
Resources needed:
staff, time, equipment, etc.
Potential obstacles
Where are the risks? How can they be minimized?
Feasibility Study
A feasibility study leads to a decision:
go ahead
do not go ahead
think again
In production projects, the feasibility study
often leads to a budget request.
In research, a feasibility study is often in the
form of a proposal.
14
CS 501: Client
In CS 501, you have two clients:
The client for the project
The professor for the course
Can you satisfy them both?
Scope
What are the boundaries of the project?
CS 501 Examples:
Static web pages with open access on the Web [Web Profiler]
Used by the general public [Digital Collections]
Varying data formats [Legal Information]
Thousands of sensors [Data mining]
Support for Windows, Mac, Unix [SALSA]
Potential Benefits
Why are you doing this project?
Examples
Create a marketable product
Improve the efficiency of an organization
Control a system that is too complex to control manually
New or improved service
Safety or security
Get a good grade on CS 501
15
Resources
Examples: CS 501
Staff: 5 to 7 students, with some help. How many hours per week?
What skills do people have?
Time: Must be completed by end of semester, including
operational system, documentation, presentation
Equipment and software: What special needs are there?
Client: Will the client be sufficiently available and helpful?
Obstacles
CS 501 projects
Start-up time. Creating a team, scheduling meetings, acquiring
software, learning new systems, ...
Business considerations. Licenses, trade-secrets, ...
Too ambitious. Nothing to show at the end of the semester.
Changing circumstances. Client leaves the university, ...
What else?
How to Minimize Risk?
CS 501 Projects
Several target levels of functionality:
required, desirable, optional
Visible software process: intermediate deliverables
Good communication within team and with
Teaching Assistant
Good processes reduce risk
16
Feasibility Report
A written document
For a general audience: client, financial management,
technical management, etc.
Short enough that everybody reads it
Long enough that no important topics are skipped
Requirements Definition and Analysis
Requirements
Definition
System and
Software design
Programming
and Unit Testing
Integration and
System Testing
Operation and
Maintenance
Example: Library of Congress
(A Partial Failure)
Outline Description
The Library of Congress requires a repository system
to store and make accessible very large amounts of
highly varied material over long periods of time.
17
Chronology
1993-94 CNRI carries out research on architectures for
digital libraries
1995-97 CNRI implements prototype repository for
Library of Congress
1998 CNRI and Library of Congress carry out requirements
definition
Repository
Identification System Search System
Users
The Repository
Storage and Representation of Complex
Objects
Data
Several representations:
thumbnail image
reference image
archival image
Metadata
Each representation may
have its own metadata
18
Repository: Research Achievements
1. CORBA implementation of repository access protocol.
2. Integration of persistent naming through handle system.
3. Use of structural metadata to describe complex objects,
elementary typology.
4. Access management framework and implementation.
5. Applet-based middleware for user interfaces.
6. Information visualization program to view the structure of
large collections.
Good Discoveries During Prototype
Structuring complex information in digital libraries
Data driven digital library interfaces
Comparison of object-oriented, relational, and file
based storage systems
Naming and identification of library objects
Boundaries of required repository system
Bad Discoveries During Prototype
Resistance to change within Library of Congress
Technical weakness of Library of Congress
Gaps in CNRI architecture
19
Mistakes
Confusion of objectives (research and implementation)
Failure to involve all stakeholders
Over-ambitious (no proper feasibility study)
The Requirements Process
Feasibility
Study
Requirements
Analysis
Requirements
Definition
Requirements
Specification
Feasibility
Report System
Models
Definition of
Requirements
Specification of
Requirements
Requirements
Document
Requirements Definition
High-level abstract description of requirements:
Specifies external system behavior
Comprehensible by customer, management and users
Should reflect accurately what the customer wants:
Services that the system will provide
Constraints under which it will operate
20
Library of Congress Requirements Study
Team (all experienced): Librarian, Software Engineer (CNRI),
Computing Project Leader (Library of Congress), + 2 others
Advisors: Mailing list of about 20 knowledgeable stakeholders.
Timetable: Preliminary report (2 months). Final report (1 month).
Functional Requirements
Example: Library of Congress repository
Support for complex digital objects
Access management
Identification
Information hiding
Open protocols and formats
Integration with other systems (scope)
Current Storage Structure
(in Unix files, by aggregate)
Index Generation
(including pre-processing)
American Memory User Interface
(retrieval, navigation, & display)
Object Administration System
Repository
NDLPWorkflow
Tracking Support
Handle-server
NDLPcollections
already released
NDLPcollections
in conversion
Coolidge collection
(for repository test)
Future
NDLPcollections
NOW FUTURE
ILS OPAC Interface Other User Interfaces
(e.g. RLG, OCLC, DLF partners)
Other applications
and materials
ILS
Handle assignment
&registration Handle resolution
Supporting infrastructure
DRAFT OVERVIEW OF ITS SUPPORT
FOR NDLP PRODUCTION AND DELIVERY OF AMERICAN MEMORY
AM user interface plus
access management
for objects/collections
21
Non-functional Requirements
Environment:
Estimates of sizes, numbers of users, etc.
Reliability and performance measures and targets
Preferred:
Example: Library of Congress repository
Hardware and software systems (e.g., IBM/Unix)
Database systems (e.g., Oracle)
Programming languages (e.g., C and C++)
Evolution of Requirements
If the requirements definition is wrong, the system will be
a failure.
With complex systems, understanding of requirements
always continues to improve.
Therefore...
The requirements definition must evolve.
Its documentation must be kept current (but clearly
identify versions).
Lecture 4
Management I: Project Management
22
The Aim of Project Management
To complete a project:
On time
On budget
With required functionality
To the satisfaction of the client
Without exhausting the team
The Project Manager
Create and maintain the schedule
Should track progress against schedule
Keep some slack in the schedule
Be continually making adjustments:
Start activities before previous activity complete
Sub-contract activities
Renegotiate deliverables
Keep senior management informed
Project Planning Methods
The Critical Path Method, Gantt charts, Activity bar charts, etc.
are roughly equivalent.
These methods are best when:
Model is updated regularly (e.g., monthly)
The structure of the project is well understood
The time estimates are reliable
Activities do not share resources
[Critical Path Method is excellent for large construction
projects.]
23
Example: An Open University Course
Deliverables:
16 Written texts (bound in pairs)
8 Television programs
8 Radio programs
4 Computer programs
1 Home experimental kit (scientific calculator)
4 Assignments and sample solutions
Flexibility
Schedule: Dates for broadcasting TV and radio programs are
fixed. Printing and mailings can be accelerated if overtime is
used.
Functionality: The course team can decide what goes into the
components of the course.
Resources: The size of the course team can be increased
slightly.
Scheduling: Critical Path Method
An activity
A dummy activity
An event
A milestone
24
Critical Path Method
Edit
Unit 3
Print
Unit 3
Revise
Unit 3
Mail
Unit 3
other
activities
START
END
Edit
Unit 3
Typeset
Unit 3
Revise
Unit 3
Mail
Units 3/4
other
activities
Edit
Unit 4
Print
Units 3/4
Revise
Unit 4
other
activities
Typeset
Unit 4
START
START
Edit
Unit 3
Script
TV 2
Make
TV 2
Edit
Unit 4
Prototype Computer 1
Program
Computer 1
Document
Computer 1
Mail
Delivery
25
Time Estimates for Activities (Weeks)
6
4
2
2
3
3
1
3
3
8
2
1 1
4
12
12
1
4
Earliest Start Dates
6
4
2
2
3
3
1
3
3
8
2
1 1
4
12
12
1
4
0
1
12
12
4
15 17
17
17
19
22
26
23 25
Latest Start Dates
6
4
2
2
3
3
1
3
3
8
2
1 1
4
12
12
1
0
11
12
14
13
15 17
17
17
20
23
26
24 25
4
26
Critical Path
0/0
1/11
12/12
12/14
4/13
15/15 17/17
17/17
17/17
19/20
22/23
26/26
23/24
25/25
Slack
0/0
1/11
17/17
23/24
10
5
0
12/12
12/14
4/13
15/15 17/17
17/17 19/20
22/23
26/26
25/25
10
0
0
0
0 0
0
2
2
9
9
1
1
3
1 1
Key Personnel
In computing, not all people are equal:
The best are at least 5 times more productive
Some tasks are too difficult for everybody
Adding more people adds communications complexity
Some activities need a single mind
Sometimes, the elapsed time for an activity can not be
shortened.
What happens to the project if a key person is sick or quits?
27
Key Personnel: Schedule for Editor
Earliest Start Date Activity
Weeks 15-16 Edit Unit 3
Week 15 Review draft of Unit 7
Week 17 Review draft of Unit 8
Week 19 Check proofs of Unit 3
Week 21 Check proofs of Unit 4
Weeks 18-19 Vacation
Week 22 Out sick
Start-up Time
On a big project, the start-up time is typically three to six
months:
Personnel have to complete previous projects (fatigue) or
recruited.
Hardware and software has to be acquired and installed.
Staff have to learn new domain areas and software (slow
while learning)
Clients may not be ready.
Experience with Critical Path Method
Administrative computing department at Dartmouth used the
Critical Path Method for implementation phase of major projects.
Experience: Elapsed time to complete projects was consistently
25% to 40% longer than predicted by model.
Analysis:
Some tasks not anticipated (incomplete understanding)
Some tasks had to be redone (change of requirements,
technical changes)
Key personnel on many activities (schedule conflicts)
System ZZZ (non-billable hours)
28
Lecture 5
(a) Documentation
(b) Requirements Analysis
Documentation
Reasons for documentation:
visibility (e.g., project plan, interim report)
user support (e.g., user manual)
team communication (e.g., interface specifications)
maintenance and evolution (e.g., requirements)
Characteristics of documentation:
accurate and kept current
appropriate for audience
maintained online (usually)
simple but professional in style and appearance
Documentation is expensive --> Quality not volume
Form of Documentation
External
Printed
Web site
Internal
Program documentation
Program context (e.g., copyright notices)
29
Requirements Definition and Analysis
Requirements
Definition
System and
Software design
Programming
and Unit Testing
Integration and
System Testing
Operation and
Maintenance
Feasibility
Study
Requirements
Analysis
Requirements
Definition
Requirements
Specification
Feasibility
Report System
Models
Definition of
Requirements
Specification of
Requirements
Requirements
Document
Requirements Analysis
1. Understand the requirements in depth:
Domain understanding
Examples: science research, application
Stakeholders
Example: companies, ministries, Danang City
30
Viewpoint Analysis
Example: University Admissions System
Applicants
University administration
Admissions office
Financial aid office
Special offices (e.g., athletics, development)
Computing staff
Operations
Software development and maintenance
Academic departments
Interviews with Clients
Clients may have only a vague concept of requirements.
Prepare before you meet with them
Keep full notes
If you don't understand, delve further
Small group meetings are often most effective
Clients often confuse the current system with the
underlying requirement.
2. Organize the requirements:
Classification into coherent clusters
(e.g., legal requirements)
Recognize and resolve conflicts
(e.g., functionality v. cost v. timeliness)
Example: Dartmouth general ledger system
31
3. Model the requirements:
Informal
Prose
Systematic
Procedural models
Data-centric models
Object models
Formal models
Procedural Models: Flowchart
Operation
Decision
Manual operation
Report
Flowchart: University Admissions
Form
received
New?
Database
record
T
Notify
student
F Update
database
Complete?
Notify
student
T
F
Evaluate
32
Procedural Models: Pseudo-code
Example: Check project project plan
check_plan (report)
if report (date_time) > due_date_time then error (too_late)
if report (client) = none then error (no_client)
if report (team) < min_teamor > max_team
then error (bad_team)
if error() = none
then comments = read_report (report)
return (comments (text), comments (grade))
else return error()
Data-Flow Models
External entities
Processing steps
Data stores or sources
Data flows
Example: University Admissions
Applicant
Application
form
Receive
application
Completed
application
Evaluate
Rejection
Offer
33
Assemble Application Stage
Applicant
Application
form
Receive
Completed
application
Supporting
information
Pending
database
Acknowledgment
Initiate
evaluation
Applicant
database
Evaluation
request
AND
AND
Acknowledgment
Process Completed Application Stage
Rejection
Evaluation
Applicant
database
Evaluation
request Acceptance
Financial
aid
Offer
Special
request
Requirements Analysis v. System Design
Dilemma.
Requirements analysis should make minimal assumptions
about the system design.
But the requirements definition must be consistent with
computing technology and the resources available.
In practice, analysis and design are interwoven. However, do
not to allow the analysis tools to prejudge the system design.
34
Lecture 6
(a) Requirements Analysis (continued)
(b) Requirements Specification
Feasibility
Study
Requirements
Analysis
Requirements
Definition
Requirements
Specification
Feasibility
Report System
Models
Definition of
Requirements
Specification of
Requirements
Requirements
Document
Methods for data modeling and design
Data flow diagrams
Entity-relation diagrams
Data dictionaries
Object models
Many of these methods blur the distinction between
analysis and design.
35
Entity-Relation Model
A Design Methodology for Relational Databases
A database of entities and relations
Tools for displaying and manipulating entity-
relation diagrams
Tools for manipulating the database (e.g., as
input to database design)
Warning: There is much confusion about
definitions and notation
Entity-Relation Diagram
An entity
A relation between
entities
An entity or relation
attribute
An inheritance
relation
Example: CS 501 Project
Student
CS501
Student
Major
Project
5 to 7
1
Member of
Person
Client
1
Tech contact
0:n
0:n
0:n
36
MARC Format for Monographs
(Books)
001 89-16879 r93
245 Campus strategies for libraries and electronic information
260 {Bedford, Mass.} : Digital Press, c1990.
650 Academic libraries--United States--Automation.
650 Libraries and electronic publishing--United States.
700 Arms, Caroline R. (Caroline Ruth)
Entity-Relation Diagram for MARC
Book
Short title
Catalog
record
Describes
Control numb
Subject
heading
Is about
Creator
Editor of
Author of
1:n
1
0:n
0:n
0:n
0:n
0:n
0:n
Data Dictionaries
A data dictionary is a list of names used by the system
Brief definition (e.g., what is "date")
What is it (e.g., number, relation)
Where is it used (e.g., source, used by, etc.)
May be combined with a glossary
As the system is implemented, the data dictionary in the
requirements is input to the system data dictionary, which is a
formal part of the system specification.
37
A Note on Object Models
This course teaches object models as a tool for design.
Some people recommend object models for requirements
analysis, but it is difficult to use them without constraining
the system design.
Non-Functional Requirements
Product requirements
performance, reliability, portability, etc...
Organizational requirements
delivery, training, standards, etc...
External requirements
legal, interoperability, etc...
Examples of Non-Functional
Requirements
Privacy (Mercury digital library)
Functional requirement:
Usage data for management of system
Non-functional requirement:
Usage data must not identify individuals
Minimizing records (NeXT)
Functional requirement:
Retain all required records
Non-functional requirement:
Discard all other records
38
Unspoken Requirements
Example:
Resistance to change at XXX
Requirements Specification
What is the purpose of the Requirements Specification?
Requirements Specification: Purpose
1. It describes the requirements to the stakeholders
Expressed in the terms that the stakeholders understand
Comprehensible from many viewpoints
Reviewed by stakeholders so that they understand
implications
Must be clear about assumptions (things left out)
39
2. It describes the requirements to the implementers
As precise and specific as possible
Expressed in terms that they understand
Comprehensible to new team members
3. It records the requirements for the future
An essential part of system evolution
4. If may be a contractual document
See you in court!
Requirements Specification: Approaches
Natural language
Structured natural language
Design description language
Requirements specification language
Graphical notation
Formal specification
See Sommerville, Chapter 7.
40
Lecture 7
Management II
Business and Legal Aspects of
Legal Environment
Software is developed in a complex legal and
economic framework. Changes in laws follow
changes in technical world.
Jurisdictions:
Vietnamese laws
International treaties
Federal and state statues
Precedents
Supreme Court
Cost of establishing precedent
Legal Topics
International
Intellectual property (copyright, patent, contract)
Tort (e.g., liability of Internet service provider)
Privacy
Free speech and its limitations (government secrets,
obscenity, blasphemy, hate)
Legal Information Institute: http://www.law.cornell.edu/
41
Copyright
A copyright gives the owner the exclusive right to:
reproduce
distribute
perform
display
license
Gradually extended to cover text, music,
photographs, designs, software, ...
Copyright
Copyright at creation
Works for hire
Contracts and licenses
First sale
Fair use
Infringement (contamination)
International differences
Moral rights
Copyright registration
Software Patents
Should be: non-obvious, novel, useful
17 years from award (20 years from application)
Poor quality of examining can lead to broad patents for
routine computing concepts
International differences
Copyright applies to the expression of ideas, patents to the
ideas themselves.
42
Contracts and Licences
Contracts allow intellectual property to be sold or licensed
Promise in exchange for adequate consideration
Written document with signature
Permanent or temporary, whole or part
Exclusive or non-exclusive
Termination, problems and difficulties
Terms and conditions as agreed
Enforceable by courts
Derivative Works
When software is derived from other software:
New code is owned by new developer
Conditions that apply to old code apply to derived work
If you write S, which is derived from A, B, C and D, you can
not distribute or licenses S unless you have right to distribute
each of A, B, C and D.
To create a software product, you must have documented
rights to use every component.
Privacy
Invasions of privacy:
intrusion
appropriation of name or likeness
unreasonable publicity
false light
Be very careful about collecting personal
data without the knowledge of the individual
43
Software Business Questions
You are employed for company X writing software.
When you leave, who owns your work? What use can you
make of the work?
You work free-lance for company X. When you finish,
who owns your work? What use can you make of the work?
Read the contract!
Your Next Job ...
Employment contract may restrict your next job (not
working for competitors, etc.)
Trade-secret information (non-disclosure agreement)
Ask when you are interviewed!
Trade Secrets and Non-Disclosure
Agreements
Trade Secret
"... information, including a formula, pattern, compilation,
program, device, method, technique, or process that derives
independent economic value from not being generally known
and not being readily ascertainable and is subject to reasonable
efforts to maintain secrecy."
Uniform Trade Secrets Act
Non-Disclosure Agreement
Legal agreement not to disclose trade secrets.
44
Some Business Models
Software developed in-house
Package licensed to customer, binary only (Microsoft
model)
Package licensed to customer, source code for customer's
modifications
Bespoke software for customer (may be owned by
supplier or customer)
Software bundled with hardware product (PalmPilot)
Free-Lance Software Development
You and a few friends create a company to develop software.
How much should you charge per hour?
You plan to work 40 hours a week for 50 weeks of the year and
want to earn $50,000.
Hourly rate = $50,000 / (40 x 50) = $25
But ...
Free-Lance Software Development
Salary $50,000
Taxes and benefits $15,000
Rent, equipment, etc. $10,000
Fees, services, etc. $15,000
Travel and misc. $10,000
TOTAL EXPENSE $100,000
Hours worked 2,000
less administration 400
less marketing 350
BILLABLE HOURS 1,250
Hourly rate = $100,000 /1,250 = $80
45
Fixed and Variable Cost: Packaged Software
Example:
The initial development cost of a software product
is $10 million.
The cost of packaging and distribution of each
copy is $5.
Technical support costs average $15 per copy.
The package sells for $200 per copy.
Fixed cost = $10 million
Variable cost = $20
Fixed and Variable Costs: Profit or Loss
$15M
$10M
$5M
Unit
sales 2,500 5,000 7,500
Community Development
Shareware
Open source (e.g., Linux, Apache, Perl, etc.)
-> Shared development
-> Market penetration
Example: TCP/IP for Vax/VMS
Software may be open source, but packaging and
services can be profitable businesses
46
Open Source
Free redistribution
Source code
Derived works
Integrity of the author's source code
No discrimination against persons or groups
Open Source
No discrimination against fields of endeavor
Distribution of license
License must not be specific to a product
License must not contaminate other software
http://www.opensource.org/osd.html
Practical Advice
Be aware of the law, but do not pretend to be a lawyer.
Use a professional for:
Contracts and licenses
Troubles (complaints, injunctions, subpoenas, etc.)
Personnel issues
When in doubt, ask help!
47
Source Code Management
Or
Configuration Management: How I
learned to Stop Worrying and Hate My
Co-workers Less
Source Code Management
Also known as Configuration Management
Source Code Managers are tools that:
Archive your development files
Serve as a single point of entry/exit when adding or
updating development files
Why You Want A Source Control
System
Supports concurrent development
Manage diverging source code bases
Records file/release versions
Easy access to all previous revisions
Can record why a revision was made
Optimal disk space usage
Youll end up doing something equivalent anyway so it
may as well be automated
48
Source Code Management Tools Are
Not
A substitute for project management
A replacement for developer communication
How They Work
Central database of source code, documentation, build
tools
Each file stored only once - all other versions are diffs of
that one copy
To Make a Change
Check out the latest version of a file
Make the changes
Update the database
What should be in the database
Source Code
Documentation
Build Tools
Often need old versions of the tools to build old versions
of the software
Ensures software is rebuilt exactly as the customer
received it
Test Suites
Anything else you might want later
49
Version Control
Companies ship several products from the same source
base (i.e. Win NT and Windows 2000 versions of MS
Office)
When tracking down bugs you want to examine the code
as it was when the product shipped
Code Sharing
Multiple people can work on the same source base without
colliding
(1) Locks individual files so only one person at a time can
modify it *OR*
(2) Allows multiple people to modify a source file and the
system will automatically merge the changes (usually)
Locking
Only one person can work on a file at once
Works fairly well if developers work on different areas of
the project and dont conflict often
Problem 1: People forget to unlock files when they are
done
Problem 2: People work around locking by editing a
private copy and checking in when the file is finally
unlocked - easy to goof and lose changes
50
Merging
Several people can work on a file at once
Before committing changes, each user merges their copy
with the latest copy in the database
This is normally done automatically by the system and
usually works, but you should not blindly accept the result
of the merge
Labeling
Label all the files in the source base that make up a product
at each milestone
Just before and just after a major change (e.g.. changing
several interfaces)
When a new version ships
Version Trees
Each file in the database has a version tree
Can branch off the version tree to allow separate
development paths
Typically a main path (trunk) for the next major version
and branches off of shipped versions for maintenance
51
Branching
When a new version ships, typically create a branch in the
version tree for maintenance
Double update: fix a defect in the latest version and then
merge the changes (often by hand) into the maintenance
version
Also create personal versions so you can make a change
against a stable source base and then merge in the latest
version later
Examples
RCS
Solaris: man rcsintro
CVS
Solaris: man cvs
www.cyclic.com/cvs/info.html
Visual SourceSafe
msdn.microsoft.com/SSAFE
ClearCase
www.rational.com
RCS
File management only
Transaction model
check out and lock
edit
check in and unlock
Little support for binaries
52
CVS
Built on top of RCS
Therefore little support for binaries
Database can be remote
No locking: merge before commit
Fast
Integrates with emacs
SourceSafe
Microsofts entry into the field
Project-based
Checkout-edit-checkin model
Built-in web site creation tools
Integrates with MSDEV
Clearcase
Clearcase is configuration management on steroids
You create a view of the database with a config spec,
which describes how to select files from the database.
When you set a view, Clearcase creates a virtual filesystem
containing only those versions of the files selected by the
config spec
53
Clearcase Features
Distributed System
Several groups at different locations can work on the same
database
Can install triggers
Example: e-mail the author of a file when some one makes
a change to it
Uses merging model like CVS, but can also lock files
More Clearcase Features
Integrates with MSDEV
Build Management
Knows to rebuild out-of-date files even if your makefile
doesnt
Slow and a bit buggy
Helpful Rules for Version Control Bliss
Archived Files Should Always Compile
Code Review Files Before Check-in
Compile and run latest archived files *as a set* before
Check-in
No Cheating (even simple bug fixes need to undergo this
process)
54
Lecture 10
Formal Specification
Why?
Precise standard to define and validate software
Why not?
May be time consuming
Methods not suitable for all applications
Ben Potter, Jane Sinclair, David Till,
An Introduction to Formal Specification and Z
(Prentice Hall) 1991
Jonathan Jacky
The Way of Z
(Cambridge University Press) 1997
55
Mathematical Specification
Example of specification
B
1
, B
2
, ... B
k
is a sequence of m x m matrices
q
1
, q
2
, ... q
k
is a sequence of m x m elementary matrices
B
1
-1
= q
1
B
2
-1
= q
2
q
1
B
k
-1
= q
k ...
q
2
q
1
The numerical accuracy must be such that, for all k,
B
k
B
k
-1
- I < A
Specification of Programming Languages
<unsigned number> ::= <unsigned integer> | <unsigned real>
<unsigned integer> ::= <digit> {<digit>}
<unsigned real> ::= <unsigned integer> . <digit> {<digit>} |
<unsigned integer> . <digit> {<digit>} E <scale factor> |
<unsigned integer> E <scale factor>
<scale factor> ::= <unsigned integer> | <sign> <unsigned integer>
<sign> ::= + | -
Pascal number syntax
Formal Specification Using Diagrams
digit
unsigned integer
digit
.
E
+
-
unsigned integer unsigned integer
unsigned number
56
Two Rules
Formal specification does not guarantee correctness
Formal specification does not prescribe the implementation
Informal: The function intrt(a) returns the largest integer
whose square is less than or equal to a.
Formal (Z):
intrt: N N
a : N
intrt(a) * intrt(a) < a < (intrt(a) + 1) * (intrt(a) + 1)
Example: Z Specification Language
Example: Algorithm
1 + 3 + 5 + ... (2n - 1) = n
2
57
Example: Program
int intrt (int a)
/* Calculate integer square root */
{
int i, term, sum;
term = 1; sum = 1;
for (i = 0; sum <= a; i++)
{
term = term + 2;
sum = sum + term;
}
return i;
}
Finite State Machine
A broadly used method of formal specification:
Event driven systems (e.g., games)
User interfaces
Protocol specification
etc., etc., ...
Finite State Machine
Example:
Therapy control console
[informal description]
58
State Transition Diagram
Patients Fields Setup Ready
Beam
on
Enter Enter Start
Stop
Select field
Select patient
(interlock)
(ok)
State Transition Table
Select
Patient
Select
Field
Enter ok Start Stop
interlock
Patients
Fields
Setup
Ready
Beam
on
Fields
Fields
Fields
Patients
Patients
Patients
Setup
Setup
Setup
Ready
Beam
on
Ready
Z Specification
STATE ::= patients | fields | setup | ready | beam_on
EVENT ::= select_patient | select_field | enter | start | stop
| ok | interlock
FSM == (STATE X EVENT) STATE
no_change, transitions, control : FSM
Continued on next slide
59
Z Specification (continued)
control = no_change transitions
no_change = { s : STATE; e : EVENT (s, e) s }
transitions = { (patients, enter) fields,
(fields, select_patient) patients, (fields, enter) setup,
(setup, select_patient) patients, (setup, select_field) fields,
(setup, ok) ready,
(ready, select_patient) patients, (ready, select_field) fields,
(ready, start) beam_on, (ready, interlock) setup,
(beam_on, stop) ready, (beam_on, interlock) setup }
Schemas
Schema:
The basic unit of formal specification.
Describes admissible states and operations of a
system.
LibSys: An Example of Z
Library system:
Stock of books
Registered users.
Each copy of a book has a unique identifier.
Some books on loan; other books on shelves available
for loan.
Maximum number of books that any user may have on
loan.
60
LibSys: Operations
Issue a copy of a book to a reader.
Reader return a book.
Add a copy to the stock.
Remove a copy from the stock.
Inquire which books are on loan to a reader.
Inquire which readers has a particular copy of a book.
Register a new reader.
Cancel a reader's registration.
LibSys
Level of Detail:
Assume given sets:
Copy, Book, Reader
Global constant:
maxloans
Schemas Describing Operations
Naming conventions for objects:
Before: plain variables, e.g., r
After: with appended dash, e.g., r'
Input: with appended ?, e.g., r?
Output: with appended !, e.g., r!
61
Operation: Issue a Book
Inputs: copy c?, reader r?
Copy must be shelved initially: c? e shelved
Reader must be registered: r? e readers
Reader must have less than maximum number of books on loan:
#(issued {r?}) < maxloans
Copy must be recorded as issued to the reader:
issued' = issued {c? r?}
The stock and the set of registered readers are unchanged:
stock' = stock; readers' = readers
Domain and Range
dom m X Y
x
ran m
y
m : X Y
dom m = { x e X : - y eY . x y}
ran m = { y eY : - x e X . x y}
Operation: Issue a Book
stock, stock' : Copy Book
issued, issued' : Copy Reader
shelved, shelved': F Copy
readers, readers' : F Reader
c?: Copy; r? :Reader
[See next slide]
Issue
62
Operation: Issue a Book (continued)
[See previous slide]
Issue
shelved dom issued = dom stock
shelved' dom issued' = dom stock'
shelved dom issued = ; shelved' dom issued' =
ran issued _ readers; ran issued' _ readers'
r : readers - #(issued {r}) maxloans
r : readers' - #(issued' {r}) maxloans
c? e shelved; r? e readers; #(issued {r?}) < maxloans
<
<
LibSys: Schema for Abstract States
Library
stock : Copy Book
issued : Copy Reader
shelved : F Copy
readers: F Reader
shelved dom issued = dom stock
shelved dom issued =
ran issued _ readers
r : readers #(issued {r}) maxloans <
Schema Inclusion
LibDB
stock : Copy Book
readers: F Reader
LibLoans
issued : Copy Reader
shelved : F Copy
r : Reader #(issued {r}) maxloans
shelved dom issued =
<
63
Schema Inclusion (continued)
Library
LibDB
LibLoans
dom stock = shelved dom issued
ran issued _ readers
Schema Decoration
Issue
Library
Library'
c? : Copy; r? : Reader
c? e shelved; r? e readers
Schema Decoration
Issue
ALibrary
c? : Copy; r? : Reader
c? e shelved; r? e readers
64
The Schema Calculus
Schema inclusion
Schema decoration
Schema disjunction:
AddCopy AddKnownTitle v AddNewTitle
Schema conjunction:
AddCopy EnterNewCopy . AddCopyAdmin
Schema negation
Schema composition
=
^
=
^
Lecture 11
Object-Oriented Design I
The Waterfall Model
Requirements
Definition
Implementation
and Unit Testing
Integration and
System Testing
Operation and
Maintenance
System and
Software design
65
Useful Texts
Grady Booch, James Rumbaugh, Ivar Jacobson, The Unified
Modeling Language. Addison-Wesley 1999.
Grady Booch, Object-Oriented Analysis and Design with
Applications, second edition. Benjamin/Cummings 1994.
Rob Pooley, Perdita Stevens, Using UML Software
Engineering with Objects and Components. Addison-Wesley
1999.
The Importance of Modeling
A model is a simplification of reality.
We build models so that we can better understand the
system we are developing.
We build models of complex system because we cannot
comprehend such a system in its entirety.
Models can be informal or formal. The more complex the
project the more valuable a formal model becomes.
BRJ
Principles of Modeling
The choice of what models to create has a profound
influence on how a problem is attacked and how a
solution is shaped.
Every model can be expressed at different levels of
precision.
The best models are connected to reality.
No single model is sufficient. Every nontrivial
system is best approached through a small set of
nearly independent models.
BRJ
66
The Unified Modeling Language
UML is a standard language for modeling software systems.
Serves as a bridge between the requirements specification
and the implementation.
Provides a means to specify and document the design of a
software system.
Is process and programming language independent.
Is particularly suited to object-oriented program
development.
Notation: Classes
Window
origin
size
open()
close()
move()
display()
name
attributes
operations
A class is a description of a set of objects that share the same
attributes, operations, relationships and semantics.
Notation: Interface
An interface is a collection of operations that specify a
service of a class or component, i.e., the externally
visible behavior of that element.
ISpelling
67
Notation: Collaboration & Use Case
Place order
A use case is a description of a set of sequence of actions
that a system performs that yields an observable result.
Chain of
responsibility
A collaboration defines an interaction, i.e., a society of
roles and other elements that work together to provide some
cooperative behavior.
Notation: Active Class
EventManager
eventlist
suspend()
flush()
An active class is a class whose objects own one or
more processes or threads and therefore can initiate
control activity.
Notation: Component & Node
orderform.java
A component is a physical and replaceable
part of a system that conforms to and provides
the realization of a set of interfaces.
Server
A node is a physical element that exists at run
time and represents a computational resource.
68
Notation: Behavioral Things:
Messages & States
display
An interaction is a behavior that comprises a set of messages
exchanged among a set of objects within a particular context to
accomplish a specific purpose.
Waiting
A state machine is a behavior that specifies the sequence of
states an object or an interaction goes through during its
lifetime in response to events.
Notation: Grouping and Annotation
A package is a general-purpose mechanism for organizing
elements into groups.
Business rules
return copy
of self
A note is a symbol for rendering constraints and
comments attached to an element or a collection of
elements.
Notation: Relationships
A dependency is a semantic relationship between two things in
which a change to one may effect the semantics of the other.
0..1 *
employer employee
An association is a structural relationship that describes
a set of links, a link being a connection among objects.
69
Notation: Relationships (continued)
A generalization is a specialization/generalization
relationship is which objects of the specialized
element (child) are substitutable for objects of the
generalized element (parent).
child parent
A realization is a semantic relationship between
classifiers, wherein one classifier specifies a
contract that another classifier guarantees to carry
out.
Diagrams in UML
A diagramis the graphical representation of a set of
elements, usually rendered as a connected graph of vertices
(things) and arcs (relationships).
Class diagramshows a set of classes, interfaces, and
collaborations with their relationships.
Object diagramshows a set of objects and their
relationships.
Use case diagram shows a set of use cases and actors (a
special kind of class) and their relationships.
Diagrams in UML (continued)
Interaction diagram shows an interaction, consisting
of a set of objects and the relationships, including the
messages that may be dispatched among them.
=> A sequence diagramemphasizes the time ordering.
=> A collaboration diagramemphasizes the structural
organization of the objects that send and receive messages.
70
Diagrams in UML (continued)
Statechart diagramshows a state machine consisting
of states, transitions, events, and activities.
Activity diagram is a statechart diagram that shows the
flow from activity to activity within a system.
Component diagramshows the organization and
dependencies among a set of components.
Deployment diagramshows the configuration of
processing nodes and the components that live on them.
The HelloWorld Example
HelloWorld
paint()
class
name
operations
Abstraction for HelloWorld
HelloWorld
paint() g.drawString
("HelloWorld", 0, 10)"
class
name
operations
annotation
71
The "Hello, World" Example
import java.awt.Graphics;
class HelloWorld extends java.applet.Applet {
public void paint (Graphics g) {
g.drawString ("Hello, World!", 10, 10);
}
}
Example from: BJR
Class Diagram
Applet
HelloWorld
paint()
Graphics
generalization
dependency
Note that the Applet
and Graphics classes
are shown elided.
Class Inheritance Diagram
Object
Component
Container
Panel
Applet
HelloWorld
ImageObserver
interface
72
Packaging Classes
applet
awt
lang
HelloWorld
java
Graphics
package
Notation for Classes and Objects
Classes
Objects
AnyClass
attribute1
attribute2
operation1()
operation2()
AnyClass
or
anObject:AnyClass
:AnyClass
anObject
The names of objects are
underlined.
or
or
Lecture 12
Object-Oriented Design II
73
Requirements: the Long Term
Believe that your software will be in use 5 years from now.
What happens at end of semester?
Packaging and hand-over
Client's technical preferences (C++, Java)
Some system decisions based on short-term considerations
Which formats, protocols, etc. do you think will last?
(IIOP, RMI, SNMP, ...)
Requirements, Design and Implementation
Remember the definitions.
Example: Consistency between two players of a board game
The requirement is .....
The design is .....
What is a requirements specification?
Modeling Classes
Given a real-life system, how do you decide what classes to
use?
What terms do the users and implementers use to describe the
system? They are candidates for classes.
Is each candidate class crisply defined?
For each class, what is its set of responsibilities? Are the
responsibilities evenly balanced among the classes?
What attributes and operations does each class need to carry
out its responsibilities?
74
Noun Identification: A Library Example
The library contains books and journals. It may have several
copies of a given book. Some of the books are reserved for
short-term loans only. All others may be borrowed by any
library member for three weeks. Members of the library can
normally borrow up to six items at a time, but members of
staff may borrow up to 12 items at one time. Only members
of staff may borrow journals.
The system must keep track of when books and journals are
borrowed and returned and enforce the rules.
Candidate Classes
Library the name of the system
Book
Journal
Copy
ShortTermLoan event
LibraryMember
Week measure
MemberOfLibrary repeat
Item book or journal
Time abstract term
MemberOfStaff
System general term
Rule general term
75
Relations between Classes
Book is an Item
Journal is an Item
Copy is a copy of a Book
LibraryMember
Item
MemberOfStaff is a LibraryMember
Is Item needed?
Operations
LibraryMember borrows Copy
LibraryMember returns Copy
MemberOfStaff borrows Journal
MemberOfStaff returns Journal
Item not needed yet.
Class Diagram
MemberOfStaff
Book Copy Journal is a copy of
1..* 1
LibraryMember
1
0..*
0..12
1
on loan
on loan
76
Rough Sketch: Wholesale System
A wholesale merchant supplies retail stores from
stocks of goods in a warehouse.
What classes would you use to model this business?
RetailStore
Warehouse
Order
Invoice
Product
Shipment
Merchant
Warehouse
Order
Invoice
Product
Merchant
RetailStore
name
address
contactInfo
financialInfo
Shipment
Responsibilities
-track status of
shipped products
Reversals
damaged()
return()
wrongItem()
responsibility
(text field)
77
Expanding a Class:
Modeling Financial Information
RetailStore
Transaction
1 *
association
Invoice
Payment Which class is
responsible for the
financial records for
a store?
Modeling Invoice
Shipment
Invoice
invoiceNumber
+goodsShipped()
-sendInvoice()
goodsShipped
PartsList
adornments
+ public
- private
RetailStore ???
invoiceRecord
Lessons Learned
Design is empirical. There is no single correct design.
During the design process:
Eliding: Elements are hidden to simplify the diagram
Incomplete: Elements may be missing.
Inconsistency: The model may not be consistent
The diagram is not the whole design. Diagrams must
be backed up with specifications.
78
Levels of Abstraction
The complexity of a model depends on its level of abstraction:
High-levels of abstraction show the overall system.
Low-levels of abstraction are needed for implementation.
Two approaches:
Model entire system at same level of abstraction, but present
diagrams with different levels of detail.
Model parts of system at different levels of abstraction.
Component Diagram
HelloWorld.class
hello.java
hello.hml
hello.jpg
executable
component
Actor and Use Case Diagram
An actor is a user of a system in a
particular role.
An actor can be human or an external
system.
A use case is a a task that an actor
needs to perform with the help of the
system.
Borrow book
BookBorrower
79
Use Cases and Actors
A scenario is an instance of a use case
Actor is role, not an individual
(e.g., librarian can have many roles)
Actor must be a "beneficiary" of the use case
(e.g., not librarian who processes book when borrowed)
In UML, the system boundary is the set of use cases.
Use Cases for Borrowing Books
Borrow copy
of book
BookBorrower
Return copy
of book
Reserve
book
Extend
loan
Relationships Between Use Cases:
<<uses>>
BookBorrower
Check for
reservation
Extend
loan
<<uses>>
<<uses>>
Borrow copy
of book
80
Relationships Between Use Cases:
<<extends>>
Borrow copy
of book
BookBorrower
Refuse
loan
<<extends>>
Use Cases in the Development Cycle
Use cases are a tool in requirements analysis
Intuitive -- easy to discuss with clients
Use cases are often hard to translate into class models
Scenarios are useful to validate design
Lecture 13
Object-Oriented Design III
81
Comments on Presentations
Presentation
Standard of graphics has been high
Some text too small (diagrams, screen dumps)
Content
Level of detail
Requirements v. design
The client defines the requirements
Well done, but time is short. What is your critical path?
Modeling Dynamic Aspects of Systems
Interaction diagrams: set of objects and their relationships
including messages that may be dispatched among them
Sequence diagrams: time ordering of messages
Collaboration diagrams: structural organization of
objects that send and receive messages
Activity diagram: flow chart showing flow of control from
activity to activity
Statechart diagram: models a state machine
Bouncing Ball Diagrams
Example: http://www.cs.cornell.edu/
Client Servers
domain
name
TCP
connection
HTTP get
82
Actions on Objects
call
return
send
create
destroy
returnCopy(c)
okToBorrow() local
status
notifyReturn(b) asynchronous
signal
<<create>>
<<destroy>>
stereotypes
Links
Copy
LibraryMember
+borrowCopy()
+returnCopy()
1
0..* on loan
libMem:LibraryMember
c:Copy
borrowCopy(c)
association
message
link
class
object
Sequence Diagram: Change in Cornell
Program
Cornellian
:MEngStudent
1 : getName()
sequence numbers added to messages
:PhDStudent
1.1 : name
2: new PhDStudent(name)
3: <<destroy>>
83
Sequence Diagram: Borrow copy of a
Book
BookBorrower
libMem:
LibraryMember
theCopy:Copy
theBook:Book
borrow(theCopy)
okToBorrow
borrow
borrow
Class Inheritance Diagram
Object
Component
Container
Panel
Applet
HelloWorld
ImageObserver
interface
Sequence Diagram:Painting Mechanism
:Thread :Toolkit :ComponentPeer target:HelloWorld
run
run callbackLoop
handleExpose
paint
84
Activity Diagram: Process Modeling
Release work
order
Assign tasks
Reschedule
[materials not ready]
[materials ready]
guard expression
branch
Activity Diagram: Parallel Activities
Decompress
Stream audio Stream video
fork
join
start state
stop state
State Diagram
State diagram for class Book
not
borrowable
returned()
returned()
borrowable
borrowed()[not last copy]
borrowed()[last copy]
guard expression
85
Implementation Modeling
Subsystem
A grouping of elements that specifies what a part of a system
should do.
Component (UML definition)
"A distributable piece of implementation of a system,
including software code (source, binary, or executable) but
also including business documents, etc., in a human system."
A component can be thought of as an implementation of a
subsystem.
Component Diagram
HelloWorld.class
hello.java
hello.hml
hello.jpg
executable
component
Components and Classes
agent.dll
AgentAction
Policy
PatternSearch
86
agent.dll
Realizes
AgentAction
PatternSearch
Policy
extended component
Classes represent logical abstractions. Components represent
physical things.
Components may live on nodes.
Classes have attributes and operations directly. Components
have operations that are reachable only through interfaces.
Interfaces
simulation.exe
render.java
IRender
dependency
interface
realization
87
Application Programming Interface
(API)
API is an interface that is realized by one or more components.
simulation.exe
IModels ILighting
IRender
Components and Replaceability
Components allow system to be assembled from binary
replaceable elements.
A component is physical -- bits not concepts
A component can be replaced by any other component(s)
that conforms to the interfaces.
A component is part of a system.
A component provides the realization of a set of
interfaces.
Lecture 14
System Architecture I
Data Intensive Systems
88
System Architecture
The overall design of a system:
Computers and networks (e.g., monolithic, distributed)
Interfaces and protocols (e.g., http, CORBA)
Databases (e.g., relational, distributed)
Security (e.g., smart card authentication, SSL)
Operations (e.g., backup, archiving, audit trails)
Software environments (e.g., languages, source control tools)
Data Intensive Systems
Examples
Electricity utility customer billing
Telephone company call recording and billing
Car rental reservations (e.g., Hertz)
Stock market brokerage (e.g., Charles Schwab)
Web sales (e.g., Amazon.com)
Example 1: Electricity Utility Billing
First attempt:
Data input Master file Transaction Bill
Each transaction handled as it arrives.
89
Criticisms of First Attempt
Where is this first attempt weak?
The requirements have not been specified!!!
Transaction Types
Create account / close account
Meter reading
Payment received
Other credits / debits
Check cleared / check bounced
Account query
Correction of error
etc., etc., etc.,
Typical Requirements
All payments to be credited on day received
Customers must be able to query account by telephone
Cutting off service for non-payment requires
management authorization
Data input staff should process n transactions per day
per person
Error rate must be below 0.01%
System available 99.9% of business hours
90
Batch Processing: Validation
Data input
Master file
Edit &
validation
read only
errors
Validated
transactions
Incoming
transactions
Batch Processing: Master File Update
Master file
update
Bills
Validated
transactions
in batches
Sort by
account
errors
Reports
Instructions
Benefits of Batch Updating
All transactions for an account are processed together
Backup and recovery have fixed checkpoints
Better management control of operations
Efficient use of staff and hardware
91
Online Inquiry
Data input Master file Transactions Bills
read only
Customer
service
Example 2: A Small-town Stockbroker
Transactions
Received by mail or over telephone
For immediate or later action
Complex customer inquiries
Highly competitive market
A Database Architecture
Database(s):
Customer and account database
Financial products (e.g., account types, pension plans,
savings schemes)
External databases (e.g., stock markets, mutual funds,
insurance companies)
92
Database Architecture
Customer &
account database
Products &
services database
External
services
Real-time Transaction
Customer &
account database
Products &
services database
External
services
Real-time
transactions
Real-time Transactions & Batch
Processing
Customer &
account database
Products &
services database
External
services
Real-time
transactions
Batch
processing
Data
input
93
Architectural considerations
Real-time service during scheduled hours + batch
processing overnight
Combine information from several databases
Database consistency after any type of failure
two-phase commit
reload from checkpoint + log
detailed audit trail
How will transaction errors be avoided?
How will transaction errors be corrected?
Example: Merger of Two Banks
Each bank has a database with its customer accounts. The
databases are used by staff at many branches and for back-office
processing.
The requirement is to integrate the two banks so that they appear
to the customers to be a single organization and to provide
integrated service from all branches.
Merger of Two Banks: Options
???
???
A
B
94
Merger of Two Banks: Architectural
Options
I. Convert everything to System A.
convert databases
retrain staff
enhance System A (software and hardware)
discard System B
II. Build an interface between the databases in
System A and System B.
III. Extend client software so that it can interact with
either System A or System B database.
Distributed Computing: General
Problem
An application that is running on one computer wishes to
use data or services provided by another:
Network connection
private, public, or virtual private network
location of firewalls
Protocols
point-to-point, multicast, broadcast
message passing, RPC, distributed objects
stateful or stateless
Quality of service
Network Choices
Public Internet:
Ubiquitous -- worldwide
Low cost
Private network:
Security
Predictable performance
Choice of protocols (e.g., IBM's SNA)
95
Quality of Network Services
Performance
Maximum throughput
Variations in throughput
Real-time media (e.g., audio)
Business
Suppliers
Trouble shooting and maintenance
Upgrades
Firewall
Public
network
Private
network
Firewall
A firewall is a computer at the junction of two network
segments that:
Inspects every packet that attempts to cross the boundary
Rejects any packet that does not satisfy certain criteria, e.g.,
an incoming request to open a TCP connection
an unknown packet type
Lecture 15
System Architecture II
Distributed and Real Time Systems
96
Comments on Requirements Report
Audience
Client and design team
Will be updated over time
Content
Level of detail -- will be used to validate the implementation
Requirements, not design
Precise, but not legalistic
Sequence Diagram: Notation
BookBorrower
libMem:
LibraryMember
theCopy:Copy
theBook:Book
borrow(theCopy)
okToBorrow
borrow
borrow
dotted line
shows object
lifetime
rectangle
shows focus
of control
Sequence Diagram: Branching
BookBorrower
libMem:
LibraryMember
theCopy:Copy
theBook:Book
1:borrow(theCopy)
2:okToBorrow
[ok]3:borrow
4:borrow
branch
[not ok]3:noborrow
97
Example: Distributed Database
two copies of the
same data
Distributed Data and Replication
Distributed Data
Data is held on several computer systems. A transaction may need
to assemble data from several sources.
Replication
Several copies of the data are held in different locations.
Mirror: Complete data set is replicated
Cache: Dynamic set of data is replicated (e.g., most recently used)
With replicated data, the biggest problem is consistency.
Example: Broadcast Search
User interface
server
User
Databases
98
Example: UseNet
Stateless Protocol v. Stateful
Stateless protocol
Example: http
Open connection
Send message
Return reply
Close connection
State in http must be sent with every message
(e.g., as parameter string or in a cookie)
Stateless Protocol v. Stateful
Stateful (session) protocol
Example: Z39.50
Open connection
Begin session
Interactive session
End session
Close connection
Server remembers the results of previous transactions
(e.g., authentication, partial results) until session is
closed.
99
Firewall
Public
network
Private
network
Firewall
A firewall is a computer at the junction of two network
segments that:
Inspects every packet that attempts to cross the boundary
Rejects any packet that does not satisfy certain criteria, e.g.,
an incoming request to open a TCP connection
an unknown packet type
The Domain Name System
.edu
server
cornell.edu
server
cs.cornell.edu
server
First attempt to resolve
www.cs.cornell.edu
1
2
3
Discussion of the First Attempt
Problems?
100
The Domain Name System
.edu
server
cornell.edu
server
cs.cornell.edu
server
Better method
3
1
almaden.ibm.com
cornell.edu
ece.cmu.edu
ibm.com
acm.org
.edu
2
Local
cache
local DNS
server
Real Time System
A real time system is a software system whose correct
functioning depends upon the results produced and
the time at which they are produced.
A soft real time system is degraded if the results
are not produced within required time constraints
A hard real time system fails if the results are not
produced within required time constraints
Example: Web Server
http message
daemon
spawned processes
TCP port 80
The daemon listens at port 80.
When a message arrives it:
spawns a processes to handle the message
returns to listening at port 80
101
Embedded Systems
Software and hardware are combined to provide an
integrated unit, usually dedicated to a specific task:
Digital telephone
Automobile engine control
GPS
Scientific instruments
The software may be embedded in the device in a manner
that can not be altered after manufacture.
Example: Autonomous Land Vehicle
Sensors
GPS
Sonar
Laser
Signal
processing
Model Control
signals
Steer
Throttle
Controls
Other Applications
Response critical
Network router
Telephone switch
Seat bag controller
Shared systems
Multi-user data processing
Time sharing
102
Techniques
Special purpose hardware
Multi-threading and multi-tasking
Parallel processing
=> digital signal processing
Interrupts
=> levels and priorities
Multi-Threading
Several similar threads operating concurrently:
Re-entrant code -- separation of pure code from
data for each thread
Testing -- single thread and multi thread
May be real time (e.g., telephone switch) or non-
time critical
Real Time Executive
Schedules and dispatches tasks in a real time system
Real time clock
Interrupt handler
Scheduler
Resource manager
Dispatcher
Must be extremely reliable
103
Timing
Timing mechanisms
Synchronous (clocked) -- periodic stimuli
Asynchronous -- wait for next signal
Example: Communications protocols may be
synchronous or asynchronous
Hardware v. Software
Design of embedded systems requires close understanding
of hardware characteristics
Special purpose hardware requires special tools and
expertise.
Some functions may be implemented in either
hardware of software (e.g., floating point unit)
Design requires separation of functions
Distinction between hardware and software may be
blurred.
Example: Dartmouth Time Shared
System
Communications
processor
Communications
processor
Central
processor
Central
processor
Central
processor
I/O
Mulitplexor
master
processor
104
Software Considerations
Resource considerations may dictate software design and
implementation:
Low level language (e.g., C) where programmer has close
link to machine
Inter-process communication may be too slow (e.g., C
fork).
May implement special buffering, etc., to control timings
Example: CD Controller
Input
block
Output
block
1
2
3 4
5
6
7
Circular buffer
Continuous Operation
Many systems must operate continuously
Software update while operating
Hardware monitoring and repair
Alternative power supplies, networks, etc.
Remote operation
These functions must be designed into the fundamental
architecture.
105
Routers and Other Network Computing
Interoperation with third party devices
Support for several versions of protocols
Restart after total failure
Defensive programming -- must survive
=> erroneous or malicious messages
=> extreme loads
Time outs, dropped packets, etc.
Evolution of network systems
Lecture 16
System Architecture III
Distributed Objects
Real-Time: Software Considerations
Resource considerations may dictate software design and
implementation:
Low level language (e.g., C) where programmer has close
link to machine
Inter-process communication may be too slow (e.g., C
fork).
May implement special buffering, etc., to control timings
106
Buffering Example: CD Controller
Input
block
Output
block
1
2
3 4
5
6
7
Circular buffer
Continuous Operation
Many systems must operate continuously
Software update while operating
Hardware monitoring and repair
Alternative power supplies, networks, etc.
Remote operation
These functions must be designed into the fundamental
architecture.
Example: Routers and Other Network
Computing
Interoperation with third party devices
Support for several versions of protocols
Restart after total failure
Defensive programming -- must survive
=> erroneous or malicious messages
=> extreme loads
Time outs, dropped packets, etc.
Evolution of network systems
107
Example: Transaction Monitor
messages
A transaction monitor: monitors transactions, routes
them across services, balances the load, restarts
transactions after failure.
Transaction
monitor
processes
Software Reuse: Application Packages
Package supports a standard application (e.g., payroll, user
interface to Internet information, mathematical algorithms)
Functionality can be enhanced by:
=> configuration parameters (e.g., table driven)
=> extensibility at defined interfaces
=> custom written source code extensions
Reuse: Object Oriented Languages
Example:
Java is a relatively straightforward language with a very rich set
of class hierarchies.
Java programs derive much of their functionality from
standard classes
Learning and understanding the classes is difficult.
=> Java experts can write complex systems quickly
=> Inexperienced Java programmers write inelegant and
buggy programs
108
Reuse: Objects - Basic Definitions
An object is a piece of code that owns attributes and
provides services through methods.
The methods operate on instance data owned by the
object.
A class is a collection of like objects.
Reuse: Objects - Characteristics
Encapsulation. An object has a public interface that
defines how other objects or applications can interact
with it.
methods
public instance data
Inheritance. Subclasses can be derived from parent
classes. They inherit or override the parents' methods
and instance data.
Polymorphism. The effect of a method can vary
depending on the class that implements it (e.g.,
display_object)
Reuse: Objects - Object Binding
Binding is the linking of the software interface between two
objects.
Static binding: The interface is determined at compile or
build time.
Straightforward
Allows type checking
Dynamic binding or late binding: The link is established
at run time.
Flexible and extensible
Complex
109
Reuse: Objects - Distributed Objects
Objects on separate computers interact through method calls
and instance data.
Major systems:
CORBA (Common Object Request Broker Architecture)
Microsoft family: OLE, COM, DCOM, Active X ...
Desirable Properties of Distributed
Objects
Different languages and operating environments
Reusable code: components
Architecture can be extensible
Future changes can be localized
Standard tools used for client/server interactions
Lecture 17
Design for Usability I
110
Q2: Finite State Machine
The cruise control system on an automobile is controlled by a
master switch and three buttons. Initially, it is turned on by the
master switch. The master switch can be turned off at any time.
When first turned on, the system enters stand-by mode.
When the system is in stand-by mode, the driver of the automobile
can press Button A to engage the cruise control at the current speed
of the automobile. When the cruise control is engaged, if the
driver presses the brake or presses Button B the system will be
disengaged and return to stand-by mode. After returning to stand-
by mode, the driver can press Button C to engage the cruise
control at the speed that it was set at previously. (After the system
is first turned on, Button C has no effect.)
When the cruise control is engaged, the driver can press Button A
to increase speed by one mile per hour or Button C to decrease
speed by one mile per hour.
State Transition Diagram
Off Standby Engaged Standby1
MS-On
B-Brake
MS-Off
A
A
C
C A
State Transition Table
MS
on
MS
off
A
B
Brake
C
Off
Standby
Engaged
Standby1
Standby
Off
Off
Off Engaged
Standby1
Engaged Engaged
Engaged Engaged
111
Question 4
When software is written, who owns the copyright?
How can somebody else be permitted to use the software?
How can copyright be transferred to somebody else?
Question 4
When software is written, who owns the copyright?
The person who writes the software
Except work for hire -- the employer
How can somebody else be permitted to use the software?
By permission from the copyright owner
(usually a license)
How can copyright be transferred to somebody else?
Copyright is property that can be sold or given away
(usually a contract)
Question 4
When you leave, who owns your work?
What use can you make of the work?
112
Question 4
When you leave, who owns your work?
The company (work for hire)
None without permission of the copyright owner
Question 4
You work free-lance for company X.
When you finish, who owns your work?
Question 4
You work free-lance for company X.
When you finish, who owns your work?
It depends on the circumstances
Have a written contract
If you hold the copyright -- unrestricted
Otherwise -- none without agreement
113
Distributed Objects and the System Life-
Cycle
All large systems change with time.
Dynamic binding of objects combined with polymorphism
permits the addition of extra object types, incremental
changes, etc. to be localized.
Development environments change with time.
Language bindings and IIOP permit changes.
Production environments changes with time.
Code can be reused in different environments.
Design for Usability
Usability of a computer system is a combination of factors:
User interface design
Functionality
Performance
Help systems and documentation
Freedom from errors
Anything else?
Iterative Design
Requirements
Design
Implementation
(prototype)
Evaluation
114
Methods for Evaluation of Usability
Observing users (user protocols)
Focus groups
Measurements
effectiveness in carrying out tasks
speed
Expert review
Client's opinions
Competitive analysis
Levels of Usability
interface design
functional design
data and metadata
computer systems and networks
conceptual
model
The Conceptual Model
The conceptual model is the user's internal model of what
the system provides:
The desk top metaphor -- files and folders
The web model -- click on hyperlinks
The library model -- search and retrieve
The form filling model -- fill form, submit
Example: The Mercury page turner
115
Interface Design
The interface design is the appearance on the screen and the
actual manipulation by the user
Fonts, colors, logos, key board controls, menus, buttons
Mouse control or keyboard control?
Conventions (e.g., "back", "help")
Example: Screen space utilization in the Mercury page turner
Principles of Interface Design
Interface design is partly an art; there are general principles:
Consistency -- in appearance, controls, and function.
Feedback -- what is the computer system is doing?
why does the user see certain results?
Users should be able to interrupt or reverse actions
Error handling should be simple and easy to comprehend
Skilled users offered shortcuts;
beginners have simple, well-defined options
The user should feel in control
Disabilities
What if the user:
is visually impaired or color blind?
does not speak English?
is a poor typist?
There is a tradition of blind programmers
Navigation of web sites need not be only visual
You may have a legal requirement to support people
with disabilities
116
Functional Design
The functional design, determines the functions that are offered
to the user
Selection of parts of a digital object
Searching a list or sorting the results
Help information
Manipulation of objects on a screen
Pan or zoom
Same Functions, Different Interface
Example: The desk top metaphor
Mouse -- 1 button (Macintosh), 2 button (Windows)
or 3 button (Unix)
Close button -- left of window (Macintosh)
right of window (Windows)
Data and Metadata
Data and metadata stored by the computer system
enable the functions and the interface
The desktop metaphor has the concept of associating a
file with an application. This requires a file type to be
stored with each file:
-- extension to filename (Windows and Unix)
-- resource fork (Macintosh)
Data validation often requires that a user interface has
access to a database (e.g., names and addresses)
117
Computer Systems and Networks
The performance, reliability and predictability of computer
systems and networks is crucial to usability
Response time
instantaneous for mouse tracking and echo of key stroke
5 seconds for simple transactions
Example: Pipelined algorithm for the Mercury page turner
Quality of Service for real time information
Design Tensions in Networked Systems
Client computers and network connections vary greatly in
capacity
Client software may run on various operating systems; it
may be current or an earlier version
System designers wish to control clients; users wish to
configure their own environments
Usability and Cost
Performance may be expensive in hardware or special
software development
User interface development may be a major part of a
software development project
Costs are multiplied if a user interface has to be used on
different computers or migrate to different versions of systems
Web browsers provide a general purpose user interface that
others maintain
118
Extensibility in Web Browsers
Data types:
helper applications, plug-ins
Protocols
HTTP, WAIS, Gopher, FTP, etc.
proxies
Executable code
CGI scripts at server
JavaScript at client
Java applets
Style sheets
Lecture 18
Design for Usability II
Q5: Object Oriented Design
A system generates weather maps using data collected from
unattended weather stations. Each weather station collects
meteorological data and produces summaries of the data.
On request, it sends the summary information to an area
computer. The area computer uses a database of digitized
maps to generate a set of local weather maps.
119
Q5: Noun Identification
A system generates weather maps using data collected from
unattended weather stations. Each weather station collects
meteorological data and produces summaries of the data.
On request, it sends the summary information to an area
computer. The area computer uses a database of digitized
maps to generate a set of local weather maps.
Candidate Classes
Library the name of the system
Book
Journal
Copy
ShortTermLoan event
LibraryMember
Week measure
MemberOfLibrary repeat
Item book or journal
Time abstract term
MemberOfStaff
System general term
Rule general term
120
Q5: Candidate Classes
System general term
WeatherMap
Data same as MeteorologicalData
WeatherStation is this a general term?
MeteorologicalData how does this relate to WeatherStation?
DataSummary how does this relate to DataSummary?
AreaComputer hardware
Database general term
DigitizedMap
Q5: Observations about the Candidate
Classes
WeatherMap is a DigitizedMap
is derived from 1...* DataSummary
WeatherStation has a set of MeteorologicalData
MeteorologicalData
DataSummary is derived from MeteorologicalData
DigitizedMap
Can Meteorological Data be an attribute of WeatherStation?
Can DataSummary be combined with WeatherMap?
Q5: Attributes and operations
WeatherStation
location
metereologicalData
collectData()
getSummary()
WeatherMap
location
date-time
geographicData
weather
gatherData()
printMap() DigitizedMap
location
geographicData
printMap()
Or should
MetereologicalData be a
separate object?
121
Class Diagram
MemberOfStaff
Book Copy Journal is a copy of
1..* 1
LibraryMember
1
0..*
0..12
1
on loan
on loan
Q5: Class Diagram
DigitizedMap
1...*
1
WeatherStation
location
metereologicalData
collectData()
getSummary()
WeatherMap
location
date-time
geographicData
weather
gatherData()
printMap()
summary
Command Line Interfaces
User interacts with computer by typing commands
Allows complex instructions to be given to computer
Facilitates formal methods of specification & implementation
Skilled users can input commands quickly
Requires learning or training
Can be adapted for people with disabilities
Can be multi-lingual
Suitable for scripting / non-human clients
122
Direct Interaction
User interacts with computer by manipulating objects on screen
Can be intuitive and easy to learn
Users get immediate feedback
Not suitable for complex interactions
Does not require typing skills
Straightforward for casual users, slow for skilled users
Icons can be language-independent
Difficult to build scripts
Only suitable for human users
Design for Direct Manipulation
Conceptual models, metaphors, icons
=> there may not be an intuitive model
Navigation around large space
Conventions are growing over the years
=> scroll bars, buttons, help systems, sliders
=> good for users, good for designers
Menus
Easy for users to learn and use
Certain categories of error are avoided
Enables context-sensitive help
Major difficulty is structure of large menus
Scrolling menus (e.g., states of USA)
Hierarchical
Associated control panels
Menus plus command line
Users prefer broad and shallow to deep menu systems
123
Information Presentation
Information to
be displayed
Presentation
software
Display
Information Presentation
Text
precise, unambiguous
fast to compute and transmit
Graphics
simple to comprehend
uses of color
shows variations
Help System Design
Help system design is difficult!
Must prototype with mixed users
Categories of help:
=> Overview and general information
=> Specific or context information
=> Tutorials (general)
=> Cook books and wizards
=> Emergency ("I am in trouble ...")
Must have many routes to same information
Never blame the user!
124
System Considerations of User
Interfaces
Personal computer cycles are there to be used
Any network transfer involves delay
Shared systems have unpredictable performance
Data validation often requires access to shared data
Mobile code poses security risks
Usability and Cost
Performance may be expensive in hardware or special
software development
User interface development may be a major part of a
software development project
Costs are multiplied if a user interface has to be used on
different computers or migrate to different versions of
systems
Web browsers provide a general purpose user interface that
others maintain
Extensibility in Web Browsers
Data types:
helper applications, plug-ins
Protocols
HTTP, WAIS, Gopher, FTP, etc.
proxies
Executable code
CGI scripts at server
JavaScript at client
Java applets
Style sheets
125
Web User Interface: Basic
Web servers
Web browser
Static pages from server
All interaction requires communication with server
Web User Interface: CGI Script
User interface
tables
CGI
Scripts
Web servers
Web browser
Scripts can configure pages
Scripts can validate information
All interaction requires communication with server
Web User Interface: JavaScript
User interface
tables
CGI
Scripts
Web servers
Web browser
JavaScripts can validate information as typed
Some interactions are local
Server interaction constrained by web protocols
Java
Script
html
126
Web User Interface: Applet
Any
server
Web servers
Web browser
Any executable code can run on client
Client can connect to any server
Applets
Device-Aware User Interfaces
Examples of devices:
desk-top computer, fast network connection
laptop computer, intermittent connectivity
PalmPilot, intermittent use of cradle
Smart telephone
Digital camera, camcorder
Device-aware user interfaces are aware of:
=> Performance of device
=> Connectivity
The Importance of Design
Good support for users is more than a cosmetic flourish
Elegant design, appropriate functionality, & responsive system:
=> a measurable difference to their effectiveness
A system that is hard to use:
=> users may fail to find important results,
or mis-interpret what they do find
=> user may give up in disgust
A computer system is only as good as the interface it provides to
its users
127
Lecture 19
Performance of Computer Systems
Moore's Law
Original version:
The density of transistors in an integrated circuit will double
every year. (Gordon Moore, Intel, 1965)
Current version:
Cost/performance of silicon chips doubles every 18 months.
Moore's Law and System Design
Design system: 2000
Production use: 2003
Withdrawn from production: 2013
Processor speeds: 1 1.9 28
Memory sizes: 1 1.9 28
Disk capacity: 1 2.2 51
System cost: 1 0.4 0.01
128
Moore's Law: Rules of Thumb
Planning assumptions:
Every year:
cost/performance of silicon chips improves 25%
cost/performance of magnetic media improves 30%
10 years = 100:1
20 years = 10,000:1
Parkinson's Law
Original: Work expands to fill the time available. (C.
Northcote Parkinson)
Planning assumptions:
(a) Demand will expand to use all the hardware available.
(b) Low prices will create new demands.
(c) Your software will be used on equipment that you have not
envisioned.
False Assumptions
Unix file system will never exceed 2 Gbytes (2
32
bytes).
AppleTalk networks will never have more than 256 hosts (2
8
bits).
GPS software will not last 1024 weeks.
Nobody at Dartmouth will ever earn more than $10,000 per month.
etc., etc., .....
129
Moore's Law and the Long Term
1965 When?
What
level?
2000?
Within your
working life?
Predicting System Performance
Mathematical models
Simulation
Direct measurement
All require detailed understanding of the
interaction between software and systems.
Queues
arrive wait in line service depart
Single server queue
130
Queues
arrive wait in line
service
depart
Multi-server queue
Mathematical Models
Queueing theory
Good estimates of congestion can be made for single-
server queues with:
arrivals that are independent, random events
(Poisson process)
service times that follow families of distributions
(e.g., negative exponential, gamma)
Many of the results can be extended to multi-server
queues.
Utilization: Rule of Thumb
utilization =
mean service time
mean inter-arrival time
When the utilization of any system component
exceeds 30%, be prepared for congestion.
131
Behavior of Queues: Utilization
mean
delay
utilization
1 0
Simulation
Model the system as set of states and events
advance simulated time
determine which events occurred
update state and event list
repeat
Discrete time simulation: Time is advanced in fixed steps
(e.g., 1 millisecond)
Next event simulation: Time is advanced to next event
Events can be simulated by random variables (e.g., arrival
of next customer, completion of disk latency)
Timescale
Operations per second
CPU instruction: 400,000,000
Disk latency: 60
read: 25,000,000 bytes
Network LAN: 10,000,000 bytes
dial-up modem: 6,000 bytes
132
Measurements on Operational Systems
Benchmarks: Run system on standard problem
sets, sample inputs, or a simulated load on the system.
Instrumentation: Clock specific events.
Serial and Parallel Processing
Single thread v. multi-thread
e.g., Unix fork
Granularity of locks on data
e.g., record locking
Network congestion
e.g., back-off algorithms
Example: Performance of Disk Array
Each transaction must:
wait for specific disk platter
wait for I/O channel
signal to move heads on disk platter
wait for I/O channel
pause for disk rotation
read data
Close agreement between: results from queueing theory,
simulation, and direct measurement (within 15%).
133
The Software Process
Requirements
Definition
System and
Software design
Programming
and Unit Testing
Integration and
System Testing
Operation and
Maintenance
Lecture 20
Coding Standards
Tools for Debugging 1
Coding Standards
Or
How to Pound all of your odd-shaped
programmers into a one size fits all hole
134
I think there may be a bug in Joes Code - Please Fix
func GreenEggsNHam(Not SamIAm, Green EggsNHam)
foreach Green TryThem in SamIAm
do EatThem(TryThem) = false
NotInACarNotOnABus(EggsNHam)
func NotInACarNotOnABus(Green EggsNHam)
EatThem(EggsNHam) = true
NotOnAPlane(EggsNHam)
foreach NotLikeThem SamIAm of EggsNHam do
if not EatThem(SamIAm) then
NotInACarNotOnABus(SamIAm)
IDoNotLikeThem(EggsNHam)
Joes Code Following a Sane Coding Standard . . .
func DepthFirstSearch(graph G, vertex v)
foreach vertex w in G
do Encountered(w) = false
RecursiveDFS(v)
func RecursiveDFS(vertex v)
Encountered(v) = true
PreVisit(v)
foreach neighbor w of v do
if not Encountered(w) then
RecursiveDFS(w)
PostVisit(v)
What are Coding Standards
Coding standards are guidelines for code style and documentation.
The dream is that any developer familiar with the guidelines can work
on any code that followed them.
Standards range from a simple series of statements to involved
documents.
135
Areas Typically Covered
Program Design
Naming Conventions
Formatting Conventions
Documentation
Possibly Even Licensing
Why Have Coding Standards
Greater consistency between developers
Easier to develop and maintain
Saves time and money
Prime Directive
Document every time you violate a standard.
No standard is perfect for every application, but failure to
comply with your standards requires a comment
136
Amblers Law of Standards
Industry Standards > organizational standards > project standards > no standards
The more commonly accepted a standard the easier it is for team members to
communicate
Invent standards when necessary, but dont waste time creating something that you
wont be able to use later.
All languages have recommended coding standards available. It is well worth your
effort to find and use industry standards
Push for organizational standards whenever possible
Good Coding Style
Names
Use full English descriptors
Use mixed case to make names readable
Use abbreviations sparingly and consistently
Avoid long names
Avoid leading/trailing underscores
Documentation
Document the purpose of every variable
Document why something is done not just what
Accessors
use getVar() and setVar() functions on all class variable unless class is
being used solely as a data structure (OOP)
Member Functions Documentation
What and why member function does what it does
Parameters / return value
How function modifies object
Preconditions /Postconditions
Concurrency issues
Restrictions
Internal Documentation
Control Structures
Why as well as what the code does
Difficult or complex code
Processing order
137
Three Rules
Coding standards neednt be onerous - find a standard that
works for your team.
Standardize early - the effort to bring your old work into
the standard will be too great otherwise.
Encourage a culture where standards are followed.
Examples of Coding Standards
http://www.ambysoft.com/javaCodingStandards.html
http://www.swtech.com/java/codestd/
http://ccs.hst.nasa.gov/ccspages/policies/standards/coding_standar
ds.html
http://www.scriptics.com/doc/styleGuide.pdf
Lecture 21
Dependable Systems I
Reliability
138
Software Reliability
Failure: Software does not deliver the service expected by
the user (e.g., mistake in requirements)
Fault: Programming or design error whereby the delivered
system does not conform to specification
Reliability: Probability of a failure occurring in operational
use.
Perceived reliability: Depends upon:
user behavior
set of inputs
pain of failure
Reliability Metrics
Probability of failure on demand
Rate of failure occurrence (failure intensity)
Mean time between failures
Availability (up time)
Mean time to repair
Distribution of failures
Hypothetical example: Cars are safer than
airplane in accidents (failures) per hour, but less
safe in failures per mile.
Reliability Metrics for Distributed Systems
Traditional metrics are hard to apply in multi-component
systems:
In a big network, at a given moment something will be giving
trouble, but very few users will see it.
A system that has excellent average reliability may give
terrible service to certain users.
There are so many components that system administrators
rely on automatic reporting systems to identify problem areas.
139
User Perception of Reliability
1. A personal computer that crashes frequently v. a machine
that is out of service for two days.
2. A database system that crashes frequently but comes back
quickly with no loss of data v. a system that fails once in three
years but data has to be restored from backup.
3. A system that does not fail but has unpredictable periods
when it runs very slowly.
Cost of Improved Reliability
$
Up time
99%
100%
Will you spend your money on new functionality
or improved reliability?
Specification of System Reliability
Example: ATM card reader
Failure class Example Metric
Permanent System fails to operate 1 per 1,000 days
non-corrupting with any card -- reboot
Transient System can not read 1 in 1,000 transactions
non-corrupting an undamaged card
Corrupting A pattern of Never
transactions corrupts
database
140
Principles for Dependable Systems
The human mind can encompass only limited complexity:
=> Comprehensibility
=> Simplicity
=> Partitioning of complexity
Principles for Dependable Systems
High-quality has to be built-in
=> Each stage of development must be done well
=> Testing and correction does not lead to quality
=> Changes should be incorporated into the structure
Quality Management Processes
Assumption:
The importance of routine:
Standard terminology (requirements, specification,
design, etc.)
Software standards (naming conventions, etc.)
Internal and external documentation
Reporting procedures
141
Quality Management Processes
Change management:
Source code management and version control
Tracking of change requests and bug reports
Procedures for changing requirements specifications,
designs and other documentation
Release control
Design and Code Reviews
Colleagues review each other's work:
can be applied to any stage of software development
can be formal or informal
The developer provides colleagues with:
documentation (e.g., specification or design), or code listing
talks through the work while answering questions
Most effective when developer and reviewers prepare well
Benefits of Design and Code Reviews
Benefits:
Extra eyes spot mistakes, suggest improvements
Colleagues share expertise; helps with training
An occasion to tidy loose ends
Incompatibilities between modules can be identified
Helps scheduling and management control
Fundamental requirements:
Senior team members must show leadership
Must be helpful, not threatening
142
Process (Plan) Reviews
Objectives:
To review progress against plan (formal or informal)
To adjust plan (schedule, team assignments,
functionality, etc.)
Impact on quality:
Good quality systems usually result from plans that are
demanding but realistic
Good people like to be stretched and to work hard, but
must not be pressed beyond their capabilities.
Statistical Testing
Determine the operational profile of the software
Select or generate a profile of test data
Apply test data to system, record failure patterns
Compute statistical values of metrics under test conditions
Statistical Testing
Advantages:
Can test with very large numbers of transactions
Can test with extreme cases (high loads, restarts, disruptions)
Can repeat after system modifications
Disadvantages:
Uncertainty in operational profile (unlikely inputs)
Expensive
Can never prove high reliability
143
Error Avoidance
Risky programming constructs
Pointers
Dynamic memory allocation
Floating-point numbers
Parallelism
Recursion
Interrupts
All are valuable in certain circumstances, but
should be used with discretion
Defensive Programming
Murphy's Law: If anything can go wrong, it will.
Defensive Programming:
Redundant code is incorporated to check system state after
modifications
Implicit assumptions are tested explicitly
Defensive Programming Examples
Use boolean variable not integer
Test i <= n not i = = n
Assertion checking
Build debugging code into program with a switch to
display values at interfaces
Error checking codes in data, e.g., checksum or hash
144
Some Notable Bugs
Built-in function in Fortran compiler (e
0
= 0)
Japanese microcode for Honeywell DPS virtual memory
The microfilm plotter with the missing byte (1:1023)
The Sun 3 page fault that IBM paid to fix
Left handed rotation in the graphics package
Good people work around problems.
The best people track them down and fix them!
Lecture 22
Dependable Systems II
Validation and Verification
Defensive Programming
Murphy's Law: If anything can go wrong, it will.
Defensive Programming:
Redundant code is incorporated to check system state after
modifications
Implicit assumptions are tested explicitly
145
Defensive Programming Examples
Use boolean variable not integer
Test i <= n not i = = n
Assertion checking
Build debugging code into program with a switch to
display values at interfaces
Error checking codes in data, e.g., checksum or hash
Terminology
Fault avoidance
Build systems with the objective of creating fault-
free systems
Fault tolerance
Build systems that continue to operate when faults
occur
Fault detection (testing and validation)
Detect faults before the system is put into operation.
Fault Tolerance
Basic Techniques:
After error continue with next transaction
Timers and timeout in networked systems
Error correcting codes in data
Bad block tables on disk drives
Forward and backward pointers
Report all errors for quality control
146
Fault Tolerance
Backward Recovery:
Record system state at specific events (checkpoints). After
failure, recreate state at last checkpoint.
Combine checkpoints with system log that allows
transactions from last checkpoint to be repeated automatically.
Fault Tolerance
General Approach:
Failure detection
Damage assessment
Fault recovery
Fault repair
N-version programming -- Execute independent
implementation in parallel, compare results, accept the
most probable.
Validation and Verification
Validation: Are we building the right product?
Verification: Are we building the product right?
In practice, it is sometimes difficult to distinguish between
the two.
That's not a bug. That's a feature!
147
Cleanroom Software Development
Software development process that aims to develop zero-defect
software.
Formal specification
Incremental development with customer input
Constrained programming options
Static verification
Statistical testing
It is always better to prevent defects than to remove them later.
Example: The four color problem.
Static and Dynamic Verification
Static verification: Techniques of verification that do not
include execution of the software.
May be manual or use computer tools.
Dynamic verification
Testing the software with trial data.
Debugging to remove errors.
Static Validation & Verification
Carried out throughout the software development process.
Validation &
verification
Requirements
specification
Design Program
148
Static Verification: Program Inspections
Program reviews whose objective is to detect faults
Code may be read or reviewed line by line.
150 to 250 lines of code in 2 hour meeting.
Use checklist of common errors.
Requires team commitment, e.g., trained leaders
So effective that it can replace unit testing
Inspection Checklist: Common Errors
Data faults: Initialization, constants, array bounds, character strings
Control faults: Conditions, loop termination, compound statements,
case statements
Input/output faults: All inputs used; all outputs assigned a value
Interface faults: Parameter numbers, types, and order; structures and
shared memory
Storage management faults: Modification of links, allocation and de-
allocation of memory
Exceptions: Possible errors, error handlers
Static Analysis Tools
Program analyzers scan the source of a program for possible
faults and anomalies (e.g., Lint for C programs).
Control flow: loops with multiple exit or entry points
Data use: Undeclared or uninitialized variables, unused
variables, multiple assignments, array bounds
Interface faults: Parameter mismatches, non-use of
functions results, uncalled procedures
Storage management: Unassigned pointers, pointer
arithmetic
149
Static Analysis Tools (continued)
Cross-reference table: Shows every use of a variable,
procedure, object, etc.
Information flow analysis: Identifies input variables on which
an output depends.
Path analysis: Identifies all possible paths through the
program.
Test Design
Testing can never prove that a system is correct. It can only
show that (a) a system is correct in a special case, or (b) that it
has a fault.
The objective of testing is to find faults.
Testing is never comprehensive.
Testing is expensive.
Testing and Debugging
Testing is most effective if divided into stages:
Unit testing at various levels of granularity
tests by the developer
emphasis is on accuracy of actual code
System and sub-system testing
uses trial data
emphasis is on integration and interfaces
Acceptance testing
uses real data in realistic situations
emphasis is on meeting requirements
150
Acceptance Testing
Alpha Testing: Clients operate the system in a realistic
but non-production environment
Beta Testing: Clients operate the system in a carefully
monitored production environment
Parallel Testing: Clients operate new system alongside
old production system with same data and compare
results
The Testing Process
System and Acceptance Testing is a major part of a software
project
It requires time on the schedule
It may require substantial investment in datasets,
equipment, and test software.
Good testing requires good people!
Management and client reports are important parts of
testing.
What is the definition of "done"?
Testing Strategies
Bottom-up testing. Each unit is tested with its own test
environment.
Top-down testing. Large components are tested with
dummy stubs.
user interfaces
work-flow
client and management demonstrations
Stress testing. Tests the system at and beyond its limits.
real-time systems
transaction processing
151
Test Cases
Test cases are specific tests that are chosen because they are
likely to find faults.
Test cases are chosen to balance expense against chance of
finding serious faults.
Cases chosen by the development team are effective in
testing known vulnerable areas.
Cases chosen by experienced outsiders and clients will be
effective in finding gaps left by the developers.
Cases chosen by inexperienced users will find other
faults.
Test Case Selection: Coverage of Inputs
Objective is to test all classes of input
Classes of data -- major categories of transaction and
data inputs.
Cornell example: (undergraduate, graduate, transfer, ...)
by (college, school, program, ...) by (standing) by (...)
Ranges of data -- typical values, extremes
Invalid data, reversals, and special cases.
Test Case Selection: Program
Objective is to test all functions of each computer program
Paths through the computer programs
Program flow graph
Check that every path is executed at least once
Dynamic program analyzers
Count number of times each path is executed
Highlight or color source code
Can not be used with time critical software
152
Program Flow Graph
if-then-else
loop-while
Fixing Bugs
Isolate the bug
Intermittent --> repeatable
Complex example --> simple example
Understand the bug
Root cause
Dependencies
Structural interactions
Fix the bug
Design changes
Documentation changes
Code changes
Moving the Bugs Around
Fixing bugs is an error-prone process!
When you fix a bug, fix its environment
Bug fixes need static and dynamic testing
Repeat all tests that have the slightest relevance
(regression testing)
Bugs have a habit of returning!
When a bug is fixed, add the failure case to the test suite
for the future.
153
Regression Testing
Applied to modified software to provide confidence that
modifications behave as intended and do not adversely
affect the behavior of unmodified code.
Basic technique is to repeat entire testing process after
every change, however small.
Real Time Software Development
Testing and debugging need special tools and
environments
Debuggers, etc., can not be used to test real time
performance
Simulation of environment may be needed to test
interfaces -- e.g., adjustable clock speed
General purpose tools may not be available
Software Engineering for Real Time
The special characteristics of real time computing require
extra attention to good software engineering principles:
Requirements analysis and specification
Development of tools
Modular design
Exhaustive testing
Heroic programming will fail!
154
Management III
Managing People
Administration
Return of laptops and wireless cards
-> Dates for return will be announced on "Notices"
-> All equipment must be returned before the
examination. Bring the receipt to the exam.
-> If an extension granted, (e.g., independent
research) must return and be issued again
If any repairs needed, please swap for replacement since
warranty runs out on December 15.
Administration
Early examination
December 7, 10:00 to 11:30, Upson 5130
Send email to rosemary@cs.cornell.edu if you plan
to take the early examination, by December 5
All laptops and wireless cards must be returned
before the examination
155
Managing People
Theoretical:
Organizational behavior
Industrial psychology
Group behavior
Cognitive fundamentals
Economic motivation
Maslow's Hierarchy of Needs
Self-realization needs
Esteem needs
Social needs
Safety needs
Physiological needs
Software Engineering Basics
Professional staff are the major cost of software
Professional staff vary greatly in productivity
=> Ability
=> Education and training
=> Motivation
=> Interaction with colleagues and leaders
=> Work environment
People are productive when happy and happy when
productive
156
Software is Built by Teams
Best size for a team is 3 to 8 people
Team members may include:
developers (from trainee to expert)
domain experts
graphic or interface designers
software librarians
testers
Teams must have:
administrative leadership (manager)
technical leadership
Group Working
50%
interaction
with
others
20%
non-productive
30%
working alone
Communication
Informal
Kitchen, smokers' doorway, after work, etc.
Walkabout (tours)
Ad hoc meetings
Staff meetings (non-technical)
Example: Tektronics
Technical meetings
Facilitation
Record of decisions
157
Administrative Leader (Manager)
Personnel
Assigning tasks
Hiring, promoting, etc.
Resources
Budgets
Space, facilities
Equipment
Project management
Relationships with other teams and clients
Project plan and schedule
Hiring Criteria
Productivity is a combination of:
Analytic ability
Verbal ability and communication skills
Education
Application domain knowledge
Adaptability and inquisitiveness
Personality and attitude
Platform experience
Programming language experience
Staff Retention
Technically interesting work
up to date hardware and software
opportunities to learn and experiment
Feeling of appreciation
management recognition
money and promotion
Working conditions
space, light, noise, parking
flexibility
Organizational dynamics
158
Firmness
Managers must be firm when needed:
Assignment of tasks must be equitable and open;
everybody will have to tackle some of the dreary tasks
Carrots are better than sticks, but poor performance must
be addressed.
Nobody is indispensable; nobody should be allowed to
think that they are indispensable
Technical Challenges
Canceling projects
Example: the Andrew window manager
Changes of environment
Example: the World Wide Web
Technical tinkering v. needed re-engineering
Turning a Group Around
To turn a weak group into a strong one is the greatest
challenge of leadership
The art of the possible
Promotion of the best over the old leaders
Using opportunities to reorganize
Resignations and terminations
Respect people who try, yet refuse to accept problem areas
Brutal and abrupt rarely equals persistent and firm
159
How to be Led
As a junior member of a team, what can you do to make it
productive?
Lecture 26
Risk in Software Engineering
Failures and Risks
Software development projects can fail in many ways:
Bad software engineering
Late, over budget
Lack of function, full of bugs, bad performance
Changing circumstances
Changing markets
Better alternatives
Changes of management
The biggest single source of problems is poor
understanding of requirements
160
Managing Risk
Manage projects to avoid risk:
Open and visible software process
=> Avoid surprises
Continual review of requirements
Willingness to modify or cancel projects
Canceling a Project
Example: Andrew Window Manager (wm)
Technically superior to X (MIT's Athena project) in 1986
but ... Digital Equipment Corporation turning X into a
product with massive support
nobody ready to support wm
Therefore wm cancelled in 1986, Andrew user interface
and applications ported to X
Failure to Cancel a Project
Example: University F developed a novel programming
language.
From 1985 to 1989, this was a promising language for
simple programming of window-based applications
By 1990, clearly not gaining acceptance beyond
University F
But development continued for many more years (about
$500K)
Not cancelled because ...
161
Too Big to Cancel!
Example: University A has antiquated administrative systems.
Senior management decides to replace them all with commercial
packages from X. The timetable and budget are hopelessly
optimistic.
Staff get dispirited.
The Chief Information Officer finds another job.
A new Chief Information Officer is appointed.
What should she do?
We are doing it the Wrong Way!
Example: University B has a (big) joint project with
Company Y to develop a new computer operating system.
After two years work, a junior software developer
persuades the university leader that the technical approach
is wrong.
What should the university do?
What should the company do?
How to Stop Gracefully
It is harder to cancel a project than to start it.
It is harder to withdraw a service than introduce it.
Considerations
The proponents of the system must now reverse their public
stance.
=> Management of expectations
Users of the service need a migration strategy.
Technical staff must have a graceful path forward.
162
Time to Complete a Software Project
Large software projects typically take at least two years from
start to finish
Formative phase -- changes of plan are easy to
accommodate
Implementation phase -- fundamental changes are almost
impossible
Yet many things can change in two years.
A Sense of Urgency
Example: A not-for-profit corporation is developing a
system for a government organization.
By 1996 all research had been completed and the system
demonstrated successfully with real users.
In 2000, the system is still not in full production
Reasons:
=> Incremental improvements to the software
=> Repeated requests for more functionality
=> Reluctance to reorganize clerical staff
Nobody had a sense of urgency
Overtaken by Events
Example: University C has a project to develop a digital
library system, with funds from Company Z , private
foundations and the government.
By 1993 an extensive system is running at the university
and Z is marketing the technology to its customers.
By 1994 it is clear that web browsers and web formats
(though technically weak) are becoming widely adopted.
=> What should the university do?
=> What should the company do?
163
Changing Requirements and Design
Example: The CNRI Handle System -- a high performance,
distributed system to map names to resources (1994-99).
In 1994 only web browser was Mosaic
In 1994 Java did not exists
In 1994 mirroring and caching utilities were not available
In 1994 commercial interest was limited
Design decisions made in 1994 had to be changed. Software
was rewritten and greatly improved in 1998/9.
If a job's worth doing, it's worth doing twice!
Changes of Leadership
Many projects are wasted because of management changes
Example: In 1988, Company W gave University D $1,000,000
to port a new operating system to its personal computers. The
work was well done, on time.
Company W changed its president and senior technical staff
during the project. The work was wasted.
A decade later and several presidents later, Company W is
releasing a modern version of the same operating system.
A graduate student from University D is now Senior Vice
President of Company W!
Client Oversight
When work is out-sourced, the client must be vigilant.
Example: Company G was the world's leader in software for
optimization (e.g., linear and integer programming). G had
implemented several packages for various manufacturers.
An operating system Company H contracted with G to
develop an optimization package for its new operating system.
The package was late, performed badly and disliked by
customers.
What went wrong? What can we learn?
164
Too Difficult!
Example: A development team at University E was given
government funds to build a high-performance gateway
from protocol x to protocol y.
A promising young developer was hired and assigned to
this task
The project was too difficult for him, but he hid his
problems for many months.
The project produced nothing of value.
What can we learn from this experience?
Engineering and Marketing
Corporate engineering & marketing divisions at cross purposes:
Examples:
Xerox's Palo Alto Research Center pioneered window
managers, Ethernet, graphical user interfaces, font managers, etc,
=> Apple, Adobe, Digital, etc. brought them to the market
IBM would not bring its first Unix workstation to the market
until the software had been largely rewritten
=> Sun's early workstations were unreliable but sold well
Senior Management Dynamics
Directors and shareholders appoint the President
=> The President does not want to admit failures
The President appoints the Chief Information Officer
=> The CIO does not want to admit failures
The CIO appoints the computing managers
=> The computing mangers do not want to admit failures
The computing managers appoint the developers
=> The developers do not want to admit failure
Everybody pretends that things are going well
165
Senior Management Dynamics
At last the troubles can not be hidden ...
Directors and shareholders try to blame the President
The President tries to blame the Chief Information Officer
The CIO tries to blame the computing managers
(and grumbles about the President)
The computing managers try to blame the developers
(and grumble about the CIO)
The developers grumble about their managers
What can we do better?
Sobering Thoughts
Major computing projects are very complex. Inevitably there
are delays and failures.
Few organizations know how to manage risk & uncertainty.
The best CIO's
=> Manage to minimize risk
=> Have the confidence of their staff who keep them
truthfully informed
=> Have the self-confidence to keep their seniors
truthfully informed
The End
Good process leads to good software:
the limits of heroic efforts
Minimize risks:
visible process
function v. time v. cost
The importance of people
Requirements, requirements, requirements!

Software Engineering New1 1062217755 2

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Software Engineering New1 1062217755 2

Uploaded by

Copyright:

Available Formats

1

You might also like