Grid Architecture

Prof. Ruay-Shiung Chang

March 2004
2
 3

Course Contents
 Grid Architecture
 Open Grid Services Architecture
 Resource and Service Management
 Building Reliable Clients and Services
 Instrumentation and Monitoring
 4

Grid Architecture
Grid Architecture
 6

Why Discuss Architecture?

 Descriptive
• Provide a common vocabulary for use when
describing Grid systems
 Guidance
• Identify key areas in which services are
required
 Prescriptive
• Define standard “Intergrid” protocols and
APIs to facilitate creation of interoperable
Grid systems and portable applications
7
 8

Water Garden in England

 9

One View of Requirements

 Identity & authentication ● Adaptation
 Authorization & policy
● Intrusion detection
 Resource discovery
 Resource characterization
● Resource
management
 Resource allocation
● Accounting &
 (Co-)reservation, workflow
payment
 Distributed algorithms
● Fault management
 Remote data access
 High-speed data transfer
● System evolution
 Performance guarantees
● etc.
 Monitoring
● etc.
● …
 Another View: “Three Obstacles
10

to Making Grid Computing Routine”

1) New approaches to problem solving
• Data Grids, distributed computing, peer-to-peer,
collaboration grids, …
1) Structuring and writing programs
• Abstractions, tools
1) Enabling resource sharing across distinct
institutions
• Resource discovery, access, reservation,
allocation; authentication, authorization, policy;
communication; fault detection and notification;
…
11
 12

Programming & Systems

Problems
 The programming problem
• Facilitate development of sophisticated apps
• Facilitate code sharing
• Requires programming environments
 APIs, SDKs, tools
 The systems problem
• Facilitate coordinated use of diverse resources
• Facilitate infrastructure sharing
 e.g., certificate authorities, information services
• Requires systems
 protocols, services
 13

The Systems Problem:

Resource Sharing Mechanisms That …
 Address security and policy concerns of
resource owners and users
 Are flexible enough to deal with many
resource types and sharing modalities
 Scale to large number of resources, many
participants, many program components
 Operate efficiently when dealing with large
amounts of data & computation
 14

Aspects of the Systems

Problem
1) Need for interoperability when different groups
want to share resources
• Diverse components, policies, mechanisms
• E.g., standard notions of identity, means of
communication, resource descriptions
1) Need for shared infrastructure services to
avoid repeated development, installation
• E.g., one port/service/protocol for remote access to
computing, not one per tool/app
• E.g., Certificate Authorities: expensive to run
 A common need for protocols & services
 Hence, a Protocol-Oriented View 15

of Grid Architecture, that Emphasizes

…
 Development of Grid protocols & services
• Protocol-mediated access to remote resources
• New services: e.g., resource brokering
• “On the Grid” = speak Intergrid protocols
• Mostly (extensions to) existing protocols
 Development of Grid APIs & SDKs
• Interfaces to Grid protocols & services
• Facilitate application development by supplying higher-
level abstractions
 The (hugely successful) model is the Internet
 16

Layered Grid Architecture

(By Analogy to Internet
Architecture)
Application

Internet Protocol Architecture

“Coordinating multiple resources”:
ubiquitous infrastructure services, Collective
app-specific distributed services Application

“Sharing single resources”:

negotiating access, controlling use Resource

“Talking to things”: communication

(Internet protocols) & security Connectivity Transport
Internet
“Controlling things locally”: Access
to, & control of, resources Fabric Link
 17

Protocols, Services,
and APIs Occur at Each Level Applications

Languages/Frameworks

Collective Service APIs and SDKs

Collective Service Protocols
Collective Services

Resource APIs and SDKs

Resource Service Protocols
Resource Services

Connectivity APIs
Connectivity Protocols Local Access APIs and Protocols
Fabric Layer
 18

Important Points
 Built on Internet protocols & services
• Communication, routing, name resolution, etc.
 “Layering” here is conceptual, does not
imply constraints on who can call what
• Protocols/services/APIs/SDKs will, ideally, be
largely self-contained
• Some things are fundamental: e.g.,
communication and security
• But, advantageous for higher-level functions to
use common lower-level functions
 19

The Hourglass Model

 Focus on architecture issues Applications
• Propose set of core services as Diverse global services
basic infrastructure
• Use to construct high-level,
domain-specific solutions
 Design principles Core
• Keep participation cost low services
• Enable local control
• Support for adaptation
• “IP hourglass” model
Local OS
 20

Hourglass
 21

Where Are We With

Architecture?
 No “official” standards exist
 But:
• Globus Toolkit™ has emerged as the de facto
standard for several important Connectivity,
Resource, and Collective protocols
• GGF has an architecture working group
• Technical specifications are being developed
for architecture elements: e.g., security,
data, resource management, information
• Internet drafts submitted in security area
 22

Fabric Layer
Protocols & Services
 Just what you would expect: the diverse mix of
resources that may be shared
• Individual computers, Condor pools, file systems, archives,
metadata catalogs, networks, sensors, etc., etc.
 Few constraints on low-level technology: connectivity
and resource level protocols form the “neck in the
hourglass”
 Defined by interfaces not physical characteristics
 23

Fabrics in general
 24

Connectivity Layer
Protocols & Services
 Communication
• Internet protocols: IP, DNS, routing, etc.
 Security: Grid Security Infrastructure (GSI)
• Uniform authentication, authorization, and message
protection mechanisms in multi-institutional setting
• Single sign-on, delegation, identity mapping
• Public key technology, SSL, X.509, GSS-API
• Supporting infrastructure: Certificate Authorities,
certificate & key management, …
 25

Not too many connections!

 26

Resource Layer
Protocols & Services
 Grid Resource Allocation Management (GRAM)
• Remote allocation, reservation, monitoring, control
of compute resources
 GridFTP protocol (FTP extensions)
• High-performance data access & transport
 Grid Resource Information Service (GRIS)
• Access to structure & state information
 Others emerging: Catalog access, code
repository access, accounting, etc.
 All built on connectivity layer: GSI & IP
 27

Collective Layer
Protocols & Services
 Index servers aka metadirectory services
• Custom views on dynamic resource collections
assembled by a community
 Resource brokers (e.g., Condor Matchmaker)
• Resource discovery and allocation
 Replica catalogs
 Replication services
 Co-reservation and co-allocation services
 Workflow management services
 etc.

Condor: www.cs.wisc.edu/condor
 28

Collectives: The Borgs

Resistance is futile. You will be assimilated.

 29

Example: High-Throughput Computing

API
System SDK

App High Throughput Computing System C-point

Protocol

Collective Dynamic checkpoint, job management, Checkpoint

(App) failover, staging Repository

Collective
Brokering, certificate authorities
(Generic)
API
SDK
Resource Access to data, access to computers,
access to network performance data Access
Protocol
Connect Communication, service discovery (DNS),
authentication, authorization, delegation
Compute
Fabric Storage systems, schedulers Resource
 30

Example:
Data Grid Architecture
App Discipline-Specific Data Grid Application

Collective Coherency control, replica selection, task management,

(App) virtual data catalog, virtual data code catalog, …

Collective Replica catalog, replica management, co-allocation,

(Generic) certificate authorities, metadata catalogs,

Access to data, access to computers, access to network

Resource
performance data, …
Communication, service discovery (DNS),
Connect authentication, authorization, delegation

Fabric Storage systems, clusters, networks, network caches, …

 31

CERN’s Data Grid Application

 32

The Compact Muon Solenoid

 Open Grid Service
Architecture
Minds are like parachutes, they
only function when they are open.
– Thomas Robert Dewar
 34

Service-Oriented Architecture
 Service: an entity that provides
some capability to its clients by
exchanging messages
 Service: defined by identifying
sequences of specific message
exchanges that cause the service to
perform some operations
 36

Service-oriented Architecture
 Great
flexibility in
implemen-
tation and
location:
because all
operations
are defined
in terms of
message
exchanges

SOAP: Simple Object Access Protocol

 37

Service-oriented Architecture
 Definition: a service-oriented architecture
is one in which all entities are services,
and thus any operation visible to the
architecture is the result of message
exchange
Message exchanges services

Underlying
infrastructure
 38

Service-oriented architecture
 Examples:
• A storage service low-level service
• A data transfer service
• A troubleshooting service high-level service
 Two important themes
 39

But, first four personality types

 40

Service-oriented architecture
 Common behaviors can reoccur in
different contexts
• A goal of OGSA design is to allow these
behaviors to be expressed in standard
ways regardless of contexts, so as to
simplify application design and
encourage code reuse
41
 42

Service-oriented architecture
 A higher-level service behavior (data
transfer) can be implemented via the
composition of simpler behaviors
(storage service)
• Ease of composition is a second major
design goal for OGSA
 43

Service-oriented architecture
 Similarity in
protocols
 44

Service-oriented architecture
 By encapsulating service operations
behind a common message-oriented
service interface, service-oriented
architecture encourages service
virtualization, isolating users from
details of service implantation and
location
 45

Service-oriented architecture
Virtualization
 46

Service-oriented architecture
 Virtualization
• Everything is becoming virtual: virtual
stores, virtual workspace, virtual
organization, virtual networks, …
• More than 60 million US workers work
remotely
 47

Service-oriented architecture
 Interaction with a given service is
facilitated by using a standard
interface definition language (IDL),
such as WSDL, to describe the
service’s interfaces.
 48

Service-oriented architecture
 Web service description language
 49

Service-oriented architecture
 IDL: a cornerstone of interoperability
and transparency
 50

Service-oriented architecture
 An IDL defines the operations
supported by a service, by specifying
the messages that a service
consumes and produces
 An interface specification describes
the messages the service expects
but does not define what the service
does in response to those messages
(i.e., its behavior)
 51

Service-oriented architecture
 52

Service-oriented architecture
 53

Service-oriented architecture
 A well-defined interface definition
language and a separation of
concerns between service interface
and implementation simplify the
manipulation and management of
services in four important respects:
service discovery, composition,
specialization, and interface
extension.
 54

Service-oriented architecture
 Service discovery
 55

Service-oriented architecture
 Service discovery
• Service Location Protocol (SLP)
• Jini: A service discovery architecture based
on Java
• Universal Plug and Play (UPnP): Microsoft's
solution to service discovery
• Salutation: A light weight network protocol
independent service discovery protocol
 56

Service-oriented architecture
 Service composition
 57

Service-oriented architecture
 Service specialization: use of
different implementations of a
service interface on different
platforms
 58

Service-oriented architecture
 Interface extension:
allow for specialized
implementations to
add additional,
implementation-
specific functionality,
while still supporting
the common
A value-added extension
interface
 59

Open Grid Service Architecture

 Objectives of OGSA
• Manage resources across distributed
heterogeneous platforms
• Deliver seamless quality of service
• Provide a common base for autonomic
management solutions
• Define open, published interfaces
• Exploit industry standard integration
technologies
 60

Open Grid Service Architecture

 Three principal elements of OGSA: OGSI,
OGSA services, OGSA applications
OGSA main architecture
 61

Open Grid Service Infrastructure

 OGSI: provides a uniform way for
software developers to model and interact
with grid services by providing interfaces
for discovery, life cycle, state
management, creation and destruction,
event notification, and reference
management.
 62

Requesting a service
 63

Important OGSI concepts and

service factory interactions
create service service
grid service handle requester
resource
allocation

service data, keep-alive,

notifications, service
service invocation discovery

service register
service service
instances registry
 64

Open Grid Service Infrastructure

 OGSI and web services: OGSA architecture
enhances Web services to accommodate
requirements of the grid. These enhancements
are specified in OGSI. Over time, it's expected
that much of the OGSI functionality will be
incorporated in Web services standards.
 65

Open Grid Service Infrastructure

 Implementation of OGSI: The Globus
Toolkit 3 (GT3) is the first full-scale
implementation of the OGSI standard.
 66

OGSA Architected Services

 OGSA architected services
 67

OGSA Architected Services

 Grid core services
 68

OGSA Architected Services

 Grid program execution and data
services
 69

OGSA Architected Services

 Grid program execution and data services
hosting
 70

Naming
 Because Grid services are dynamic
and stateful, we need a way to
distinguish one dynamically created
service instance from another.
 Thus, we need a naming scheme for
Grid service instances.
71
 72

Naming
 73

Naming
 74

Naming
 75

Naming
 76

Naming
 77

Naming
 OGSI defines a two-level naming
scheme for Grid service instances
based on simple, abstract, long-lived
Grid service handles (GSH).
 GSH can be mapped by handle
resolution services to concrete but
potentially short-lived Grid service
references (GSR).
 78

Naming
 A GSH is a globally unique name that
distinguishes that specific Grid
service instance from all other Grid
service instances that have existed,
exist now, or will exist in the future.
 A GSH is represented using a
Uniform Resource Identifier.
 79

Naming
 A GSH carries no protocol- or
instance-specific information such as
network address or supported
protocol bindings.
 All other instance-specific
information is encapsulated into a
single abstraction called a Grid
service reference (GSR).
 80

Naming
Resolve (GSH)
client handle
resolver
time<T time>T

GSR1 GSR2

service migrate service

instance at time T instance
 81

Service lifetime management

 The introduction of transient service
instances raises the issue of
determining the service’s lifetime,
that is, determining when a service
can or should be terminated so that
associated resources can be
recovered.
 82

Happiness for a lifetime

If you want happiness for an hour --
take a nap. If you want happiness for
a day -- go fishing. If you want
happiness for a month -- get
married. If you want happiness for a
year -- inherit a fortune. If you want
happiness for a lifetime -- help
someone else. - Chinese Proverb
 83

Service lifetime management

 OGSA addresses this problem
through a soft-state approach in
which Grid service instances are
created with a specified lifetime.
 Three steps:
• Negotiating an initial lifetime
• Explicit terminating
• Requesting a lifetime modification
 84

Soft-state used in RSVP

 What is RSVP? (Resource
( reSerVation
Protocol)
 85

Soft-state used in RSVP

 86

Soft-state used in RSVP

 87

Soft-state used in RSVP

Path message
 88

Soft-state used in RSVP

Merge the reservation

Resv message
 89

OGSA Implementations
Container (good for code reuse)

demarshaling/decoding
protocol adaptation Grid service
termination layer implementation
Grid service invocation

protocol Grid service

termination implementation

protocol Grid service

implementation
termination
 90

OGSA Implementations
 91

OGSA Implementations
 92

OGSA Implementations
 93

Future Directions
 Service and tools
 Implementation
 Semantics
 Scalability
 …