Fundamentals of Websphere MQ Series

bijugs@acm.org

Agenda
Websphere MQ Introduction Websphere MQ Objects Websphere MQ Administration Basics Hands-on part 1 Distributed MQ

Websphere MQ Clusters

Hands-on part 2 Websphere MQ Programming Basics Hands-on part 3

bijugs@acm.org

Introduction

bijugs@acm.org

Messaging and Queuing

Telephone answering machine Punched Cards and processing Clerical Data Input and processing Information Management Systems (IMS) Customer Information Control System (CICS) Telecommunications Access Methods (TCAM)

Virtual Telecommunications Access Method

bijugs@acm.org

Messaging and Queuing

Messaging is

an availability adaptation technique used for the transfer of information between two entities without regard for the immediate availability and accessibility of either

Queuing

a time adaptation technique user for saving information until the intended message receiver is ready to receive it

Messaging and queuing is both asynchronous and connection less Note: entities can be program, persons, systems etc
bijugs@acm.org

Requirements
Distributed applications and systems Increase in program to program communication Heterogeneous Platforms Multiple communication protocols Maximize resource utilization Development complexity and resource skill set

bijugs@acm.org

Websphere MQ (MQ Series until '02)

IBM middleware for messaging & queuing Launched on MVS/ESA in 1992 Currently available on all major platforms

Windows UNIX (AIX\Linux\Sun Solaris, HP-UX) AS/400 Z/OS

Common API (MQI) on all platforms Supports multiple programming languages

C, C++, JAVA, VB, COBOL,PL/1,Assembler

Supports JMS standard APIs for messaging

bijugs@acm.org

MQ Objects

bijugs@acm.org

MQ Message
String of bytes meaningful to the entity using it. Two parts to MQ messages Application Data

Content and structure defined and understood by the application program Contains control information like type of message and priority which are understood and used by MQ

Message Descriptor

Default length is 4 MB; the max length is 100 MB Length can be restricted by the definition of MQ objects and the system storage Messages can be segmented or grouped

bijugs@acm.org

MQ Messages

Message Types (Functional)

Datagram Request Reply Report

Message Retention Types

Persistent Non-persistent

bijugs@acm.org

MQ Message Descriptor
Version Message ID/Correlation ID Persistent/Non persistent Priority Date and time Life time of a message Coded Character Set Id Format Sender application and type Report options/Feedback Back out counter Segmentation/Grouping information

bijugs@acm.org

MQ Object Types
Queue manager Queue Process Definitions Channels Name lists System Default Objects Clusters Authentication Information Objects

bijugs@acm.org

MQ Object Names
Up to 48 characters for all objects except channels Up to 20 characters for channels

Uppercase A- Z Lowercase a - z Numerics 0 - 9 Period (.) Underscore (_) Forward slash (/)* Percent sign (%)*

Blanks are not allowed

* - If used, names should be enclosed in double quotes since these are special characters
bijugs@acm.org

Managing MQ Objects
Control commands typed in through key board. MQSC commands through key board or from files Programmable Command Format (PCF) Messages MQ Administration Interface (MQAI) call from a program MQ Explorer snap-in and MQ services snap-in running under Microsoft management Console*

* - Applicable to Windows platform only

bijugs@acm.org

Message Queue Manager (MQM)

Manages queues of messages for applications Provides application programming interface MQI Uses existing network facilities to transfer messages Coordinates database and queue data updates Segments messages and assembles them Allows message grouping and disassembling Can send message to more than one destination Provides administrative functions on objects Special events like triggers and instrumentation events are generated

bijugs@acm.org

MQ Queues
For Applications Local Queues Remote Queues Alias Queues Model Queues Dynamic Queues Used by MQ Initiation Queues Transmission Queues Cluster Transmission Queues Dead-letter Queues Command Queues Reply-to queues Event Queues

bijugs@acm.org

Process definition
Defines an application that starts in response to a trigger Attributes include application type, application ID (path) and data specific to the application

bijugs@acm.org

Channels
Logical communication link Communication path from one MQM to another Used to move messages between MQMs Shields applications from underlying protocols Message Channels connects two MQMs

Uses Message Channel Agents (MCA) Unidirectional Requires a sender, receiver channels and a protocol

MQI Channels connects MQ client to server

Bidirectional Used for MQ calls and responses only

bijugs@acm.org

Namelist
Contains a list of MQ objects; queue, cluster, authinfo Typically used by trigger monitors to identify a group of queues Used in clustered environment to inform the availability of an MQ object to clusters Maintained independent of application and can be modified without stopping it Multiple applications can use a Namelist

bijugs@acm.org

MQ Overview1

bijugs@acm.org

MQ Overview

MQ Client Application MQPUT MQI Channel

MQ Server (MQM1)
Remote Q

MQ Server (MQM2)
Local Q

MQI Channel

Sender Channel

Receiver Channel

bijugs@acm.org

Putting Together1

bijugs@acm.org

System Default Objects

bijugs@acm.org

System Default Objects

bijugs@acm.org

System Default Objects

bijugs@acm.org

Basic MQ Administration

bijugs@acm.org

Basic Administration*
Define MQ Objects Display MQ Objects Alter MQ Objects Delete MQ Objects Miscellaneous

* - Control Commands and MQSC Commands

bijugs@acm.org

MQM Administration

CRTMQM

Create MQ Manager

e.g. CRTMQM /q TESTQMGR /q sets the MQM as default

STRMQM

Start MQ Manager Stop MQ Manager

e.g. STRMQM TESTQMGR

ENDMQM

e.g. ENDMQM TESTQMGR

DLTMQM
bijugs@acm.org

Delete MQ Manager

e.g. ENDMQM TESTQMGR

MQM Administration

Stop MQ Manager ENDMQM Options By default MQM is quiesced and so takes time -c to notifies all applications to stop but doesn't wait for MQM to end -c is same as issuing ENDMQM with no options -w waits until apps stop and MQM ends -i immediate and doesn't wait for apps to disconnect -p pre-emptive and should be used with caution

bijugs@acm.org

Managing MQM Objects

MQSC Commands by running RUNMQSC Verify a command without running it Run a command on a local queue manager Run a command on a remote queue manager To display MQM properties

STRMQM TESTQMGR (if not started already) RUNMQSC or RUNMQSC TESTQMGR display QMGR (MQSC Command) END

RUNMQSC can accept commands through key board or from a file

RUNMQSC < command.in > execution.out

bijugs@acm.org

Managing Queues

Create a local queue

Display a local queue

DEFINE QLOCAL(TEST.LOCAL.QUEUE) DISPLAY QLOCAL(TEST.LOCAL.QUEUE)

Alter a local queue

Delete a local queue

ALTER QLOCAL(TEST.LOCAL.QUEUE) + GET (DISABLED) DELETE QLOCAL(TEST.LOCAL.QUEUE) PURGE

bijugs@acm.org

Managing Trigger Processes

Create a Process

DEFINE PROCESS(PROC1) REPLACE + DESCR('test proc') APPLTYPE(WINDOWS)+ APPLICID('c:\test\prog.exe')

Display a Process Alter a Process

DISPLAY PROCESS(PROC1) ALTER PROCESS(PROC1)USERDATA('12') DELETE PROCESS(PROC1)

Delete a Process

bijugs@acm.org

Enabling Triggers

Enabling trigger in a queue

DEFINE QLOCAL(TEST.LOCAL.QUEUE)REPLACE + TRIGGER TRIGTYPE(first)PROCESS(PROC1)+ INITQ(SYSTEM.DEFAULT.INITIATION.QUEUE)

Trigger Types

every first N messages

bijugs@acm.org

Managing Channels

Create a Channel

DEFINE CHANNEL(MQMA.MQMB)CHLTYPE(sdr)+ XMITQ(MQMB) TRPTYPE(tcp)+ CONNAME(SERVER1.XYZ.COM)

Display a Channel Alter a Channel

DISPLAY CHANNEL(MQMA.MQMB)

Delete a Channel

ALTER CHANNEL(MQMA.MQMB)+ CONNAME (SERVER2.XYZ.COM) DELETE CHANNEL(MQMA.MQMB)

bijugs@acm.org

Managing Namelist

Create a Namelist

DEFINE NAMELIST(CLUSTERNL)+ NAMES(CLUSA,CLUSB)+ NLTYPE(CLUSTER)REPLACE

Display a Namelist Alter a Namelist

DISPLAY NAMELIST(CLUSTERNL)

DEFINE NAMELIST(CLUSTERNL)+ NAMES(CLUSA,CLUSB,CLUSC)+ NLTYPE(CLUSTER)

Delete a Namelist

DELETE NAMELIST(CLUSTERNL)

bijugs@acm.org

Hands-on Part 1 MQ Administration

bijugs@acm.org

Distributed MQ

bijugs@acm.org

Distributed MQ
Transferring messages between Queue Managers Queue Managers can be on the same machine or world apart Queue Managers can be on the same platform or hetrogenous platforms Implemented using Distributed Queue Management (DQM) features of MQ including MQ Clusters

bijugs@acm.org

Distributed MQ Components
Local and remote Queue Managers and queues Local is one to which an application is connected Remote queue definition Local definition of a queue in a remote MQM Transmission Queues Special local queue to store message temporarily before transmitting to remote destination Message channel agents (MCA) Software that handles sending and receiving Channels One way communication link between MQMs

bijugs@acm.org

Distributed MQ Components
Dead letter queue Special queue to store undelivered messages Transport Service Communication protocol independent of MQ MQ supports multiple protocols Platform dependent Channel Initiators and Listeners Initiators are trigger monitors for sender channels Listeners start receiver channels on receiving request from the sender MCA Channel Exit Programs Used for additional processing like encryption

bijugs@acm.org

Distributed MQ Components
M
App Q

M Channel Transport Service (TCP) C A

Listener

C A
XMIT Q
Channel Initiator

App Q

Init Q

M C A Channel Transport Service (TCP)

M C A
XMIT Q

App Q

App Q

QM1

QM2

bijugs@acm.org

Sample DQM Definition1

CHAD Channel Auto Definition

bijugs@acm.org

Sample DQM Definition

On QMA
Define QREMOTE(Q1)+ RNAME(Q1)+ RQMNAME (QMB)+ XMITQ(QMB) DEFINE QLOCAL(QMB)+ USAGE(XMITQ) DEFINE CHANNEL(A.B)+ CHLTYPE(sdr)+ XMITQ (QMB)+ TRPTYPE(tcp)+ CONNAME(SERVERB.COM) DEFINE QLOCAL(Q2) DEFINE CHANNEL(B.A)+ CHLTYPE(rcvr)+ TRPTYPE(tcp)

On QMB
DEFINE QLOCAL(Q1) DEFINE CHANNEL(A.B)+ CHLTYPE(rcvr)+ TRPTYPE(tcp) Define QREMOTE(Q2)+ RNAME(Q2)+ RQMNAME (QMA)+ XMITQ(QMA) DEFINE QLOCAL(QMA)+ USAGE(XMITQ) DEFINE CHANNEL(B.A)+ CHLTYPE(sdr)+ XMITQ (QMA)+ TRPTYPE(tcp)+ CONNAME(SERVERA.COM)

bijugs@acm.org

Channel Types & Combinations

Channel Types Sender Receiver Server Requester Cluster Sender Cluster Receiver Valid Combinations Sender - Receiver Requester - Server Requester - Sender Server - Receiver Cluster Sender Cluster Receiver

bijugs@acm.org

Queue Addressing & Aliases

Queue Name + Queue Manager Name + Transmission Queue (if remote queue) Remote Queue Definition Resolves the remote queue location Queue Manager Alias Definition Specifies alternative names for the message descriptor of a send message Reply-to Queue Alias Definition Specifies alternative names for the reply information in the message descriptor Aliases are characterized by a blank RNAME in the remote queue definition

bijugs@acm.org

Remote Queue Definition

Remote Queue Definition

-

Resolves the remote queue address using the remote queue manager, remote queue name and the transmission queue If the transmission queue is not specified, a transmission queue with the name of the remote queue manager will be used If not available, the default transmission queue of the queue manager will be used

bijugs@acm.org

Define QREMOTE(Q1)RNAME(Q1)+ RQMNAME (QMB)XMITQ(QMB)

Queue Manager Alias

Queue Manager Alias

Define QREMOTE(QMB_Test)RNAME()+ RQMNAME(QMB) - Define QREMOTE(QMB_Test)RNAME()+ RQMNAME(QMB)XMITQ(QMC)
-

Messages with MQM QMB_Test in the descriptor will be modified to QMB before being forwarded XMITQ parameter can be used to steer the message to use a specified channel If transmission queue is not specified, the logic detailed in the previous slide will be used

bijugs@acm.org

Reply-to Queue Alias

Can be used to alter the reply-to queue and queue manager without changing the application At the sender queue manager

In PUT call leave the reply to queue manager blank and set the queue name to QMA_Reply At the replying queue manager

Define QREMOTE(QMA_Reply) + RNAME (Answer_QMA)RQMNAME(QMA_Return) - Define QREMOTE(QMA_Return)RNAME()+ RQMNAME(QMA)

-

Define QLOCAL(QMA_Return)USAGE(XMITQ)

bijugs@acm.org

Reaching Remote Queue Managers

Point to point Multi-hopping Using shared channels Using multiple channels Using MQ clusters

bijugs@acm.org

DMQ Commands

RUNMQCHI -q IQ (Control Command) START CHINIT INITQ(IQ) (MQSC Command)

Start Channel Initiators

Stopping Channel Initiators Alter the init queue to disable GET from queue Start MQM Listeners

RUNMQLSR -t TCP -m MQM-p 1414 (control Command)* START LISTENER IPADDR(X.X.X.X) PORT(nnnn) TRPTYPE(tcp) (MQSC Command)

*-This requires editing of /etc/services and /etc/inetd.conf files to add MQSeries 1414/tcp #MQ Listener MQSeries stream tcp nowait mqm /usr/mqm/bin/amqcrsta -q mqm.name
bijugs@acm.org

DMQ Commands

ENDMQLSR -m MQM (control Command) STOP LISTENER IPADDR(X.X.X.X) PORT(nnnn) TRPTYPE(tcp) (MQSC Command)

Stop MQM Listeners

RUNMQCHL -c CHANNEL -m QM (Control Command) START CHANNEL(channel-name) (MQSC Command)

Start MQ Channel Stop MQ Channel

STOP CHANNEL(channel-name) (MQSC Command)

Display MQ Channel Status

DISPLAY CHSTATUS(channel-name) (MQSC Command)

bijugs@acm.org

DMQ Commands

To test a channel

PING CHANNEL(channel-name) (MQSC Command)

bijugs@acm.org

MQ Clusters

bijugs@acm.org

DQM vs Clusters
DQM Components Local MQM Remote Q definition Xmit Q definition Sender Channel Remote MQM Local Q Receiver Channel Listener Cluster Definitions Local MQM Cluster receiver channl Cluster sender channel Remote MQM Cluster receiver channl Cluster sender channel

bijugs@acm.org

Cluster Advantages
Reduced system administration due to reduced number of definitions Increased availability and scalability due to easy administration and multiple instances of queues Work load balancing using multiple instances of queues

bijugs@acm.org

Cluster Component Details

MQ Cluster is a network of logically related MQMs and need to be unique in an enterprise Cluster Queue Manager is a MQM that is a member of a MQ Cluster and should be unique in the cluster Cluster Queue is a queue hosted in a cluster queue manager and is available to all MQM in a cluster Repository is a collection of information about the queue managers that are part of a cluster Full Repository is one which stores a complete set of information about all MQMs in the cluster Partial Repository is one which stores information which is required to deal with interested MQMs

bijugs@acm.org

Cluster Component Details

Repository Queue manager is a cluster queue manager which stores a full repository

SYSTEM.CLUSTER.REPOSITORY.QUEUE

Cluster Receiver Channel is the receiving end of a channel on which cluster MQMs can receive messages from other MQMs in the cluster Cluster Sender Channel is the sending end of a channel on which cluster MQMs can send cluster information to one of the full repositories Cluster Transmission Queue transmits all messages from a cluster MQM to any MQM in the cluster

SYSTEM.CLUSTER.TRANSMIT.QUEUE

Command Queue is used to make requests and get responses from full repositories bijugs@acm.org

Clustered MQ Components
Cluster CLUA
App Q

Cluster Receiver Channel Transport

XMIT Q

Service (TCP) Cluster Sender Channel

App Q

Full Repository

Full Repository

App Q1

Cluster Receiver Channel Transport Service (TCP)

App Q1

XMIT Q

QM1

QM2

bijugs@acm.org

Cluster Definition
On QM1
RUNMQSC ALTER QMGR REPOS(CLUA) DEFINE CHANNEL(TO.QM1)+ CHLTYPE(CLUSRCVR)+ TRPTYPE(tcp)+ CONNAME(QM1.COM)+ CLUSTER(CLUA) DEFINE CHANNEL(TO.QM2)+ CHLTYPE(CLUSSDR)+ TRPTYPE(tcp)+ CONNAME(QM2.COM)+ CLUSTER(CLUA) DEFINE QLOCAL(Q1)+ CLUSTER(CLUA)
bijugs@acm.org

On QM2
RUNMQSC ALTER QMGR REPOS(CLUA) DEFINE CHANNEL(TO.QM2)+ CHLTYPE(CLUSRCVR)+ TRPTYPE(tcp)+ CONNAME(QM1.COM)+ CLUSTER(CLUA) DEFINE CHANNEL(TO.QM1)+ CHLTYPE(CLUSSDR)+ TRPTYPE(tcp)+ CONNAME(QM1.COM)+ CLUSTER(CLUA) DEFINE QLOCAL(Q)+ CLUSTER(CLUA)

Adding new QM (QM3) to Cluster

RUNMQSC DEFINE CHANNEL(TO.QM3)+ CHLTYPE(CLUSRCVR)+ TRPTYPE(tcp)+ CONNAME(QM3.COM)+ CLUSTER(CLUA) DEFINE CHANNEL(TO.QM1)+ CHLTYPE(CLUSSDR)+ TRPTYPE(tcp)+ CONNAME(QM1.COM)+ CLUSTER(CLUA)
Note: QM3 doesn't store a full repository and points to QM1 repository

bijugs@acm.org

Adding new QM on DHCP to Cluster

RUNMQSC DEFINE CHANNEL(TO.QM3)+ CHLTYPE(CLUSRCVR)+ TRPTYPE(tcp)+ CONNAME('')+ CLUSTER(CLUA) DEFINE CHANNEL(TO.+QMNAME+)+ CHLTYPE(CLUSSDR)+ TRPTYPE(tcp)+ CONNAME(QM2.COM)+ CLUSTER(CLUA)
Note: the channel definition TO.+QMNAME+ will be replaced by TO.QM2

bijugs@acm.org

Verifying a Cluster
Each MQM require a channel initiator to monitor the system defined init Q SYSTEM.CHANNEL.INITQ START CHINIT Each MQM require a channel listener to listen to incoming network requests START LISTENER PORT(1414)+ TRPTYPE(TCP) Verify all the available MQMs in the cluster DISPLAY CLUSQMGR(*) Verify all the channel statuses DISPLAY CHSTATUS(*)

bijugs@acm.org

Load Balancing Using Cluster

Define the same queue in multiple queue managers in the cluster MQ series will use the default algorithm (round robin) to identify MQM and transfer message Default algorithm can be changed using the cluster workload user exit

bijugs@acm.org

Hands-on Part 2 Creating MQ Clusters

bijugs@acm.org

MQ Programming

bijugs@acm.org

Messaging and Queuing Patterns

One to one One to many Many to one Publish/Subscribe Request/reply One-way datagram or send-and-forget

bijugs@acm.org

One to One Pattern 1

bijugs@acm.org

One to Many Pattern 1

bijugs@acm.org

Many to One Pattern 1

bijugs@acm.org

Publish/Subscribe Pattern 1

bijugs@acm.org

Request/Reply Pattern 1

bijugs@acm.org

Send and Forget Pattern1 (Datagram)

bijugs@acm.org

Transaction Management
ACID unit of work Single phase commit Two phase commit with XA specification compliant resource managers BackoutCount and BackoutThreshold attributes to avoid looping

bijugs@acm.org

Message Queuing Interface (MQI)

bijugs@acm.org

MQI Data Structures

bijugs@acm.org

MQ Programming Sequence
Connect to MQM Open MQ Objects Perform n operations Close MQ Objects Disconnect from MQM
MQCONN

MQOPEN

BASIC OP

MQCLOSE

MQDISC

bijugs@acm.org

MQ JAVA Programming

bijugs@acm.org

JAVA Programming Environment

Required jar file

Normally found in the MQ installation directory Environment variable set-up Add com.ibm.mq.jar to the classpath

com.ibm.mq.jar

install_dir is platform dependent Import required package in the code

CLASSPATH=install_dir\lib\com.ibm.mq.jar

Import com.ibm.mq.*;

bijugs@acm.org

Connection Modes

Connection Modes

Binding Mode

Client Connection Mode

bijugs@acm.org

MQ Series JAVA Classes

MQEnvironment

Contains the static variables that control the environment in which a MQQueueManager is constructed Represents the MQ series queue manager Provides inquiry, set,put,get operations for queues Represents the descriptor and data of a MQ message Contains options which control MQQueue.put() method Contains options which control MQQueue.get() method Contains completion codes and error code constants

MQQueueManager

MQQueue

MQMessage

MQPutOptions

MQGetOptions

MQException

bijugs@acm.org

Setting up the Connection

MQEnvironment.hostname=TEST.SERVER.COM MQEnvironment.channel=TEST.CLIENT.CHNL MQEnvironment.port=1638 MQEnvironment.userId=userABC MQEnvironment.password=password MQEnvironment.properties.put (MQC.TRANSPORT_PROPERTY,MQC.TRANSPORT_MQ SERIES) MQQueueManager qMgr = new MQQueueManager (TEST);

bijugs@acm.org

Interacting with Queues - Put

Int openOptions = MQC.MQOO_OUTPUT| MQC.MQOO_FAIL_IF_QUIESCING; MQQueue queue = qMgr.accessQueue (TestQ,openOptions); MQPutMessageOptions pmo = new MQPutMessageOptions(); MQMessage outMsg = new MQMessage(); outMsg.format = MQC.MQFMT_STRING; String msgString = Test message; outMsg.writeString(msgString); queue.put(outMsg,pmo);

bijugs@acm.org

Interacting with Queues - Get

Int openOptions = MQC.MQOO_INPUT_SHARED| MQC.MQOO_FAIL_IF_QUIESCING; MQQueue queue = qMgr.accessQueue (TestQ,openOptions); MQGetMessageOptions gmo = new MQGetMessageOptions(); gmo.options = gmo.options + MQC.MQGMO_SYNCPOINT; gmo.options = gmo.options + MQC.MQGMO_WAIT; gmo.waitInterval = 3000; MQMessage inMsg = new MQMessage(); queue.put(inMsg,gmo);

bijugs@acm.org

Closing the Connection

qMgr.commit(); queue.close(); qMgr.disconnect();

bijugs@acm.org

Hands-on Part 3 MQ Programming

bijugs@acm.org