Work Flows

Workflow Manager, you define a set of instructions called a workflow to execute mappings you build in the
Designer. Generally, a workflow contains a session and any other task you may want to perform when
you run a session. Tasks can include a session, email notification, or scheduling information. A workflow
is a set of instructions that tells the Integration Service how to run tasks such as sessions, email
notifications, and shell commands.

Workflow Manager Tools

The Workflow Manager consists of three tools to help you develop a workflow:
♦Task Developer. Use the Task Developer to create tasks you want to run in the workflow.
♦Workflow Designer. Use the Workflow Designer to create a workflow by connecting tasks with links.
You can also create tasks in the Workflow Designer as you develop the workflow.
♦ Worklet Designer. Use the Worklet Designer to create a worklet.

Workflow Tasks
1. Assignment.
2. Command
3. Control
4. Decision
5. Email
6. Event- Raise
7. Event-Wait
8. Session
9. Timer
Steps to develop Workflow
1. Create a workflow. Create a workflow in the Workflow Designer
2. Add tasks to the workflow. You might have already created tasks in the Task
Developer. Or, you can add tasks to the workflow as you develop the workflow in the
Workflow Designer.
3. Connect tasks with links. After you add tasks to the workflow, connect them with
links to specify the order of execution in the workflow.
4. Specify conditions for each link. You can specify conditions on the links to create
branches and dependencies
5. Validate workflow. Validate the workflow in the Workflow Designer to identify errors.
6. Save workflow. When you save the workflow, the Workflow Manager validates the
workflow and updates the repository.
7. Run workflow. In the workflow properties, select an Integration Service to run the
workflow. Run the workflow from the Workflow Manager, Workflow Monitor, or pmcmd.
You can monitor the workflow in the Workflow Monitor.
à Before you can run a workflow, you must assign an Integration Service to run it. You can choose an
Integration Service to run a workflow by editing the workflow properties. You can also assign an
Integration Service from the menu ( Workflows > Edit. Or ServicesàAssign Integration service)
Links

Use links to connect each workflow task. You can specify conditions with links to create
branches in the workflow. The Workflow Manager does not allow you to use links to
create loops in the workflow. Each link in the workflow can run only once.
Link Conditions

Use predefined or user-defined workflow variables in the link condition. If the link
condition evaluates to True, the Integration Service runs the next task in the workflow. If
the link condition evaluates to False, the Integration Service does not run the next task
in the workflow.

Scheduling a Workflow
à By default, the workflow runs on demand. à Each workflow has an associated scheduler. A scheduler is
a repository object that contains a set of schedule settings. You can create a non-reusable scheduler for
the workflow
à To create a reusable scheduler:
1. In the Workflow Designer, click Workflows > Schedulers.

Sessions
à A session is a set of instructions that tells the Integration Service how and when to move data from
sources to targets.
The session has the following tabs, and each of those tabs has multiple settings:
♦ General tab. Enter session name, mapping name, and description for the Session
task, assign resources, and configure additional task options.
♦ Properties tab. Enter session log information, test load settings, and performance
configuration.
♦ Config Object tab. Enter advanced settings, log options, and error handling
configuration.
♦ Mapping tab. Enter source and target information, override transformation properties,
and configure the session for partitioning.
♦ Components tab. Configure pre- or post-session shell commands and emails.
♦ Metadata Extension tab. Configure metadata extension options.
Understanding Buffer Memory
à When you run a session, the Integration Service process starts the Data
Transformation Manager (DTM). The DTM allocates buffer memory to the session at
runtime based on the DTM Buffer Size setting in the session properties.
à The Integration Service allocates at least two buffer blocks for each source and target partition. Use the
following calculation to determine buffer block requirements:
[(total number of sources + total number of targets)* 2] = (session buffer blocks)

You configure buffer memory settings by adjusting the following session parameters:
♦ DTM Buffer Size. The DTM buffer size specifies the amount of buffer memory the Integration Service
uses when the DTM processes a session. Configure the DTM buffer size on the Properties tab in the
session properties.
♦ Default Buffer Block Size. The buffer block size specifies the amount of buffer memory used to move a
block of data from the source to the target. Configure the buffer block size on the Config Object tab in the
session properties
The Integration Service specifies a minimum memory allocation for the buffer memory and buffer blocks.
By default, the Integration Service allocates 12,000,000 bytes of memory to the buffer memory and
64,000 bytes per block.
Configuring Automatic Memory Settings
à You can configure the Integration Service to determine buffer memory size and session cache size at
runtime. When you run a session, the Integration Service allocates buffer memory to the session to move
the data from the source to the target.
à You enable automatic memory settings by configuring a value for the Maximum Memory Allowed for
Auto Memory Attributes or the Maximum Percentage of Total Memory Allowed for Auto Memory
Attributes. If the value is set to zero for either of these attributes, the Integration Service disables
automatic memory settings and uses default values.

To configure automatic memory settings for the DTM buffer size:

1.Open the session, and click the Config Object tab.
2.Enter a value for the Maximum Memory Allowed for Auto Memory Attributes.
Note: You must enable automatic memory settings by configuring a value for the
Maximum Memory Allowed for Auto Memory Attributes. If the value is set to zero, the
Integration Service disables automatic memory settings and uses default values.
3.Enter or select a value of auto for the default buffer block size.
4.Click the Properties tab.
5.Enter or select a value of auto for the DTM buffer size.

Configuring Session Cache Memory

The Integration Service can determine memory requirements for the following session
caches:
♦Lookup transformation index and data caches
♦Aggregator transformation index and data caches
♦Rank transformation index and data caches
♦Joiner transformation index and data caches
♦Sorter transformation cache
♦XML target cache

Configuring Maximum Memory Limits

When you configure automatic memory settings for session caches, configure the
maximum memory limits. Configuring memory limits allows you to ensure that you
reserve a designated amount or percentage of memory for other processes. You can
configure the memory limit as a numeric value and as a percent of total memory.
Because available memory varies, the Integration Service bases the percentage value
on the total memory on the Integration Service process machine.

To configure automatic memory settings for session caches:

1.Open the transformation in the Transformation Developer or the Mappings tab of the
session properties.
2.In the transformation properties, select or enter auto for the following cache size
settings:
♦Index and data cache
♦Sorter cache
♦XML cache
3.Open the session in the Task Developer or Workflow Designer, and click the Config
Object tab

4.Enter a value for the Maximum Memory Allowed for Auto Memory Attributes.
This value specifies the maximum amount of memory to use for session caches.
Note: You must enable automatic memory settings by configuring a value for the
Maximum Memory Allowed for Auto Memory Attributes and the Maximum Percentage of
Total Memory Allowed for Auto Memory Attributes. If either of these values is set to
zero, the Integration Service disables automatic memory settings and uses default
values.
5.Enter a value for the Maximum Percentage of Total Memory Allowed for Auto Memory
Attributes.
This value specifies the maximum percentage of total memory the session caches may
use.
Note: You must enable automatic memory settings by configuring a value for the
Maximum Memory Allowed for Auto Memory Attributes and the Maximum Percentage of
Total Memory Allowed for Auto Memory Attributes. If either of these values is set to
zero, the Integration Service disables automatic memory settings and uses default
values.

Configuring Performance Details

àYou can configure a session to collect performance details and store them in the
PowerCenter repository. Collect performance details for a session to view performance
details while the session runs.

à If you configure the session to store performance details, you must also configure the Integration
Service that runs the session to store the run-time information at the verbose level.

To configure performance details:

1.In the Workflow Manager, open the session properties.
2.On the Properties tab, select Collect Performance Data to view performance details
while the session runs.
3.Select Write Performance Data to Repository to view performance details for previous
session runs
4.Click OK.
5.Save the changes to the repository.

Pre- and Post-Session SQL Commands

à You can specify pre- and post-session SQL in the Source Qualifier transformation
and the target instance when you create a mapping.
à When you create a Session task in the Workflow Manager you can override the SQL
commands on the Mapping tab.
à You might want to use these commands to drop indexes on the target before the
session runs, and then recreate them when the session completes.
à The Integration Service runs pre-session SQL commands before it reads the
source. It runs post-session SQL commands after it writes to the target.
à You can use parameters and variables in SQL executed against the source and
target. Use any parameter or variable type that you can define in the parameter file.
à You can enter a parameter or variable within the SQL statement, or you can use a
parameter or variable as the command. For example, you can use a session parameter,
$ParamMyPreSQL, as the source pre-session SQL command, and set
$ParamMyPreSQL to the SQL statement in the parameter file

Guidelines for Entering Pre- and Post-Session SQL Commands

Remember the following guidelines when creating the SQL statements:
♦Use any command that is valid for the database type.
♦Use a semicolon (;) to separate multiple statements.
♦The Integration Service ignores semicolons within /* ...*/.
♦If you need to use a semicolon outside of comments, you can escape it with a
backslash (\).
♦The Workflow Manager does not validate the SQL.

Error Handling
à You can configure error handling on the Config Object tab. You can choose to stop or continue the
session if the Integration Service encounters an error issuing the pre- or post- session SQL command.

Using Pre- and Post-Session Shell Commands

The Workflow Manager provides the following types of shell commands for each
Session task:
♦Pre-session command. The Integration Service performs pre-session shell
commands at the beginning of a session. You can configure a session to stop or
continue if a pre-session shell command fails.
♦Post-session success command. The Integration Service performs post-session
success commands only if the session completed successfully.
♦Post-session failure command. The Integration Service performs post-session
failure commands only if the session failed to complete.

Session Parameters
Session parameters represent values you can change between session runs, such as
database connections or source and target files. You use a session parameter in
session or workflow properties and define the parameter value in a parameter file. You
can also create workflow variables in the workflow properties and define the values in a
parameter file.

Parameter Naming Convention Description

Type
Session Log $PMSessionLogFile Built-in parameter that defines the
File name of the session log between
session runs. Enter
$PMSessionLogFile in the Session
Log File name field on the
Properties tab.
Number of $DynamicPartitionCount Built-in parameter that defines the
Partitions number of partitions for a session.
Use this parameter with the Based
on Number of Partitions run time
partition option.
Source File $InputFileName User-defined parameter that defines
a source file.
Enter the source file parameter in
the Source Filename field on the
Mapping tab.
Lookup File $LookupFileName User-defined parameter that defines
a lookup file name.
Enter the lookup file parameter in
the Lookup Source Filename field on
the Mapping tab.
Target File $OutputFileName User-defined parameter that defines
a target file name.
Enter the target file parameter in the
 Output Filename field on the
Mapping tab.
Reject File $BadFileName User-defined parameter that defines
a reject file name.
Enter the reject file parameter in the
Reject Filename field on the
Mapping tab.

Database $DBConnectionName User-defined parameter that

Connection defines relational database
connections for a source, target,
lookup, or stored procedure
across session runs.
Enter the database connection
parameter in the relational
connection session properties.
General Session $ParamName User-defined parameter that
Parameter defines any other session
property. For example, you can
use this parameter to define a
table owner name, table name
prefix, FTP file or directory name,
lookup cache file name prefix, or
email address. You can use this
parameter to define source,
lookup, target, and reject file
names, but not the session log file
name or database connections.

Naming Conventions
In a parameter file, folder and session names are case sensitive. You need to use the
appropriate prefix for all user-defined session parameters.

ParametersParameter Naming Convention

Type
Database Connection $DBConnectionName
Reject File $BadFileName
Source File $InputFileName
Target File $OutputFileName
Lookup File $LookupFileName
General Session $ParamName
Parameter
Edit this page (if you have permission) |
Google Docs -- Web word processing, presentations and spreadsheets.