You are on page 1of 255

Introduction

By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Course Objectives
By the end of this course you will: Understand how to use the major PowerCenter components for development Be able to build basic ETL mappings and mapplets* Be able to create, run and monitor workflows Understand available options for loading target data Be able to troubleshoot most problems

* A mapplet is a subset of a mapping


#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

About Informatica
Founded in 1993 Leader in enterprise solution products Headquarters in Redwood City, CA Public company since April 1999 (INFA) 2000+ customers, including over 80% of Fortune 100 Strategic partnerships with Oracle ,IBM, HP, Accenture, SAP, and many others Worldwide distributorship

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Products
PowerCenter PowerAnalyzer ETL batch and real -time data integration BI reporting web-browser interface with reports, dashboards, indicators, alerts; handles real -time metrics Centralized metadata browsing cross-enterprise, cross enterprise, including PowerCenter, PowerAnalyzer, DBMS, BI tools, and data modeling tools Data access to mainframe, mid-size system and complex files

SuperGlue*

PowerExchange

Data access to transactional applications and PowerCenter Connect products real-time services
* Uses PowerCenter to extract metadata and PowerAnalyzer to display reports
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Resources
www.informatica.com provides information (under Services) on: Professional Services Education Services my.informatica.com sign up to access: Technical Support Product documentation (under Tools online documentation) Velocity Methodology (under Services) Knowledgebase Webzine Mapping templates devnet.informatica.com sign up for Informatica Developers Network Discussion forums Web seminars Technical papers
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Professional Certification


Informatica offers three distinct Certification titles:
Exam A: Architecture and Administration Exam C: Advanced Administration Exam A: Architecture and Administration Exam B: Mapping Design Exam D: Advanced Mapping Design Exams A, B, C, D plus Exam E: Enablement Technologies

For more information and to register to take an exam:


http://www.informatica.com/services/Education+Services/Professional+Certification/ #209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Extract, Transform and Load


Operational Systems
RDBMS Mainframe Other

Decision Support
Data Warehouse

Transaction level data Optimized for transaction response time Current Normalized or De-normalized data

Aggregate data Cleanse data Consolidate data Apply business rules De-normalize data

Aggregated data Historical data

Transform ETL Load

Extract

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

PowerCenter Client Tools


Repository Designer Workflow Workflow Rep Server Manager Manager Monitor Administration Console

Manage repository: Connections Folders Objects Users and groups

Build ETL mappings

Build and start workflows to run mappings

Monitor and start workflows

Administer repositories on a Repository Server: Create/upgrade/delete Configuration Start/stop Backup/restore

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Design and Execution Process


By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Design and Execution Process


1. Create Source definition(s) 2. Create Target definition(s) 3. Create a Mapping with Transformations 4. Create a Session Task 5. Create a Workflow with Task components 6. Run the Workflow and verify the results

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

1. Source Object Definitions


By the end of this section you will: Be familiar with the Designer interface Be familiar with Source Types Be able to create Source Definitions Understand Source Definition properties Be able to use the Data Preview option

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Methods of Analyzing Sources


Source Analyzer

Repository Server TCP/IP

Import from: - Relational database - Flat file - XML object Create manually

Repository Agent Native Repository

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Analyzing Relational Database Sources


Source Analyzer ODBC Relational DB Source
Table View Synonym
DEF

Repository Server TCP/IP Repository Agent Native Repository

DEF

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Analyzing Relational Database Sources


Editing Source Definition Properties

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Analyzing Flat File Sources


Source Analyzer
Mapped Drive NFS Mount Local Directory

Flat File
DEF

Fixed Width Delimited


Repository Server TCP/IP Repository Agent Native Repository

DEF

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Flat File Wizard


Three-step wizard Columns can be renamed within wizard Text, Numeric and Datetime datatypes are supported Wizard guesses datatype

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Flat File Source Properties

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Data Previewer
Preview data in
Relational database sources Flat file sources Relational database targets Flat file targets

Data Preview Option is available in


Source Analyzer Warehouse Designer Mapping Designer Mapplet Designer
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Using Data Previewer in Source Analyzer Data Preview Example


From Source Analyzer, select Source drop down menu, then Preview Data

Enter connection information in the dialog box


A right mouse click on the object can also be used to preview data

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Using Data Previewer in Source Analyzer Data Preview Results

Data Display View up to 500 rows

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Target Object Definitions

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

2. Target Object Definitions


By the end of this section you will: Be familiar with Target Definition types Know the supported methods of creating Target Definitions Understand individual Target Definition properties

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Creating Target Definitions


Methods of creating Target Definitions Import from relational database Import from XML object Create automatically from a source definition Create manually (flat file or relational database)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Creating Target Automatically from Source


Drag-and-drop a Source Definition into the Warehouse Designer Workspace

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Target Definition Properties

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Mappings

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

3. Mappings
By the end of this section you will be familiar with: The Mapping Designer interface Transformation objects and views Source Qualifier transformation The Expression transformation Mapping validation

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Mapping Designer
Transformation Toolbar Mapping List

Iconized Mapping

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Transformations Objects Used in This Class


Source Qualifier: reads data from flat file & relational sources Expression: performs row-level calculations Filter: drops rows conditionally Sorter: sorts data Aggregator: performs aggregate calculations Joiner: joins heterogeneous sources Lookup: looks up values and passes them to other objects Update Strategy: tags rows for insert, update, delete, reject Router: splits rows conditionally Sequence Generator: generates unique ID values
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Other Transformation Objects


Normalizer: normalizes records from relational or VSAM sources Rank: filters the top or bottom range of records Union: merges data from multiple pipelines into one pipeline Transaction Control: allows user-defined commits Stored Procedure: calls a database stored procedure External Procedure : calls compiled code for each row Custom: calls compiled code for multiple rows Midstream XML Parser: reads XML from database table or message queue Midstream XML Generator: writes XML to database table or message queue More Source Qualifiers: read from XML, message queues and applications

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Transformation Views
A transformation has three views:
Iconized shows the transformation in relation to the rest of the mapping Normal shows the flow of data through the transformation Edit shows transformation ports (= table columns) and properties; allows editing
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Source Qualifier Transformation


Represents the source record set queried by the Server. Mandatory in Mappings using relational or flat file sources

Ports

All input/output Convert datatypes For relational sources: Modify SQL statement User Defined Join Source Filter Sorted ports Select DISTINCT Pre/Post SQL

Usage

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Functions Character


ASCII CHR CHRCODE CONCAT INITCAP INSTR LENGTH LOWER LPAD LTRIM REPLACECHR REPLACESTR RPAD RTRIM SUBSTR UPPER

Character Functions Used to manipulate character data CHRCODE returns the numeric value (ASCII or Unicode) of the first character of the string passed to this function CONCAT is for backward compatibility only. Use || instead

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Functions Conversion

TO_CHAR (numeric) TO_DATE TO_DECIMAL TO_FLOAT TO_INTEGER

Conversion Functions

Used to convert datatypes

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Functions Data Cleansing


INSTR IS_DATE IS_NUMBER IS_SPACES ISNULL LTRIM METAPHONE REPLACECHR REPLACESTR RTRIM SOUNDEX SUBSTR TO_CHAR TO_DATE TO_DECIMAL TO_FLOAT TO_INTEGER

Used to process data during data cleansing METAPHONE and SOUNDEX create indexes based on English pronunciation (2 different standards)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Functions Date


ADD_TO_DATE DATE_COMPARE DATE_DIFF GET_DATE_PART LAST_DAY ROUND (Date) SET_DATE_PART TO_CHAR (Date) TRUNC (Date)

Date Functions

Used to round, truncate, or compare dates; extract one part of a date; or perform arithmetic on a date To pass a string to a date function, first use the TO_DATE function to convert it to an date/time datatype

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Functions Numerical and Scientific


ABS CEIL CUME EXP FLOOR LN LOG MOD MOVINGAVG MOVINGSUM POWER ROUND SIGN SQRT TRUNC

Numerical Functions Used to perform mathematical operations on numeric data

Scientific Functions Used to calculate geometric values of numeric data

COS COSH SIN SINH TAN TANH

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Informatica Functions Special and Test


Special Functions Used to handle specific conditions within a session; search for certain values; test conditional statements IIF(Condition,True,False)

ABORT DECODE ERROR IIF LOOKUP

IS_DATE IS_NUMBER IS_SPACES ISNULL

Test Functions Used to test if a lookup result is null Used to validate data

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Default Values Two Usages


For input and I/O ports, default values are used to replace null values For output ports, default values are used to handle transformation calculation errors (not-null handling)

Selected port

Validate the default value expression

Default value for the selected port

ISNULL function is not required

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Mapping Validation

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Connection Validation
Examples of invalid connections in a Mapping:
Connecting ports with incompatible datatypes Connecting output ports to a Source Connecting a Source to anything Except a Source Qualifier or Normalizer transformation* Connecting an output port to an output port or an input port to another input port

*used for Cobol files


#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Mapping Validation
Mappings must: Be valid for a Session to run Be end-to-end complete and contain valid expressions Pass all data flow rules Mappings are always validated when saved; can be validated without being saved Output Window displays reason for invalidity

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Example Mapping

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

4. Session Task
Server instructions to run the logic of ONE specific mapping

e.g. source and target data location specifications, memory allocation, optional Mapping overrides, scheduling, processing and load instructions

Becomes a component of a Workflow (or Worklet) If configured in the Task Developer, the Session Task is reusable (optional)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

5. Workflow Structure
A Workflow is set of instructions for the Informatica Server to perform data transformation and load Combines the logic of Session Tasks, other types of Tasks and Worklets The simplest Workflow is composed of a Start Task, a Link and one other Task
Link

Start Task

Session Task

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

6. Workflow Monitor
The Workflow Monitor is the tool for monitoring Workflows and Tasks Choose between two views:

Gantt chart Task view

Gantt Chart view

Task view

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Target Load Plan


By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Target Load Plan


Target Load plan, informs the server the order in which to read Source Qualifiers for sending data to Targets A target load plan decides the loading order of targets in a mapping (If we use multiple source qulifiers). A target load order group is the collection of source qualifiers, transformations, and targets linked together in a mapping. You can set the target load order if you want to maintain referential integrity when inserting, deleting, or updating tables that have the primary key and foreign key constraints. It is called as Constraint-based Loading (Only one Source qualifier)

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint based Constraint-based Loading

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint-based Loading
PK1

FK1 PK2

FK2

To maintain referential integrity, primary keys must be loaded before their corresponding foreign keys here in the order Target1, Target2, Target 3
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Setting Constraint-based Loading

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint-based Loading Terminology


Active transformation
Can operate on groups of data rows and/or can change the number of rows on the data flow Examples: Source Qualifier, Aggregator, Joiner, Sorter, Filter

Active source
Active transformation that generates rows Cannot match an output row with a distinct input row Examples: Source Qualifier, Aggregator, Joiner, Sorter (The Filter is NOT an active source)

Active group
Group of targets in a mapping being fed by the same active source

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint-Based Loading Restrictions Cannot have two active groups


PK1

Example 1
With only one Active source, rows for Targets1, 2, and 3 will be loaded properly and maintain referential integrity

FK1 PK2 FK2

PK1

Example 2
With two Active sources, it is not possible to control whether rows for Target3 will be loaded before or after those for Target2

FK1 PK2 FK2

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Example
Target Tables CREATE TABLE DEPT1 (DEPTNO NUMBER(2) CONSTRAINT PK_DEPT1 PRIMARY KEY, DNAME VARCHAR2(14) , LOC VARCHAR2(13) ) ; CREATE TABLE EMP1 (EMPNO NUMBER(4) CONSTRAINT PK_EMP1 PRIMARY KEY, ENAME VARCHAR2(10), JOB VARCHAR2(9), MGR NUMBER(4), HIREDATE DATE, SAL NUMBER(7,2), COMM NUMBER(7,2), DEPTNO NUMBER(2) CONSTRAINT FK_DEPTNO1 REFERENCES DEPT);

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Types of Transformation
1. Passive transformation
1. Operates on one row of data at a time AND 2. Cannot change the number of rows on the data flow 3. Example: Expression,Lookup,Stored Procedure transformation

2. Active transformation
1. Can operate on groups of data rows AND/OR 2. Can change the number of rows on the data flow 3. Examples: Aggregator, Filter, Source Qualifier
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Example
Passive
Tax=Sal *0.1

14 Rows

14 Rows

Active

Deptno=10

14 Rows

5 Rows

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Types of Transformation
1. Connected transformation
1. 2. 3. Is connected in mapping All active and passive can use as connected Connected transformation receives multiple input ports and can provides multiple outputs Is unconnected in mapping Stored and lookup transformations only can use as unconnected unconnected transformation receives multiple input ports and provides only one output

2. Unconnected transformation
1. 2. 3.

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Example
Connected

Unconnected

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Expression Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Expression Transformation
Perform calculations using non-aggregate functions (row level)

Ports Mixed Variables allowed Create expression in an output or variable port Usage Perform majority of data manipulation

Click here to invoke the Expression Editor

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

An expression formula is a calculation or conditional statement for a specific port in a transformation Performs calculation based on ports, functions, operators, variables, constants and return values from other transformations

Expression Editor

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Expression Validation
The Validate or OK button in the Expression Editor will:
Parse the current expression

Remote port searching (resolves references to ports in other transformations)


Parse default values Check spelling, correct number of arguments in functions, other syntactical errors

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Example Expression
Example Expression

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Tasks
Are two types 1. Reusable Tasks 2. Non - Reusable Tasks

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

1. Reusable Tasks
Three types of reusable Tasks
Session Set of instructions to execute a specific Mapping Command Specific shell commands to run during any Workflow Email Sends email during the Workflow

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Reusable Tasks
Use the Task Developer to create reusable tasks These tasks will then appear in the Navigator and can be dragged and dropped into any workflow

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Reusable Tasks in a Workflow


In a workflow, a reusable task is represented with the symbol

Reusable

Non-reusable

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

1. Session Tasks

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Task
Server instructions to run the logic of ONE specific mapping e.g. source and target data location specifications, memory allocation, optional Mapping overrides, scheduling, processing and load instructions Becomes a component of a Workflow (or Work let)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Creating a Session Task


Created to execute the logic of a mapping (one mapping only) Session Tasks can be created in two ways
1. Using Task Developer (Reusable) 2. Using Workflow Designer (Workflow-specific) (Workflow specific)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Using Task Developer (Reusable)


In Work flow manager Click on Select the Session button from the Task Toolbar

Or Select menu Tasks | Create and select Session from the drop -down menu

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Using Workflow Designer (Work Flow -Specific)


In Work flow manager Click on Select the Session button from the Task Toolbar

Or Select menu Tasks | Create and select Session from the drop -down menu

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Task Properties and Parameters


Properties Tab Session Task Session parameter Parameter file

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Task Setting Source Properties


Mapping Tab Session Task Select source instance Set connection Set properties

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Task Setting Target Properties


Mapping Tab Session Task

Select target instance Set connection

Set properties

Note: Heterogeneous targets are supported

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Reusable Session Configurations

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Configuration
Define properties to be reusable across different sessions Defined at folder level Must have one of these tools open in order to access

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Configuration (cont d)

Available from menu or Task toolbar

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Configuration (cont d)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Task Config Object

Within Session task properties, choose desired configuration

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Session Task Config Object Attributes

Attributes may be overridden within the Session task

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

2. Command Task
Specify one or more Unix shell or DOS commands to run during the Workflow Runs in the Informatica Server (UNIX or Windows) environment Command task status (successful completion or failure) is held in the pre-defined task variable $command_task_name.STATUS Each Command Task shell command can execute before the Session begins or after the Informatica Server executes a Session

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Command Task
Specify one (or more) Unix shell or DOS (NT, Win2000) commands to run at a specific point in the workflow Becomes a component of a workflow (or worklet) If created in the Task Developer, the Command task is reusable If created in the Workflow Designer, the Command task is not reusable Commands can also be invoked under the Components tab of a Session task to run pre- or post session -

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Command Task (cont d)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Command Task (cont d)

Add Cmd Remove Cmd

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Command Task(Example)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Command Task(Example)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

3. Email Task
Configure to have the Informatica Server to send email at any point in the Workflow Becomes a component in a Workflow (or Worklet) If configured in the Task Developer, the Email Task is reusable (optional) Emails can also be invoked under the Components tab of a Session task to run pre- or post-session

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Email Task (cont d)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Email Task (Example)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Non-Reusable Tasks

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Non-Reusable Tasks
Six additional Tasks are available in the Workflow Designer
Decision Assignment Timer

Control Event Wait Event Raise

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

1. Decision Task
Specifies a condition to be evaluated in the Workflow Use the Decision Task in branches of a Workflow Use decision task instead of multiple link conditions .

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Decision Task
The Decision task has a predefined variable called $Decision_task_name.condition that represents the result of the decision condition. The Integration Service evaluates the condition in the Decision task and sets the predefined condition variable to True (1) or False (0).

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Decision Task(Example)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

2. Assignment Task
Assigns a value to a Workflow Variable Variables are defined in the Workflow object

General Tab Expressions Tab

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Assignment Task

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

3. Timer Task
Waits for a specified period of time to execute the next Task
General Tab Timer Tab

Absolute Time Datetime Variable Relative Time

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Timer Task modes


Absolute time
vWe specify the exact date and time or we can choose a user-defined workflow variable to specify the exact time. The next task in workflow will run as per the date and time specified.

Relative time
v We instruct the Power Center Server to wait for a specified period of time after the Timer task, the parent workflow, or the top-level workflow starts

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Timer Task(Example)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

4. Control Task
Stop or ABORT the Workflow
Properties Tab General Tab

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Control Task(Example)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

5. Event Wait Task


Pauses processing of the pipeline until a specified event occurs Events can be:
Pre-defined file watch User-defined created by an Event Raise task some where in the workflow

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Event Wait Task File watch

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

6. Event Raise Task


Represents the location of a user defined event The Event Raise Task triggers the user defined event when the Informatica Server executes the Event Raise Task

General Tab

Properties Tab

Used with the Event Wait Task

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Event Wait user defined Example

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Event Wait user defined Example

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Event Wait user defined Example

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Event Wait user defined Example

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Filter Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Filter Transformation
1. Is a condition created with following elements
1. Port or column 2. Operator 3. Operand Example Deptno
1

=
2

10
3

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Filter Transformation
1. Is Active 2. Is connected 3. Use the Filter transformation to filter out rows in a mapping 4. Use to cleanse data (filter unwanted data) 5. The naming convention for a Filter transformation is FIL_TransformationName 6. Condition to reject rows of records which have FIRST_NAME null IIF(ISNULL(FIRST_NAME),FALSE,TRUE)
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Filter Transformation
Drops rows conditionally

Ports All input / output Specify a Filter condition Usage Filter rows from input flow

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Example Filter

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Worklets
An object representing a set or grouping of Tasks Can contain any Task available in the Workflow Manager Worklets expand and execute inside a Workflow A Workflow which contains a Worklet is called the parent Workflow Worklets CAN be nested Reusable Worklets create in the Worklet Designer Non-reusable Worklets create in the Workflow Designer

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Re-usable Worklet
In the Worklet Designer, select Worklets | Create

Worklets Node Tasks in a Worklet

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Using a Reusable Worklet in a Workflow

Worklet used in a Workflow

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Non-Reusable Worklet
1. Create worklet task in Workflow Designer Right-click on new worklet and select Open Worklet Workspace switches to Worklet Designer 2.

3.

NOTE: Worklet shows only under Workflows node

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Debugger
By the end of this section you will be familiar with: Creating a Debug Session Debugger windows and indicators Debugger functionality and options Viewing data with the Debugger Setting and using Breakpoints Tips for using the Debugger

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Debugger Features
Wizard driven tool that runs a test session View source / target data View transformation data Set breakpoints and evaluate expressions Initialize variables Manually change variable values Data can be loaded or discarded Debug environment can be saved for later use

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Debugger Interface
Edit Breakpoints Debugger Mode indicator Solid yellow arrow is current transformation indicator

Flashing yellow SQL indicator

Output Window Debugger Log

Transformation Instance Data window

Target Instance window

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Set Breakpoints
1. Edit breakpoint 2. Choose global or specific transformation 3. Choose to break on data condition or error. Optionally skip rows. 4. Add breakpoint(s) 5. Add data conditions

6. Continue (to next breakpoint)


#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Debugger Tips
Server must be running before starting a Debug Session When the Debugger is started, a spinning icon displays. Spinning stops when the Debugger Server is ready The flashing yellow/green arrow points to the current active Source Qualifier. The solid yellow arrow points to the current Transformation instance

Next Instance proceeds a single step at a time; one row moves from transformation to transformation Step to Instance examines one transformation at a time, following successive rows through the same transformation
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Sorter Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Sorter Transformation
Can sort data from relational tables or flat files Sort takes place on the Informatica Server machine Multiple sort keys are supported The Sorter transformation is often more efficient than a sort performed on a database with an ORDER BY clause Is useful to eliminate duplicate records so it is called as active transformation

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Sorter Transformation
Sorts data from any source, at any point in a data flow
Sort Keys

Ports Input/Output Define one or more sort keys Define sort order for each key Example of Usage Sort data before Aggregator to improve performance
Sort Order

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Sorter Properties

Ensure sufficient memory is available on the Informatica Server (else Session Task will fail)

Cache size can be adjusted. Default is 8 Mb.

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregator Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregator Transformation
By the end of this section you will be familiar with: Basic Aggregator functionality Creating subtotals with the Aggregator Aggregator expressions Aggregator properties Using sorted data

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregator Transformation
Performs aggregate calculations

Ports Mixed I/O ports allowed Variable ports allowed Group By allowed Create expressions in variable and output ports Usage Standard aggregations

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregate Expressions
Aggregate functions are supported only in the Aggregator Transformation

Conditional Aggregate expressions are supported:

Conditional SUM format: SUM(value, condition)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregator Functions
AVG COUNT FIRST LAST MAX MEDIAN MIN PERCENTILE STDDEV SUM VARIANCE

Return summary values for non-null data in selected ports Use only in Aggregator transformations Use in output ports only Calculate a single value (and row) for all records in a group Only one aggregate function can be nested within an aggregate function Conditional statements can be used with these functions

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregator Properties
Sorted Input Property

Instructs the Aggregator to expect the data to be sorted Set Aggregator cache sizes for Informatica Server machine

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregating Unsorted Data


Unsorted data Group By: - store - department - date

No rows are released from Aggregator until all rows are aggregated

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregating Sorted Data


Data sorted by: - store - department - date Group By: - store - department - date

Each separate group (one row) is released as soon as the last row in the group is aggregated

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Aggregate cache. The Integration Service stores data in the aggregate cache until it completes aggregate calculations. It stores group values in an index cache and row data in the data cache. Aggregate expression. Enter an expression in an output port. The expression can include non-aggregate expressions and conditional clauses Group by port. Indicate how to create groups. The port can be any input, input/output, output, or variable port. When grouping data, the Aggregator transformation outputs the last row of each group unless otherwise specified. Sorted input. Select this option to improve session performance. To use sorted input, you must pass data to the Aggregator transformation sorted by group by port, in ascending or descending order.

Components of the Aggregator Transformation

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Monitoring Workflows

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Monitoring Workflows
By the end of this section you will be familiar with: The Workflow Monitor GUI interface Monitoring views Server monitoring modes Filtering displayed items Actions initiated from the Workflow Monitor Truncating Monitor Logs

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Workflow Monitor
The Workflow Monitor is the tool for monitoring Workflows and Tasks Choose between two views:

Gantt chart Task view

Gantt Chart view

Task view

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Monitoring Current and Past Workflows


The Workflow Monitor displays only workflows that have been run Displays real-time information from the Informatica Server and the Repository Server about current workflow runs

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Monitoring Operations
Perform operations in the Workflow Monitor
Stop, Abort, or Restart a Task, Workflow or Worklet Resume a suspended Workflow after a failed Task is corrected Reschedule or Unschedule a Workflow

View Session and Workflow logs Abort has a 60 second timeout


If the Server has not completed processing and committing data during the timeout period, the threads and processes associated with the Session are killed

Stopping a Session Task means the Server stops reading data

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Monitoring in Task View


Start Task Server Workflow Worklet Completion Time Time

Status Bar

Start, Stop, Abort, Resume Tasks,Workflows and Worklets

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Filtering in Task View


Monitoring filters can be set using drop down menus. Minimizes items displayed in Task View

Right-click on Session to retrieve the Session Log (from the Server to the local PC Client)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Filter Toolbar

Select type of tasks to filter Select servers to filter Filter tasks by specified criteria Display recent runs

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Truncating Workflow Monitor Logs


Workflow Monitor

Repository Manager Repository Manager s Truncate Log option clears the Workflow Monitor logs

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Joiner Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Joiner Transformation
By the end of this section you will be familiar with: When to join in Source Qualifier and when in Joiner transformation Homogeneous joins Heterogeneous joins Joiner properties Joiner conditions Nested joins

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

When to Join in Source Qualifier


If you can perform a join on the source database, then you can configure it in the Source Qualifier The SQL that the Source Qualifier generates, default or custom, executes on the source database at runtime Example: homogeneous join 2 database tables in same database

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

When You Cannot Join in Source Qualifier


If you cannot perform a join on the source database, then you cannot configure it in the Source Qualifier

Examples: heterogeneous joins

An Oracle table and a DB2 table

A flat file and a database table Two flat files

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Joiner Transformation
Performs heterogeneous joins on different data flows
Active Transformation Ports All input or input / output M denotes port comes from master source Examples Join two flat files Join two tables from different databases Join a flat file with a relational table

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Joiner Conditions

Multiple join conditions are supported

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Joiner Properties
Join types:
Normal (inner) Master outer Detail outer Full outer Set Joiner Caches

Joiner can accept sorted data (configure the join condition to use the sort origin ports)
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Nested Joins
Used to join three or more heterogeneous sources

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Mid-Mapping Join (Unsorted)


The unsorted Joiner does not accept input in the following situations:
Both input pipelines begin with the same Source Qualifier Both input pipelines begin with the same Joiner

The sorted Joiner does not have these restrictions.

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Performance Tuning
By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Performance Tuning
First step in performance tuning is to identify the performance bottleneck in the following order: Target Source Mapping Session System The most common performance bottleneck occurs when the Informatica Server writes to a target database.

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Target Bottlenecks
Identifying
A target bottleneck can be identified by configuring the session to write to a flat file target.

Optimizing
Dropping Indexes and Key Constraints before loading. Increasing commit intervals. Use of Bulk Loading / External Loading.

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Source Bottlenecks
Identifying Add a filter condition after Source qualifier to false so that no data is processed past the filter transformation. If the time it takes to run the new session remains about the same, then there is a source bottleneck. In a test mapping remove all the transformations and if the performance is similar, then there is a source bottleneck. Optimizing Optimizing the Query by using hints. Use Informatica Conditional Filters if the source system lacks indexes.
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Identifying If there is no source bottleneck, add a Filter transformation in the mapping before each target definition. Set the filter condition to false so that no data is loaded into the target tables. If the time it takes to run the new session is the same as the original session, there is a mapping bottleneck. Optimizing Configure for Single-Pass reading Avoid unnecessary data type conversions. Avoid database reject errors. Use Shared Cache / Persistent Cache
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Bottlenecks

Session Bottlenecks
Identifying
If there is no source, Target or Mapping bottleneck, then there may be a session bottleneck. Use Collect Performance Details. Any value other than zero in the readfromdisk and writetodisk counters for Aggregator, Joiner, or Rank transformations indicate a session bottleneck. Low (0-20%) BufferInput_efficiency and BufferOutput_efficiency counter values also indicate a session bottleneck.

Optimizing
Increase the number of partitions. Tune session parameters. DTM Buffer Size (6M 128M) Buffer Block Size (4K 128K) Data (2M 24 M )/ Index (1M -12M) Cache Size Use incremental Aggregation if possible.
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Session Bottlenecks - Memory


Configure the index and data cache memory for the Aggregator, Rank, and Joiner transformations in the Configuration Parameters dialog box The amount of memory you configure depends on partitioning, the transformation that requires the largest cache, and how much memory cache and disk cache you want to use

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

System Bottlenecks
Identifying If there is no source, Target, Mapping or Session bottleneck, then there may be a system bottleneck. Use system tools to monitor CPU usage, memory usage, and paging. On Windows :- Task Manager On Unix Systems tools like sar, iostat. For E.g. sar u (%usage on user, idle time, i/o waiting time) Optimizing Improve network speed. Improve CPU performance Check hard disks on related machines Reduce Paging

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Incremental Aggregation
First Run creates idx and dat files. Second Run performs the following actions: For each i/p record, the Server checks historical information in the index file for a corresponding group, then: If it finds a corresponding group, it performs the aggregate operation incrementally, using the aggregate data for that group, and saves the incremental change If it does not find a corresponding group, it creates a new group and saves the record data When writing to the target Informatica Server Updates modified aggregate groups in the target Inserts new aggregate data Deletes removed aggregate data Ignores unchanged aggregate data Saves209 Nilagiri Block Adithyadata in the index and @8801408841,8790998182 modified aggregate Enclave Ameerpet,HYD data files # , , ,

Incremental Aggregation
You can find options for incremental aggregation on the Transformations tab in the session properties The Server Manager displays a warning indicating the Informatica Server overwrites the existing cache and a reminder to clear this option after running the session
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Commit Points
A commit interval is the interval at which the Informatica Server commits data to relational targets during a session The commit point can be a factor of the commit interval, the commit interval type, and the size of the buffer blocks The commit interval is the number of rows you want to use as a basis for the commit point The commit interval type is the type of rows that you want to use as a basis for the commit point Can choose between the following types of commit interval Target-based commit Source-based commit During a source-based commit session, the Informatica Server commits data to the target based on the number of rows from an active source in a single pipeline

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Commit Points
During a target-based commit session, the Informatica Server continues to fill the writer buffer after it reaches the commit interval When the buffer block is filled, the Informatica Server issues a commit command As a result, the amount of data committed at the commit point generally exceeds the commit interval

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Commit Points
During a source-based commit session, the Informatica Server commits data to the target based on the number of rows from an active source in a single pipeline These rows are referred to as source rows A pipeline consists of a source qualifier and all the transformations and targets that receive data from the source qualifier An active source can be any of the following active transformations: Advanced External Procedure Source Qualifier Normalizer Aggregator Joiner Rank Mapplet, if it contains one of the above transformations
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Commit Points

When the Informatica Server runs a source-based commit session, it identifies the active source for each pipeline in the mapping The Informatica Server generates a commit row from the active source at every commit interval When each target in the pipeline receives the commit row, the Informatica Server performs the commit
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Commit Points

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint-based Loading
PK1

FK1 PK2

FK2

To maintain referential integrity, primary keys must be loaded before their corresponding foreign keys here in the order Target1, Target2, Target 3
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Setting Constraint-based Loading

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint-based Loading Terminology


Active transformation
Can operate on groups of data rows and/or can change the number of rows on the data flow Examples: Source Qualifier, Aggregator, Joiner, Sorter, Filter

Active source
Active transformation that generates rows Cannot match an output row with a distinct input row Examples: Source Qualifier, Aggregator, Joiner, Sorter (The Filter is NOT an active source)

Active group
Group of targets in a mapping being fed by the same active source

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Constraint-Based Loading Restrictions Cannot have two active groups


PK1

Example 1
With only one Active source, rows for Targets1, 2, and 3 will be loaded properly and maintain referential integrity

FK1 PK2 FK2

PK1

Example 2
With two Active sources, it is not possible to control whether rows for Target3 will be loaded before or after those for Target2

FK1 PK2 FK2

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Source Qualifier Transformation


By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Source Qualifier Transformation


Represents the source record set queried by the Server. Mandatory in Mappings using relational or flat file sources

Ports

All input/output Convert datatypes For relational sources: Modify SQL statement User Defined Join Source Filter Sorted ports Select DISTINCT Pre/Post SQL

Usage

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Source Qualifier Properties


Is active transformation Used to read data from tables and flat files Useful to remove duplicates User can modify SQL SELECT statement (DB sources) Source Qualifier can join homogenous tables User can modify WHERE clause User can modify join statement

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Source Qualifier Properties


User can specify ORDER BY (manually or automatically) Pre- and post-SQL can be provided SQL properties do not apply to flat file sources

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Transformation
By Ram Reddy

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Transformation
Can use in two ways
1. Connected 2. Un connected

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

1. Connected Lookup Transformation

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Transformation
By the end of this section you will be familiar with: Lookup principles Lookup properties Lookup conditions Lookup techniques Caching considerations Persistent caches

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

How a Lookup Transformation Works


For each mapping row, one or more port values are looked up in a database table or flat file If a match is found, one or more table values are returned to the mapping. If no match is found, NULL is returned
Lookup value(s) Lookup transformation

Return value(s)

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Transformation
Looks up values in a database table or flat file and provides data to other components in a mapping
Ports Mixed L denotes Lookup port R denotes port used as a return value (unconnected Lookup only see later) Specify the Lookup Condition Usage Get related values Verify if records exists or if data has changed

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Conditions

Multiple conditions are supported

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Properties
Lookup table name Lookup condition Native database connection object name Source type: Database or Flat File

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Properties cont d

Policy on multiple match: Use first value Use last value Report error

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Caching
Caching can significantly impact performance Cached
Lookup table data is cached locally on the Server Mapping rows are looked up against the cache Only one SQL SELECT is needed

Uncached
Each Mapping row needs one SQL SELECT Rule Of Thumb: Cache if the number (and size) of records in the Lookup table is small relative to the number of mapping rows requiring the lookup

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Persistent Caches
By default, Lookup caches are not persistent; when the session completes, the cache is erased Cache can be made persistent with the Lookup properties When Session completes, the persistent cache is stored on the server hard disk The next time Session runs, cached data is loaded fully or partially into RAM and reused A named persistent cache may be shared by different sessions Can improve performance, but stale data may pose a problem

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Caching Properties


Override Lookup SQL option

Toggle caching Cache directory

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Lookup Caching Properties (cont d)


Make cache persistent Set Lookup cache sizes

Set prefix for persistent cache file name Reload persistent cache
#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

2. Unconnected Lookups

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Unconnected Lookups
By the end of this section you will know: Unconnected Lookup technique Unconnected Lookup functionality Difference from Connected Lookup

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Unconnected Lookup
Physically unconnected from other transformations NO data flow arrows leading to or from an unconnected Lookup Lookup data is called from the point in the Mapping that needs it Lookup function can be set within any transformation that supports expressions
Function in the Aggregator calls the unconnected Lookup

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Unconnected Lookup Technique


Use lookup lookup function within a conditional statement
Condition Row keys (passed to Lookup)

IIF ( ISNULL(customer_id),:lkp.MYLOOKUP(order_no))

Lookup function

Condition is evaluated for each row but Lookup function is called only if condition satisfied

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Unconnected Lookup Functionality


One Lookup port value may be returned for each Lookup

Must check a Return port in the Ports tab, else fails at runtime

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Connected versus Unconnected Lookups


CONNECTED LOOKUP UNCONNECTED LOOKUP

Part of the mapping data flow Returns multiple values (by linking output ports to another transformation) Executed for every record passing through the transformation More visible, shows where the lookup values are used Default values are used

Separate from the mapping data flow Returns one value- by checking the Return (R) port option for the output port that provides the return value Only executed when the lookup function is called Less visible, as the lookup is called from an expression within another transformation Default values are ignored

#209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841, 8790998182 , , ,

Update Strategy Transformation


By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Business Challenge
When you design a data warehouse, you need to decide what type of information to store in targets. As part of the target table design, you need to determine whether to maintain all the historic data or just the most recent changes.

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Update Levels
Within a session: When you configure a session, you can instruct the Integration Service to either treat all rows in the same way (for example, treat all rows as inserts), or use instructions coded into the session mapping to flag rows for different database operations. Within a mapping: Within a mapping, you use the Update Strategy transformation to flag rows for insert, delete, update, or reject.

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Flagging Rows Within a Mapping


Operation Constant Numeric Value

Insert Update Delete Reject

DD_INSERT DD_UPDATE DD_DELETE DD_REJECT

0 1 2 3

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Update Strategy Transformation


Used to specify how each individual row will be used to update target tables (insert, update, delete, reject)

Ports All input / output Specify the Update Strategy Expression IIF or DECODE logic determines how to handle the record Example Updating Slowly Changing Dimensions

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Update Strategy Expressions


IIF ( score > 69, DD_INSERT, DD_DELETE ) Expression is evaluated for each row Rows are tagged according to the logic of the expression Appropriate SQL (DML) is submitted to the target database: insert, delete or update DD_REJECT means the row will not have SQL written for it. Target will not see that row Rejected rows may be forwarded through Mapping

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Operations for Individual Target Tables


You can set the following update strategy options: 1. Insert. Select this option to insert a row into a target table. 2. Delete. Select this option to delete a row from a table.
1. 2. 3. Update. You have the following options in this situation: Update as Update. Update each row flagged for update if it exists in the target table. Update as Insert. Insert each row flagged for update.

3. Update else Insert. Update the row if it exists. Otherwise, insert it. 4. Truncate table. Select this option to truncate the target table before loading data.
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Normalizer Transformation
By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Normalizer Transformation
Is Active and connected transformation Is useful to read data from Cobol files Is associated with Cobol file Is useful to convert a single input record into multiple output records Is useful to generate GCID (Generated Column id )

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Example
Year 2012 Year Account Q1 1000 Q2 Q3 Q4 2012 2012 2012 2012 Saving 2000 3000 4000 Account Month Saving Saving Saving Saving 1 2 3 4 Amount 1000 2000 3000 4000

Normalizer

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Sequence Generator Transformation


By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Sequence Generator Transformation


1. Is Passive and connected 2. Use the Sequence Generator to create unique primary key values, replace missing primary keys, or cycle through a sequential range of numbers. 3. It has two ports NEXTVAL,CURRVAL. 4. We can not modify ports

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Creating Keys
1. You can create primary or foreign key values with the Sequence Generator transformation by connecting the NEXTVAL port to a target or downstream transformation. 2. You can use a range of values from 1 to 9,223,372,036,854,775,807 with the smallest interval of 1. 3. Called as surrogate keys
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Replacing Missing Values


1. You Use the Sequence Generator transformation to replace missing keys by using NEXTVAL with the IIF and ISNULL functions. Example: To replace null values in the ORDER_NO column, you create a Sequence Generator transformation with the properties and drag the NEXTVAL port to an Expression transformation. In the Expression transformation, drag the ORDER_NO port into the transformation along with any other necessary ports. Then create an output port, ALL_ORDERS. In ALL_ORDERS, you can then enter the following expression to replace null orders: IIF( ISNULL( ORDER_NO ), NEXTVAL, ORDER_NO )
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Sequence Generator Transformation


Generates unique keys for any port on a row

Ports Two predefined output ports, NEXTVAL and CURRVAL No input ports allowed Usage Generate sequence numbers Shareable across mappings

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Sequence Generator Properties

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Union Transformation
By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Union Transformation
Useful to combine two similar structures (Number of columns and corresponding data types must be same in two source pipelines Types of Groups
1. 2. Input (two or more) Output

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Union Transformation
Is active transformation Is useful to combine multiple similar input pipelines into one output pipe line Supports heterogeneous data sources It works similar to UNION ALL function in data base

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

MFD

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Parameters and Variables


By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Parameters and Variables


By the end of this section you will understand: System variables Mapping parameters and variables Parameter files

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Parameters and Variables


Apply to all transformations within one Mapping Represent declared values Variables can change in value during run-time Parameters remain constant during run-time Provide increased development flexibility Defined in Mapping menu

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Parameters and Variables


Format is $$VariableName or $$ParameterName Can be used in pre and post-SQL Successful completion of the session the integration service stores the variable value in the repository . Variables can be used in generating sequence numbers Variables can be used in incremental loading

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Parameters and Variables


Sample declarations

Set datatype User-defined names Set aggregation type Set optional initial value

Declare Mapping Variables and Parameters in the Designer Mappings/Mapplets menu


# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapping Parameters and Variables

Apply parameters or variables in formula


# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Functions to Set Mapping Variables


SETMAXVARIABLE($$ Variable , value ) Sets the specified variable to the higher of the current value or the specified value SETMINVARIABLE($$ Variable , value ) Sets the specified variable to the lower of of the current value or the specified value SETVARIABLE($$ Variable , value ) Sets the specified variable to the specified value SETCOUNTVARIABLE($$ Variable ) Increases or decreases the specified variable by the number of rows leaving the function(+1 for each inserted row, -1 for each deleted row, no change for updated or rejected rows)
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Parameter Files
You can specify a parameter file for a session in the session editor File extension must be .prm or .txt Parameter file contains folder.session or folder.WF:workflow name.ST:session name and initializes each parameter and variable for that session. For example:
[Production.s_m_MonthlyCalculations] $$State=MA $$Time=10/1/2000 00:00:00 $InputFile1=sales.txt $DBConnection_target=sales $PMSessionLogFile=D:/session logs/firstrun.txt
#

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Parameters & Variables Initialization Priority


1. 2. 3. 4. Parameter file Repository value Declared initial value Default value

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Real Time Examples


if you want use a constant value in many places of mapping then parameter or variable can be used

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Real Time Examples


Variables can be used in Incremental data load A typical example of mapping variable in informatica, Lets say you get a full file from your source and you are supposed to design your mapping to load only incremental data, precisely the new and updated records after the previous run are to be processed processed. Create a mapping variable $$var_date Filter source where create/update date > $$var_date. Assign $$$sessionstarttime value to var_date. If the session is successful, session start time will be saved to the variable. In the next run the transaction generated after the previous run will only be considered.
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Real Time Examples


To pass a value from one session to another. Use workflow variable to assign the value of mapping variable in informatica in any session and you can access it in to next session through workflow variable. This can be achieved by session properties Pre-session variable assignment , and Post session on success variable Post-session assignment .

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Real Time Examples


Easy deployment to production server. your informatica code needs to deployed on production server and as always you will not have access to production server, So to deploy the code properly we need to provide instructions to deployment team to perform cosmetic changes in informatica code. Majority of the cases we are not so lucky that deployment team can perform those tasks without any support. To overcome this problem one can parameterize all the stuffs which are subject to change on a different server.

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

By Ram Reddy

Mapplets

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplets
By the end of this section you will be familiar with: Mapplet Designer Mapplet advantages Mapplet types Mapplet rules Active and Passive Mapplets Mapplet Parameters and Variables

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet Designer

Mapplet Designer Tool Mapplet Output Transformation

Mapplet Input and Output Transformation Icons

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet Advantages
A mapplet is a reusable object that you create in the Mapplet Designer Useful for repetitive tasks / logic Represents a set of transformations Mapplets are reusable Use an instance of a Mapplet in a Mapping Changes to a Mapplet are inherited by all instances Server expands the Mapplet at runtime

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet Types
Mapplets are two types 1. Active 2. Passive A mapplet contains at least one active transformation then it is called as Active Mapplet A mapplet contains all passive transformation then it is called as Passive Mapplet

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

A Mapplet Used in a Mapping

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

The Detail Inside the Mapplet

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Unsupported Transformations
Do not use the following in a mapplet: XML source definitions Normalizer transformation Pre or post stored procedure transformation COBOL Sources Target definitions Other mapplets(nested mapplets)

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet Input Transformation


Use for data sources outside a Mapplet

Passive Transformation Connected Ports Output ports only Usage Only those ports connected from an Input transformation to another transformation will display in the resulting Mapplet
#

Transformation

Connecting the same port to more than one transformation is disallowed Pass to an Expression transformation first

Transformation

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Data Source Outside a Mapplet


Source data is defined OUTSIDE the Mapplet logic

Mapplet Input Transformation

Resulting Mapplet HAS input ports When used in a Mapping, the Mapplet may occur at any point in mid-flow

Mapplet

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Data Source Inside a Mapplet


Source data is defined WITHIN the Mapplet logic

Source Qualifier

No Input transformation is required (or allowed) Use a Source Qualifier instead

Resulting Mapplet has no input ports When used in a Mapping, the Mapplet is the first object in the data flow
#

Mapplet

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet Output Transformation


Use to contain the results of a Mapplet pipeline. Multiple Output transformations are allowed. Passive Transformation Connected Ports Input ports only Usage Only those ports connected to an Output transformation (from another transformation) will display in the resulting Mapplet One (or more) Mapplet Output transformations are required in every Mapplet
# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet with Multiple Output Groups

Can output to multiple instances of the same target table


# 209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Mapplet Parameters and Variables


Same idea as mapping parameters and variables Defined under the Mapplets | Parameters and Variables menu option A parameter or variable defined in a mapplet is not visible in any parent mapping A parameter or variable defined in a mapping is not visible in any child mapplet

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Notes
You can expand the mapplet in the Mapping Designer by selecting it and clicking Mappings > Expand If you use a Sequence Generator transformation, you must use a reusable Sequence Generator transformation. If you use a Stored Procedure transformation, you must configure the Stored Procedure Type to be Normal.

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Reusable Transformations
By Ram Reddy

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Reusable Transformations
By the end of this section you will be familiar with: Reusable transformation rules Different ways to develop reusable transformations Transformation Developer Promoting transformations to reusable

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Reusable Transformations
Define once, reuse many times Reusable Transformations
Can be a copy or a shortcut Edit Ports only in Transformation Developer Can edit Properties in the mapping Instances dynamically inherit changes Caution: changing reusable transformationscan invalidate mappings Note: Source Qualifier transformations cannot be made reusable

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

We have two ways

Ways to develop reusable transformations

1. Using transformation tool 2. Promoting existing transformations to reusable

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Transformation Developer
Make a transformation reusable from the outset, or test it in a mapping first

Reusable transformations

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

Promoting a Transformation to Reusable


Check the Make reusable box (irreversible)

209 Nilagiri Block Adithya Enclave Ameerpet,HYD @8801408841,8790998182 , , ,

You might also like