QA Backend Database and ETL Testing Approach

W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

............................................................................................................................................1 Introduction.........................................................................................................................3 1 Backend testing phases ....................................................................................................3 2 Testing methodology .......................................................................................................4 3 Data testing types..............................................................................................................4 3.1 ETL and stored procedure unit testing.......................................................................4 3.2 QA Backend Testing.................................................................................................4 3.2.1 Verify adequacy of data mappings ....................................................................5 3.2.2 Tables, columns, column types, defaults, and rules ...........................................5 3.2.3 Keys and indexes................................................................................................6 3.2.4 Stored procedure tests ........................................................................................6 3.3 Technical shakedown tests.........................................................................................7 3.4 Operational readiness testing (ORT).........................................................................7 4 Backend Testing Entry, Exit Criteria................................................................................7 4.1 QA Entry Criteria.......................................................................................................7 4.2 QA Exit Criteria.........................................................................................................8

W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

Introduction
The characteristics of database, data warehouse applications pose a challenge for every project: How can we be sure our data is accurate and reliable when there are enormous amounts of it and when it comes from multiple systems and schemas with different data structures? The testing approach described here combines practical experience and highlights the planed approach. Challenges of Database Backend Testing

Data selection from multiple source systems, multiple related tables and analysis that follows Volume, quality and the complexity of the data Inconsistent and redundant data in schemas Inconsistent and inaccurate reports displayed to users Non-availability of history / audit data used to create data entities

Database schemas are often different, business entities change to portray different functional meaning, and format and usage of data captured in a new system can be totally different Data field lengths might change and pose data integrity issues Databases can be successfully accessed and updated Data has been extracted, cleansed or scrubbed, and loaded correctly Data conversion / transformations convert data as expected to each target

General Objectives of Backend Testing

Calculations or manipulations associated with data are performing as expected and yield accurate results All expected data has been loaded Test transformations: data is transformed correctly according to business rules and design specs. ETLs and stored procedures correctly reject, default, update, correct, ignore and reports invalid data. Business rules for data have been implementation Interfaces between the application and database can operate correctly

Backend testing phases

There are several phases in backend testing. The first step is to acquire design specifications for the database schemas. The second step is to develop test specification design. A third step is to implement the tests with SQL code, Quest TOAD and other tools. The test specification design should contain information concerning components to be tested (individual tables, PL/SQL, database objects), new requirements for regression testing (earlier data bugs), integration testing (several pieces of the database put together).
W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

Testing methodology

Use of requirements traceability to enable full test coverage of business data requirements In depth review of backend test cases Identification and manipulation of test data to ensure full test coverage

Provision of appropriate test tools to speed the process of test execution & evaluation Regression testing after data issues are fixed

Data testing types

The following are types of testing performed for data warehousing projects. 1. ETL, stored procedure unit testing 2. Data model, data mapping verifications 3. QA backend testing 4. Technical shakedown testing 5. Operation readiness testing

3.1

ETL and stored procedure unit testing

The objective of Unit testing (done by ETL or PL/SQL developer) involves testing of business transformation rules implemented during the ETL process, error conditions, mapping fields at all levels of load from stagging to ODS to data marts. Testing involves the following (and much more)

Check correctness of field mappings specifications to data loaded in tables. Check for the duplication of values generated using sequence (key) generator.

Check for the correctness of surrogate keys, which uniquely identifies rows in database. Check for data type constraints of the fields present in staging and later levels of data loading Check for the population of status and error messages into target table. Check for string columns that have been left and right trimmed.

Check visually in data warehouse designer tool if every transformation has a meaningful and testable description.

3.2

QA Backend Testing
The objective of Integration Testing is to ensure that workflows are executed as scheduled with correct dependency. Integration testing includes the following and much more:
W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

1. o o o

Check for the execution of workflows at the following stages Source to Staging Staging to ODS ODS to Data Mart

2. Check target tables are populated with correct number of records and that all fields correctly populated 3. Performance of the workflow is recorded and analysis is performed performance results. 4. Verify the dependencies among workflows between source to staging, staging to ODS and ODS to data mart have been properly defined. 5. Check for Error log messages in appropriate file and write defect reports if needed.
3.2.1

Verify adequacy of data mappings

Every data field that is going to be migrated from the source system to the target system must be defined and examined to ensure compliance with field lengths, data types, domain values permitted, system rules, integrity checks and any other possible issues. The detailed data map is critical to understanding where information is going as well as whether there are any known or avoidable obstacles in the way of successfully arriving there. The data map must detail an in-depth cross-referencing of all mutual fields across the source system and the target system. Data maps should include: Names of applicable to and from fields Lengths and data types of these fields Method of verifications Any logic involved in mapping such as string truncations or validations against business rules

Detailed descriptions of all transformations.

3.2.2

Tables, columns, column types, defaults, and rules

Verify the following to identify differences between specification and actual tables. This is a sample checklist of verifications. Many more tests are likely.

Table names Column names for each table

Column types for each table (int, varchar, char, text, datetime. specially the number of characters for char and varchar) Whether a column allows NULL or not Default value definitions Whether a default is bound to correct table columns Rule definitions are enforced Whether each rule is bound to correct table columns
W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

Whether access privileges are granted to correct groups

3.2.3

Keys and indexes

Verify the following items and verify with design specification

Primary key for each table (every table should have a primary key) Foreign keys Column data types between a foreign key column and a column in other tables Indices, clustered or non-clustered; unique or not unique

3.2.4

Stored procedure tests

Verify the following comparing each with design specifications

Whether a stored procedure is installed correctly in the database Stored procedure name Parameter names, parameter types and the number of parameters

Stored procedure outputs Verify

When output is zero (zero rows affected) When some records are extracted and others are not Output contains records What each stored procedure is supposed to do What each stored procedure is not supposed to do

Write simple queries to see whether each stored procedure populates correct data

Stored procedure parameters Verify

Check parameters that are required Call stored procedures with valid and invalid data Call procedures with boundary data Make each parameter invalid and run each procedure

Stored procedure return values

Whether a stored procedure returns values and if values are correct When a failure occurs, nonzero must be returned.

Stored procedure error messages:

Make stored procedure fail and cause every error message to occur at least once Find out any exception that doesnt have a predefined error message
W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

Others:

Whether a stored procedure grants correct access privilege to a group/user See if a stored procedure hits any trigger error, index error, and rule error Look into a procedure code and make sure major branches are test covered.

3.3

Technical shakedown tests

Due to the complexity in integrating various source systems and tools, there are expected to be several teething problems with the environments. A technical shakedown test will be conducted prior to commencing backend testing to ensure the following points are proven:

Hardware is in place and has been configured correctly (including Informatica architecture, source system connectivity and business Objects). All ETL software has been migrated to the testing environments correctly. All required connectivity between systems are in place and all tester permissions are defined and working. End-to-end transactions (both online and batch transactions) have been executed and do not fall over.

3.4

Operational readiness testing (ORT)

This is the final phase of testing which focuses on verifying the deployment of software and the operational readiness of the application. The main areas of testing in this phase include: Deployment readiness tests: 1. Tests the deployment of the database solution 2. Tests overall technical deployment checklist and timeframes 3. Tests the security aspects of the system including user authentication and authorization, and user-access levels.

4 4.1

Backend Testing Entry, Exit Criteria QA Entry Criteria

ETL unit tests are complete to include DB load verification in DEV

ETL code, DB issues, changes requests, and open defects from development have been documented Data track release notes have been delivered and no critical ETL or DB issues are open. Approved data model and mapping document (LLDs) available minimum of two weeks before QA testing begins (required to prepare test cases) All QA environment schema are set up according to the test plan for current build/iteration ETL code migrated from development environment in QA environment
W. Yaddow, Data Quality Analyst, ETL Tester January, 2010

QA has been provided all access information and rights to DB and ETL code in QA region one week before QA start date. Successful completion ETL sanity testing in QA env by dev team.

4.2

QA Exit Criteria

No critical defects unfixed; No more than 3 high or medium severity defects are open (negotiable with PM & business) 80% or more of build functionality can be tested in the UI functionality might fail because of JAVA / report code. Database platform performance is such that test team can productively work to schedule Fewer than 15% of build fixes during QA failed

W. Yaddow, Data Quality Analyst, ETL Tester January, 2010