Executive Summary

In order to deliver business value with a DW, establish a manageable technical footprint early and understanding how to build the program around business needs. DW is not an infrastructure project (e.g. storage), its an exercise in effective data management and business intelligence development. Starting a DW project off on the right foot depends on organizational readiness. Ensure the organization has correct people, processes and technologies to support the DW plan. Effective planning means establishing justifiable business goals and a realistic implementation roadmap that enables early business value. InfoTechs DW survey revealed that organizations that demonstrate business value early, show sustained growth in the usage of their DW. Business value is determined by articulating the business functional and non-functional requirements throughout the DW lifecycle. Requirements must be translated into a blueprint that includes architecture design, application development, data integration, data quality, and data model and interface design.

Info-Tech Research Group

Data warehouse programs improve decision-making by enabling BI processes & applications

Over 40% of organizations with BI dont have a data warehouse. A data warehouse is part of the BI architecture but is not necessary for BI.

Info-Tech Research Article:

To learn more about BI, read Business Intelligence: Core Concepts and Technologies.
Info-Tech Research Group 2

Start building by understanding the bricks & mortar of a data warehouse

Repository Types Definition Use Case
Part of every data warehouse environment. Common metadata is critical to ensure data consistency. Optional. Typically used when batch loading for various source systems happens at different times and requires a holding area for transformations and joins prior to loading for the DW.

Update Frequency
Frequent near real time or longer.

Data Complexity
High

Metadata

Contains reference information about the primary data. It describes what the data means for end users and captures data lineage. Temporary storage area (separate server) dedicated to performing transformations and joins, etc. prior to loading data into the EDW.
Provides a snapshot of enterprise wide operational/transactional data for a given moment in time.

Staging Area

Hourly or longer.

Low

Operational Data Store (ODS)

Optional. Used for high availability and high speed sharing of transaction Frequent information for operational and tactical near real decision making. An ODS can also be used time. as a staging area. Departmental or line of business focused data for tactical and strategic decision making. Can be used on its own or in conjunction with other repository types. Provides a single view of the enterprise across all functional areas and lines of business for tactical and strategic decision making. Can be used on its own or in conjunction with other types. Daily or longer.

Low

Data Mart

Subsets of data specific to a functional area or department, geographical region or time period. Enterprise Data Warehouse a central repository of integrated enterprise wide information at varying levels of detail.

Medium

Daily or longer.

High

EDW Info-Tech Research Group

The DW components that are required will vary depending on the kind of decisions that need to be supported
esign choices must address the kinds of decisions involved.
Decision Type Frequency of Decisions Data Latency Needs Query Frequency Query Complexity Analytical Complexity Scope of Data Involved

Operational

Tactical

Strategic
Weekly monthly decisions Historical

High frequency on daily basis Near real-time

Low frequency on daily basis Hourly to Daily Medium

High

Low

Low Low Subject Specific No History

Medium Medium to High Subject Specific, crossfunctional and historical

High High Enterprise wide and historical

Operational Data Store (ODS) Info-Tech Research Group

Data Mart

EDW 4

This is a rough approximation and does not reflect all use cases.

Set yourself up for success make sure the enterprise is ready for DW
Organizations that have a need for data warehousing may still face challenges when attempting to run a DW project. Understanding your readiness will set you up for success. Readiness is a function of three factors:
Are your people ready? Strong Sr. management and lower level business/ IT support? End-user support? IT team has requisite skills? Do you have strong processes? Data governance is present? Specific business benefits identified? Adequate budget/ timeline? Is the technology ready? Data integration tools and procedures are in place? Appropriate software/ hardware in place (e.g. DBMS, etc.)?
Info-Tech Research Group 5

Enterprises want to be sure to

Assess your readiness to better prepare for your initial DW project

se Info-Techs Data Warehouse Readiness Assessment tool to help determine your preparedness for undertaking a data warehouse project. This tool will help you: Determine the readiness of your staff. Assess business process readiness. Assess technological capabilities. Determine overall organizational readiness. Identify next steps.
Info-Tech Insight:
Not all readiness factors are make-or-break. Understand which factors are critical to success and if/how shortfalls and obstacles can be overcome.
Info-Tech Research Group 6
Data Warehouse Readiness Assessment Tool
This questionnaire is designed to provide a high level assessment of your organizations readiness to embark on a data warehousing project. The questions target key readiness criteria that are predictors of an organizations success with data warehousing. Use the assessment to discover and address gaps in your organizations readiness. Select one of the three options for each of the questions below. Once completed, the readiness tool will give you a recommendation and a measure of your organizations readiness for each category below.

People Assessment
1. Sponsorship: Do you have an executive sponsor formally supporting the project? a. Yes, we have strong sponsors representing both the business and IT that are equal stakeholders in the project. b. We have executive sponsorship for either the business or IT, but not both. c. The business is currently unaware that this project is being considered, this is an IT led initiative. 2. End User Support: What degree of support do you have from target end users of the DW? a. We have business representatives on the project planning team. b. We have spoken to the end users and they have offered verbal support of the program. c. We have not spoken to end users. 3. End User Expectations: What expectations do end users currently have about the DW program? a. End users recognize that the data warehouse will be a work in progress and that it will not address all of their needs as soon as it is deployed. b. End users recognize that there will be some initial problems, but they have unrealistic expectations about how the system will affect their work. c. End users believe that the system will perform perfectly as soon as it is deployed and will radically change (positively or negatively) the way they do their job. a a

Recommendations

No recommendation. You are on track.

No recommendation. You are on track.

Radically incorrect expectations are usually a function of overselling the value of a DW (positive expectations) or a lack of confidence in IT delivery/projects (negative expectations). Reset positive expectations by communicating a realistic delivery schedule for business access to the DW and functionality. To reset negative expectations, achieve quick wins by targeting areas where you can deliver measurable value early in the process. Advertise successes by launching an internal DW marketing program.

Define the business plan - align with enterprise goals and develop a realistic execution strategy and timeline
The output of the planning process will be a business plan that identifies:
Problem to be solved. Alignment with corporate strategy. Opportunities. Risks. Project recommendations. Technology investments. Roles and responsibilities. Alternatives. Implementation roadmap. Approval sought.

Use InfoTechs Data Warehouse Business Plan template to jump start the planning process:

A business plan is a living document that will evolve over time as requirements change.
Info-Tech Research Group 7

Define the scope of the DW: it must have a manageable scope that builds concrete business value
here are many ways a DW can contribute to the business: Providing an enterprise view of business events and entities (e.g. customers & products). By enabling advanced analytics that improve decision making. improving overall network performance by reducing the strain on operational databases and speeding up querying response times.

Business Value

he scope of the project should center around the enterprise information at the level of granularity required by users to support their business processes. This dictates what goes into the DW and how it is structured.

he business and IT must work in tandem to gain a comprehensive understanding of the types of reports and analysis that end users need. Once the business requirements are known, IT requirements and design points can be distilled.
Info-Tech Research Group 8

Enumerate the specific roles the project will require, and staff those roles appropriately
As with other systems projects, you will need a project sponsor, project manager, business analyst, database administrator, developer, and testing and quality assurance coordinator. You will need a DW analyst who can architect and deliver your DW one with experience in this specialized field. In certain cases, this role is split even further into a data architect, who designs the architecture, and a technical leader who delivers it. You will also need someone with experience in data modeling and design, preferably someone who is comfortable with one or more modeling tools. You will need a source data analyst (usually separate from the business analyst) who has deep expertise in the organizations source systems and processes (this usually a spot best filled by an internal rather than external candidate). A DW middleware specialist should be familiar with the best ways of moving and transforming your data, and with ETL and data quality tools. Finally, you should have someone on the project that is conversant with the data access tools you will be rolling out (meaning that they can install and support them as well as develop queries and reports. See the Appendix for sample job descriptions used for hiring for these roles. Info-Tech Research Group 9 Information Systems Services

Role
Business Analyst

Team Function
Identifies and defines DW purpose and target user groups. Ensures that the DW fulfills the enterprise's strategic objectives.

Data Architect

Defines data collection, transformation, distribution, and loading. Defines the data models that are the foundation of the DW. Tests DW tools. Assesses the need for expansion of the DW.

End-User Support Allows user reporting and access. Trains end users and provides support. Leadership and Management Sponsors the DW and makes it a priority within the organization. Develops project plans. Ensures that the DW remains aligned with business needs.

DW requirements gathering is unique plan to take an iterative approach to collecting them

Ask users what they need from DW. This is challenging given user ignorance of their own needs, and alone is not enough to clarify requirements. Employ a project manager or BA that possesses at least a fundamental understanding of the relevant business and practices, and can ask the right questions to clarify unknown terms and concepts. Gather requirements iteratively throughout the design process that said, keep your eyes out for scope creep. Ask the following questions: What functions are you trying to perform in the business? What type of data do you require to do these functions? Where do you get this data today? Do you relate or integrate this data with any other data?

DW requirements gathering is unique: Users typically dont know their wants and needs as DW is very new to most users. DW design requires much broader, and more difficult to define business concepts (compare Business Process Reengineering [BPR]). This makes requirements gathering more important and builds a strong argument for using a prototype approach to design.

Info-Tech Research

For a more detailed guide to requirements gathering, see the Solution Set, Overcome the Barriers to Good Requirements Management.
Info-Tech Research Group

10

Data mart focused architecture approaches deliver value early, but can be expensive over time
Independent data marts focus on delivering subject oriented data to key user groups (e.g. sales and marketing) for organizations that do not need an enterprise wide view.

Pros
Targets specific business unit needs. Easy to construct. Quick time to results.

Cons Difficult to obtain

enterprise view. Redundant data costs. High long term ETL costs. High long term maintenance and support costs.
11

Info-Tech Insight:
If you are just starting out, consider beginning with single scalable independent mart and grow it into an EDW.
Info-Tech Research Group

Appliance vendors offer superior speed, DBMS are the most cost effective, DW software (virtual appliances) strikes a balance
DW Appliances: consists of both hardware (e.g. servers) and custom software that are optimized (e.g. massively parallel processing) for DW specific tasks (e.g. query). Virtual Appliances: are software based solutions for DW that can be deployed on a variety of hardware configurations for commodity hardware. Database Management Systems (DBMS): is commodity software for database management (e.g. Oracle, Microsoft SQL, etc.) that are used for DW, but are typically not optimized for DW performance.

Sample DW Appliance Vendors

Sample DW Software Vendors

Sample Database Management Systems (DBMS)

Info-Tech Research Group

12

Data volume & transformation complexity should guide your choice of transform/load procedures
Evaluate the transformation requirements before deciding on a loading approach. Use ETL if the transformation rules required for preparing the data are complex and cannot be performed using database stored procedures. ETL is also ideal for populating data marts that exist on the same physical infrastructure as the DW, as there will be no need to move data off the DW platform and, therefore, no network impact will occur. Leverage ELT when loading small data sets with relatively simple transformation logic.
Info-Tech Research Group

>10 source systems and >1TB of

transactional data Use ETL.

<10 source systems and >5 TB of

transactional data Use ELT.

ETL ETL
10
# of Source Systems

Direct Load
5

ELT ELT

1 TB

Data Volume

5 TB
13

Appendix I
Business Intelligence Introduction: Business intelligence: Core Concepts and Technologies Data Warehousing Notes Data Warehousing: ETL vs. ELT Early TCO Analysis Cuts Data Warehousing Costs Information Roadmap Optimizes Enterprise Data Use Information Architecture Roadmap Data Warehousing: Staffing Dictates Success Job Descriptions: Data Warehouse Architect Enterprise Data Architect Database Developer Database Administrator Database Analyst Business Requirements Analyst Project Manager Business Intelligence Specialist Data/Data Mining Specialist Application Integration Specialist User Interface Analyst IT Security Manager EDI Specialist For more see InfoTechs Job Descriptions Section
Info-Tech Research Group

Requirements Notes: Requirements Gathering: What Every IT Manager Needs to Know Requirements Models: The What, When, and Why

Custom vs. COTS: Tailor Requirements Gathering for a

Elicitation Techniques Bring Requirements to the Surfa

Choose the Right Elicitation Techniques to Improve Req The Politics of Collecting Business Requirements Business Requirements Template Functional Specifications Template Use Case Template Requirements Tracking Template Quality of Service Requirements Template

14

Appendix II Survey Demographics

Industry

Info-Tech Research Group

15