Guide to the Engineering Management Body of Knowledge

Domain 1: Market Research, Assessment and Forecasting

Key Words and Concepts

1.1 Market Analysis

The ultimate success of any commercial organization depends upon its ability to determine the its customers’ needs. Even before product and service offerings are designed, a seller must learn about its customers and competitors. Sellers obtain this information through a variety of market research methods, environmental scanning efforts and competitive intelligence activities. In order to achieve meaningful results, engineering managers need to possess knowledge of the basic tools and techniques required to gather and analyze this customer-driven data.

1.1.1 Knowledge of Basic Tools and Techniques

Becoming masterful in market analysis is a priority for the savvy engineering manager. To compete today, product viability must be assessed before product development begins.

Specifically, market analysis is the identification and assessment of the target market, the competition and the demand for a product or service. Market analysis is a dynamic process and there is no one single or most effective methodology to use. Conducting market research is both an art and a science and generally involves a team of associates who bring together different mindsets, resources, skills and experiences to the research and analysis process.

Market analysis is conducted before committing to any business strategy and may be a relatively simple or highly sophisticated process depending upon the tools and techniques used to gather, analyze and interpret data and report findings.

The American Management Association defines marketing research as: the function which links the consumer, customer and public to the marketer through information — information used to identify and define marketing opportunities and problems; generate, refine, and evaluate marketing actions; monitor marketing performance; and improve understanding of marketing as a process.¹

The steps comprising market analysis can be fraught with the potential for error (Figure 1-1). Decision makers and researchers must work closely together when agreeing to the objectives of the research and developing the questions to be answered by the study.

Figure 1- 1 Steps in the Marketing Research Process and Potential Sources of Error ²

1.1.2 Performing Market Analysis, Interpreting the Results and Making Appropriate Recommendations

The following steps comprise the market analysis process.

.1 Describe the Objective(s)

Begin the market analysis by specifically stating desired results. Consider potential obstacles and marketplace opportunities.

Questions to ask:

Is the goal too narrow for our focus (e.g., a new product or service) or is it too broad (e.g., opens up new markets for an existing product or refines an existing product for new markets)?

What has been done in the recent past that was effective? By your company? By competitors?

What is your relationship with your target market? What improvements can be made?

What differentiates your company (and its product or service) from the competition?

Who are your customers?

What do your customers need and/or want?

.2 Determine Information Needs and Data Sources

Once the objectives have been described, begin an informal survey using information that is already available. Collect company data on existing and emerging competitors. Contact knowledgeable individuals within your organization, informally poll key customers, collect information and insights from people informed about your industry and product, consult trade organizations and publications, and explore data sources on the Internet. Frequently the information gathered at this early stage will lead to a redefinition of the objectives.

.3 Research Design — Focused, Formal, In-Depth Data Collection

After completing your informal survey and clearly defining your research objectives, determine how to best collect the data necessary to complete your market analysis. Your formal research design is likely to be more detailed and extensive than the initial informal research activities and should be very targeted to your market analysis goal(s). Take care in data collection because using a faulty data collection method leads to inaccurate results.

Experts agree that maintaining a balance of qualitative and quantitative research data is vital to the success of the research project. Figure 1-2 provides examples of both qualitative and quantitative data collection methods:

Figure 1- 2 Examples of Quantitative and Qualitative Primary Research Methods

Quantitative data may involve statistical analysis of the number of individuals who have purchased an item in the past two years or the number of respondents to a survey who indicated that they were interested in a particular product motivation. For quantitative data to be useful, it must include a representative sample of the target market, and the researcher must be careful to ensure that data collected matches the objective and what is being measured. Quantitative data can be difficult or costly to collect but is needed to provide a quasi-objective and measurable means of understanding the target market.

Qualitative research is inherently subjective and is designed to reveal customer attitudes, beliefs and preferences. Although the data yield supplied by qualitative research methods may appear initially to be cumbersome and overwhelming, the information to be garnered is essential as a means of predicting customer behavior. Qualitative research consists primarily of the use of open-ended questions directed at customers and consumers in group or individual interviews.

Your research plan should also include the use of secondary research sources. Secondary sources include:

Census data

Government records

Company records

Various trade and industry sources

Case studies

Archival research

Secondary research data has already been collected by other sources and is often compiled for other purposes. Nevertheless, there is often useful information that can be derived from these external sources. Associations are a tremendous resource of industry-specific data. There are innumerable sources of information available through the Internet including online journals, databases and research organizations. Considerations such as cost, time, objectivity and ability will dictate whether data collection is best done by internal staff or outside consultants.

Note: Generally, the collection of primary data is expensive, while the gathering of secondary data may be of little or no cost at all.

.4 Select Sample and Size

Take a representative sampling of the target market to be sure to collect data from an appropriate range and number of sources. This will ensure that the data is reasonably representative of the potential customer base. Survey across cultural lines, socio-economic strata, industry sectors, or national boundaries. Be certain also to select a sample size that is small enough to be manageable, yet large enough to yield useful information from both over-represented and underrepresented populations.

Survey design is the crux of the research and is often best left to professional marketers and statisticians; however, if engineers are to be trained on this, they must have a knowledge of sampling methods, sampling instruments and how to determine sample size (which is a balance between acceptable confidence range and cost). Refer to a statistics textbooks for further instruction on survey design.

.5 Collect Data — Ensure Validity

Stay focused on the objective of the survey and be certain to measure what you intend to be measuring. Be on the alert for skewed data that appears to be a mismatch with the bulk of the information gathered. Is one interviewer biasing focus group outcomes? Are certain geographical areas yielding higher returns of mail-in surveys than others? These variances can give important insights into customer behavior or can be sources of invalid outcomes due to faulty data collection methods.

Double-check all data received from outside sources. Never assume that a government report, external research firm, or pollster has conducted valid research. Check the facts. Examine sample size and characteristics. Search for unexplained variances from past research outcomes. Compare the results from several sources before drawing conclusions.

.6 Tabulate, Analyze, and Report Data

Once data has been collected, assess its significance. Do this individually or as part of group. Avoid fallacies of logic, misinterpretations, and bias by making sure that conclusions drawn are supported by the data. To do this, use a information gathering technique such as SWOT or PEST (Figure 1-3) can be used to examine the data and expand the breadth of the risks being considered.³

Figure 1- 3 SWOT and PEST Models of Data Analysis ⁴

The process of market analysis concludes with a report detailing research findings and recommendations (see Figure 1-4). The structure of the report is similar to other engineering reports, although for market analysis reports, it is key to distinguish the recommendations in a section separate from the findings. In this way, key recommendations will be readily accessible to anyone reading the report.

Figure 1- 4 Structure, Content and Focus of a Market Analysis Report

.7 Market Analysis Report Content

It helps to clearly delineate content in the body or the report in sequential sections.

Purpose: Describe the purpose of the research and research objectives. Explain the most important feature of the research. Discuss the background of the project.

Conclusions: List all conclusions reached as a result of doing the analysis.

Supporting data: Support the conclusions by referencing research findings.

Data sources: List research contributors and describe contributions made by each. List works by other researchers used in the study. Describe ways in which research proves or disproves other researchers’ work.

Results: Describe what you expected to find before beginning research. How did the project change over time? How did results differ from expectations? Describe the results. How were results checked? Represent the results using text, tables, figures, charts and graphs.

Consequences: Describe the consequences of the research. What does it mean for the subject? How will it affect future research on this subject?

Suggested audience: Describe the ideal audience for the report. Who would be most impacted by the research? Who would best understand the consequences of the research? Who needs to see the research to make marketing and design decisions? Is confidentiality an important consideration when releasing findings?

The Focus box in Figure 1-4 refers to those who will be impacted by the decision to send, or not send, a product to market. Discuss the product under consideration by listing its features and benefits.

Describe the potential customer for the product. Is the customer business-to-consumer (B2C) and/or business-to-business (B2B)?

B2C: Considerations include demographic, geographic (including population shifts) and psychographic information.

B2B: Considerations include the type of industry, size of each company, company culture, company’s buying behavior, centralized or decentralized purchasing, and geographic location.

Describe customer needs as indicated by the research. Outline why customers buy this product. What benefits will customers derive from the product? Identify why customers buy from the company. Consider issues of price, features and benefits, distribution channels, service, brand, and image. Review how long customers researched have been purchasing from the company.

Describe the competition. Explore how the competition has changed. Is it stronger or weaker than in the past? Are you now dealing with different competitors than previously? Uncover how the competition differs (pricing, distribution channels, product features, promotion, size, geographical location(s) and number of locations) from your company. Describe what the competition is doing that is similar. Detail how the competition impacts the company product(s). Determine which competitors fall into the categories of leaders, challengers, and followers. Describe each competitor’s market share.

Assess channel partners. Describe how the product is distributed. Determine what channel partners do for the company. Describe benefits channel partners offer to customers. Determine whether better channel partners are available to market the product under consideration.⁵

1.1.3 Communicating Results and Recommendations to Non-Marketing Personnel

There are a number of media you can use to communicate market analysis results to non-marketing personnel. Your company website, newsletters, information sheets, technical reports, a slideshow or video, and live presentations are the most popular methods to publicize findings. A communication plan is essential to ensure that the right information gets to the right people at the right time.

A communication plan identifies who will be interested in the research findings and how much you intend to communicate with them. Begin by identifying the target audience and assessing the most effective way to reach them. Base decisions on which communication medium will be most attention-getting, clear, concise, expedient, and useful to the target audience. Once a medium has been selected, use the following guidelines for presenting information.

Simplify the language so that readers or listeners without backgrounds in research or marketing can readily understand the report or presentation contents.

Create simple materials that readers can easily interpret.

Use inviting graphics in materials intended for non-marketing audiences.

Make certain the report or presentation is targeted to listener or reader needs. Design engineers will be impacted by the research differently than supply chain experts. Customize each presentation to the focus, concerns, tasks, and functions of audience members.

1.2 Best Practices and Lessons Learned

The results of marketing analysis are only one indication of whether a potential product will find its niche in the marketplace. The question now becomes, Can we produce and market this product using traditional thinking and performance methods? Often the answer is no. Then the question becomes, What do we need to learn as an organization to be more competitive in the market? The answer often lies in making a closer study of competing organizations.

1.2.1 Knowledge of Benchmarking Techniques and Sources of Information to Identify Best Practices in Other Companies

Benchmarking is the process of improving performance by comparing the cost, cycle time, productivity, or quality of a specific process or method to another that is widely considered to be an industry standard or best practices. This approach to process improvement is so effective because it is based on actual practices, not theory.

Studying best practices provides the greatest opportunity for gaining a strategic, operational and financial advantage. While benchmarking readily integrates with strategic initiatives such as continuous improvement, reengineering and total quality management (TQM), it is a discrete process that delivers value to the organization on its own.

The benchmarking process (Figure 1-5 below) varies greatly across industries and nations. Some anchor the effort to performance standards, while others begin the process by defining goals. For the purpose of market research, anchoring to processes will provide the engineering manager a comparative method for improving market share, distribution, sales, and budgetary considerations with an eye toward best practices in each area.

Figure 1- 5 The Benchmarking Process ⁶

.1 Select a Process to Benchmark

Selection of a process depends upon what the organization needs to learn in order to better measure and improve internal practices. If marketing analysis shows that the chief competitor is able to bring a product to market faster than your company because of inefficiencies in your product development process, then product modification processes become your focus.

.2 Determine the Project’s Scope

Clearly define the scope of your project. The product development process, for example, encompasses a number of smaller sub-processes. To determine the scope of your project, you should first determine which of those sub-processes negatively impact development time. If, after detailed study, you find that development is slowed at the point where products are modified for foreign market requirements, then this modification process is the scope of your project.

.3 Choose a Relevant Measure

Keeping in mind that searching out best practice options is the goal, investigate competitors to isolate the measures they use to keep their product modification process within acceptable timelines. Then apply these measures to your company’s existing process.

.4 Study Performance-Boosting Processes

Pinpoint the ways in which your competitors excel when faced with challenges. For example, how does your competitor compensate when changing government regulations delay product modifications? Is there a protocol they use to deal with this sort of unexpected setback?

.5 Judge Appropriateness and Apply Practices

Given that your competitor’s organization differs from yours in myriad ways, not all practices relevant to their success can be efficiently transplanted into your operation. Judge the relevance of the competitor’s methodologies. It may be necessary to make adjustments to their strategies prior to implementing them at your company.

.6 Identify Cultural Issues and Other Implementation Factors

The corporate culture and/or geographic location of the company you are benchmarking against is likely different from yours. As a result, its management practices, division of labor, size, and accepted protocols may differ greatly. Be sure to examine whether the benchmarked practices will be accepted within your firm, and if they can be implemented as intended given your organizational structure, size and culture.

.7 Plan and Implement Changes

After refining the best practices discovered during the benchmarking process and ensuring a good fit with your organization, implement the new processes and measure their effectiveness in meeting your organization’s goals.

1.2.2 Applying Best Practices and Lessons Learned to Improve Functioning

Adaptation of best ideas is where organizations gain their competitive advantage, and benchmarking is where many of those ideas originate. It is not sufficient to simply access and copy a competitor’s strategies. To make a business case for recommended changes, the return on investment (ROI) for business restructuring must justify the change.

When analyzing the findings provided by a benchmarking study, the primary question to ask is, How can what we learned be applied in our organization to build our capacity, capitalize on our strengths, exploit our opportunities, and overcome our obstacles? Benchmarking is more likely to generate paybacks when it is driven by strategic objectives. Look to the organization’s strategic plan to find out where the company intends to go. Look to the findings of the benchmarking project to inspire thoughts about how to get there.⁷

1.3 Business Research and Forecasting Tools and Techniques

Strategic planning must take into account a number of variables, including economic and environmental conditions. Recessionary trends, for example, may impact sales figures if strategic planners have failed to project for such eventualities. Making accurate predictions regarding future market conditions is done through a process known as business forecasting.

1.3.1 Knowledge of Forecasting Models, Methods, and Techniques

Formal business forecasting is done using statistical models and econometrics which literally means measuring the economy. The most reliable and common statistical methods used in business today are regression analysis, smoothing, decomposition, and Box-Jenkins (see also Section 1.5.1).

Forecasting, especially economic forecasting, consists of the scientific use of statistics combined with selection of what to measure and what formula or model to use, and then deciding which explanatory variables will provide the most accurate forecast when the statistics are plugged into the selected formula.

Although business forecasting can be done by an individual, a cross-departmental alliance of engineering managers using a structured discussion methodology such as a Delphi Technique (see Section 1.4.1) will likely produce a more comprehensive study of economic trends, as well as a more accurate prediction of how those trends will impact your organization’s future profitability, productivity and performance.

A wide variety of forecasting software is available, including spreadsheet applications and statistical packages. Enterprise resource planning (ERP) software also contains forecasting modules. When selecting the software, it is important to consider what it is designed to measure. Some packages focus on sales and marketing forecasts, while others are specific to supply chain management.

Statistical forecasting predicts the value of one dependent variable by measuring its relationships with two or more other independent variables. For example, sales (the dependent variable) can be estimated based on age and gender (independent variables). A different forecast is derived if sales (the dependent variable) are estimated based on amount spent on advertising and the size of sales staff (independent variables).

1.3.2 Applying Forecasting Techniques, Interpreting Results, and Making Appropriate Recommendations

In engineering circles, business decisions are made using net present value (NPV) and rate of return on investment (ROI) in order to assess the accuracy of business forecasts. The data generated by forecasting depends upon both historical data and statistical manipulation. Because of this, when using cost/parameter relationships, remember: (1) the uncertainty inherent in the extrapolation of statistics, and (2) whether the indicated relationship is logically sound and reasonable.

Forecasting of any type only creates predictions. It is much like listening to a weather report. Sometimes the report is exactly on target, and other times the sun shines when the forecaster has predicted rain. In business forecasting, statistical results may reveal that, based on a company’s history of success with one product line, and forecasted economic indicators, expanding or enhancing a product line is a wise decision. Two weeks later, a catastrophic event may slow or even halt the predicted economic growth.

Many companies choose to run a number of independent projections before making a decision to move ahead. Often, both in-house experts and external consultants are employed to examine identical historical data using different statistical methods. Alternatively, a sole forecaster may manipulate a number of differing data sets to produce comparative results. When interpreting findings and formulating recommendations based on a forecast, a number of reasonable options may exist. Engineering managers will want to ask:

If we move ahead using the predictive findings will the NPV and ROI hold in the face of faulty predictions or unforeseen circumstances?

What risks are attached to making an investment decision now based on these projections?

What are the best case and worst case scenarios that could play out as the result of accepting and acting upon forecasted findings?

Does the historical data used in formulating the forecast have ongoing relevance to future situations?

Was all relevant data available and included in making the forecast?

Before making recommendations based on forecasted information, engineering managers are wise to conduct an ROI analysis using the same suppositions used in the forecast. To do this, the manager should assemble a group of five to seven people with differing backgrounds and perspectives to consider the issues. During the meeting, adhere to the following guidelines:

State the issue to be addressed.

Ask each person to complete an ROI worksheet independently (see Figure 1-7).

Tabulate the results by adding the points assigned to each item under consideration and record them on the ROI Summary.

Total the values in each row to find which strategies might best optimize return on investment.

Weigh outcomes during a discussion with the assembled group.

Figure 1- 6 Steps in Forecasting

Figure 1- 7 ROI Worksheet and Worksheet Summary ⁸

1.3.3 Communicating Results and Recommendations

Using the same reporting format as described in Figure 1-4, Structure, Content and Focus of a Market Analysis Report, document the results of the forecast and decide how to best represent your recommendations. Oral presentations, slideshows, newsletters, or thorough use of the company’s Intranet are all viable options for communicating outcomes and recommendations. Include information about the ROI summary and the rationale used to reach each recommendation.

1.4 Risk Analysis

The analysis of risk is viewed as a field in itself, and the demand for an orderly and formal treatment of risk is great. The process of identifying and reducing potential losses related to bringing a new product or service to market is, in itself, risky. Issues of time, cost, scope, and quality are considered when planning to bring a product to market. Timing is critical when introducing a new, refined, or enhanced product.

What could be worse than introducing a product a week after a leading competitor premiers a similar item? It could be worse if the company’s product quality was inferior to the competitor’s. Shifting government regulation could narrow the market available to the product. A design flaw could render the product unsafe for use by the customer. The product may be lacking the features required to generate sales. Customer whims could shift, leaving your product without a consumer base.

Risk identification is seldom a straightforward process. Regardless of how complete and thorough market research is, there are both visible and invisible risks to assess throughout the marketing project. Alternatively, risks can be positive in nature. The projections of product sales may have underestimated product demand, and now customer demand is higher than predicted.

1.4.1 Knowledge of Basic Tools and Techniques of Risk Analysis

Six steps comprise the risk identification process. All steps involve the use of a cross-departmental team often referred to as the risk analysis team.

.1 Risk Identification

Review documentation: Assemble the team to review all documentation generated by the marketing analysis effort. Marketing research, benchmarking data, and forecasting projections must be reviewed on a regular basis to check for accuracy, completeness, and ongoing applicability as the marketing process progresses.

Conduct a SWOT: Involve the documentation review team in assessing the strengths, weaknesses, opportunities, and threats (SWOT) inherent to the marketing plan. Differing perspectives can unearth risks that marketing personnel may have missed.

Brainstorming: Through the use of a structured, facilitated brainstorming session, identify potential risks and categorize them according to their probability of occurrence. High, moderate, and low categories can be used to prioritize risks so that the engineering managers know what to address first.

Interviewing: The team interviews other members of the organization who have worked on similar past projects to gain insights about how risks were identified and addressed.

Root cause identification: This is an inquiry into the essential causes of the risks that have surfaced after conducting the first four steps in this process. A simple method of root cause identification is to ask the question, Why? seven times, testing each response to the question with an additional, Why? After seven rounds, teams have usually uncovered the root cause of the risk.

Delphi Technique: At this point in the risk identification process, the team will likely find some disagreement among themselves about what is viewed as risky and how to prioritize risks. The Delphi is an anonymous strategy used for building consensus around risk. A facilitator uses a questionnaire to solicit ideas about important risk concerns. Responses are then summarized and redistributed to team members for further comment. This process continues until a consensus has been reached. The Delphi helps reduce bias and prevents one person from unduly influencing the outcome.⁹

The work of the team can be captured on a checklist for use as the project progresses. Segment the checklist into high-, moderate-, and medium-risk categories; use it as a monitoring instrument. While checklists are quick and convenient, they are not exhaustive. Take care to explore items that are not on the checklist, and to promote ongoing environmental scanning for risks that become evident after the list has been developed.

.2 Risk Register

The primary outputs from Risk Identification are the initial entries into the risk register, a key component of the risk management plan. The risk register ultimately contains the outcomes of all risk management processes as they are conducted. The register contains the following items:

Identified risks

Potential responses to the identified risks

Root causes of each risk

Documentation of risk categories¹⁰

For further information about risk registers, see Section 1.4.3.1.

1.4.2 Performing a Risk Analysis, Interpreting the Results, and Making Appropriate