IMPORTANCE PERFORMANCE ANALYSIS: THE NEED TO BRIDGE SOLITUDES FOR ITS EFFECTIVE USE Tzung-Cheng Huan, National Chia-yi

University Jay Beaman, Auctor Consulting Associates Introduction Importance-Performance Analysis, IPA, was formulated by Martilla (1977). It was dubbed action grid analysis, AGA, by Blake, Shrader, and James (1978). It is not just an analysis methodology. It is implicitly a theory of behavior. It was introduced as a way of understanding clients needs and desires so as to make good management decisions about how to respond to them. By finding out about what people think about importance and performance on manipulable attributes of a product, it is reasonable to think that one can come to some reasonable conclusions about modifying performance on attributes to e.g. increase profit or client satisfaction effectively. A bibliography (about 50 articles) covering IPA (a) use in leisure research and (b) special methodological innovations is available from Beaman (1999). Applying IPA following the example of Martilla and James (1977) is simple. In the leisure research context participants in a survey could be asked a series of questions about specific facilities and services (product attributes) being important to them and also asked for a rating of performance on each attribute. In a broader context, data have been collected using a fixed allocation of points, regarding service delivery staffs or managers opinions about what clients or each other think about importance of and performance on attributes and related matters (Dawson & Buerger, 1992; Richardson, 1987; Steele & Fletcher, 1991; Havitz, Twynam, & DeLorenzo, 1991). There are also other IPA extensions dealing with completion and observance (Dolinsky & Caputo 1991; Hammitt, Bixler & Noe 1996). The rationale for IPA is easily understood. Assume you manage a shopping and dining area in a leisure facility. Therefore, you are responsible for attributes that influence the experiences of your clients and are charged with improving performance e.g. by getting tourists to stop, eat and shop. Figure 1 is what Shrader and James (1978) would call an action grid for your facility. Assume you carry out a survey thus obtaining importance and performance information from you clients. This allows you to compute average importance and performance ratings for attributes and grand means across attributes. The grand means can establish an alternative axis system referred to as crosshairs. Means and crosshairs are plotted in Figure 1. There one sees that Dining has a relatively high mean for importance while having a relatively low one for performance (Figure 1). It is in the action grid quadrant labeled Concentrate here. (lower right) Concentrate here relates to there being high importance and poor performance. Shopping is of high importance and performance so it is in the high-high quadrant identified by Keep up the good work. Other quadrants are Possible overkill and Low Priority.

From what has presented it would appear that by a very simple survey and preparing a graph based on averages, you can identify that you should be putting effort into improving the dining experience. What is more you appear to easily prepare a graphic that is readily understood by those to whom you present your case for action. Problem and methodology As the introduction of IPA shows, it appears to be a reasonable analysis approach that is easily used. Yielding an easily understood graphic to employ in presenting research results facilitates using it to convince a non-technical audience that a recommendation is based on sound scientific research. However, if certain assumptions are not met, IPA results can be deceptive. While maintaining that IPA is a useful tool, this presentation deals with getting valid results. In fact, there have been various debates in the literature about such matters as (a) how to set crosshairs (heavy axes through about (3,3) at the general mean- e.g. Mills 1984; Burns 1988), (b) what to do about missing rating values (e.g. Hollenhorst, S., Olson, D., & Fortney, R. 1992) and (c) using medians when rating responses have a badly skewed distribution (even Martilla and James 1977). Based on addressing real analysis issues in using IPA data for camping in Terra Nova National Park, Vaske, Beaman and Grenier (1995; see mention in Mills 1984) have dealt with matters relating to respondents to an IPA survey (a) coming from segments with different action grids (population hetrogenity) and (b) exhibiting ipsative variability in their responses (respondent tendency to e.g. rate high or low). In this research we restrict our attention to dealing with some practical concerns relating to (a) getting grids for homogeneous segments (including issues of within party

heterogeneity); (b) the need to consider performance expectations;(c) making appropriate use of observations with missing and deviant observations (including recognizing standard versus exceptional conditions of participation); (d) using deviant survey responses as triggers for supplemental data collection; (e) setting the crosshairs; and (f) interpreting and using action grid and other survey results appropriately. The methodology pursued is based on deductive logic. Logical analysis is applied to scenarios for IPA data structure that can be expected in application of the technique to leisure services and facilities. For example, it is reasonable to assume that for many facilities there are multiple user segments (e.g. local repeat recreational campers and firsttime out of area natural experience campers; parties with and without children) that (a) will want some different facility/service attributes and (b) will differ on the importance or performance of attributes of common interest. There is also the matter of segments having different performance expectations (some expect flush toilets or nice paved parking for a festival). It is also reasonable to assume that some experiences will be influenced by poor weather or other conditions resulting in a performance rating that e.g. reflects an exogenous force over which management has little or no control. Logical analysis of valid use of IPA data with heterogeneity of users, skewed ratings because of exogenous factors and various performance expectations for attributes, allows one to reach conclusions about what can be achieved by IPA and how to proceed so as to achieve valid results. We carry out logical analysis and present and interpret the results of it in practical terms. Results Results of analysis are guidance on data collection and analysis. It is found that a key matter is viewing the definition of an IPA research project as a formulation of a behavioral model for the population to be studied. A priori logic or e.g. segment definitions based on previous studies must be used to define data collection so segments with differing action grids are recognized and included or excluded (e.g. because too small to sample) from a survey. Furthermore, how/whether particular attributes apply to particular segments must be recognized. For attributes for segments, it is necessary to establish normal responses e.g. as those given in average participation conditions. In so far as poor to bad conditions with respect to an attribute produce skewing of responses, it is found that information for the deviant (skewed low) observations should (a) drive collection of information confirming the reason for skewing and its consequences (e.g. for future use) and (b) deviant ratings should be excluded in forming action grids. Results are presented on placing crosshairs, using medians, dealing with missing observations of ratings etc. Discussion Discussion focuses on a variety of matters. The goal is to clarify the practical importance of results. For example, the discussion stresses the importance of recognizing segments and standard conditions that apply to them. Conventional action grid analysis (IPA) has not considered skewing of distributions as either a consequence of heterogeneity or of the expected occurrence of exceptional conditions over which management has little or no control. Managers and other personnel, in many cases, will recognize conditions in which performance ratings will be down because most users will

reflect the e.g. exogenous condition in their performance rating. Discussion of such matters leads to covering more technical matters such as the use of medians to deal with skewed data being inappropriate. It also leads to the practical matter of recognizing deviant observations and getting clear information from respondents about why they occur (rather than trying to draw inferences without such information). One focus of discussion is that IPA is not a tool for final decisions. It is a tool for problem conceptualization and initial analysis. It is pointed out that IPA does not, and will not, yield quantitative information that should be the basis for final decisions on expenditures of resources. Rather, it serves to narrow options thereby identifying which studies of scale, scope, benefits, etc. are needed to determine if resources should be expended on particular actions. Applications This part of the paper enumerates applications and potential applications. Actual applications that were made in Parks Canada are mentioned. Comment on potential applications focuses on appropriate versus inappropriate contexts for applying IPA. References Blake, B. F.; Schrader, L. F.; James, W. L.(1978). New Tools for Marketing Research: The Action Grid. Feedstuff, 50(19):38-39, 1978. Beaman (1999). The Result of a Search for Performance-Importance (AGA) Literature for 1977 to 1997 That Is Published. Unpublished manuscript: word document. 272 Abstracts ~ CCLR-11, 2005 Burns, T. (1988). Using Importance Performance To Measure The Opinions Of National Park Concessionaires, Pp. 167-74 in Proceedings of The 19th Conference on Tourism Research: Expanding Boundaries. Montreal: Travel And Tourism Research Association. Dawson, C. P., & Buerger, R. B. (1992). Importance-Performance Analysis: Congruity/Disparity Between Charter Boat Captains And Customers, pp. 41-45 in Proceedings Of The 1992 Northeastern Recreation Research Symposium., USDA, Forest Service Gen. Tec. Rep. NE-176, 171p.. Dolinsky, A. L., & Caputo, R. K. (1991). Adding A Competitive Dimension To Importance-Performance Analysis: An Application To Traditional Health Care Systems. Health Marketing Quarterly, 8(3/4):61-79. Fletcher, J. E.; Kaiser, R. A.; Groger, S. (1992). An Assessment of The Importance And Performance of Park Impact Fees in Funding Park and Recreation Infrastructure. Journal Of Park And Recreation Administration, 10(3):75-87. Hammitt, W.; Bixler, R. D.; Noe, F. P.(1996). Going Beyond Importance-Performance Analysis To Analyze The Observance-Influence Of Park Impacts. Journal of Park and Recreation Administration, 14(1):45-62. Havitz, M. E.; Twynam, G. D.; DeLorenzo, J. M. (1991). Importance-Performance Analysis As a Staff Evaluation Tool. Journal Of Park And Recreation Administration, 9(1):43-54.

Hollenhorst, S., Olson, D., & Fortney, R. (1992). Uses Of Importance-Performance Analysis To Evaluate State Park Cabins: The Case Of The West Virginia State Park System. Journal Of Parks And Recreation Administration, 10(1):1-11. Martilla, J., & James, J. (1977). Importance-Performance Analysis. Journal Of Marketing, 41(1):77-79. Mills, A. S. (1984). Importance/Performance Analysis of Facilities and Services at Two Arkansas Projects. Recnotes: Information Exchange Bulletin (CORPS), 84(1):57. OLeary, J.; Adams, M. B.; Parker (1981). Importance-Performance Analysis: An Aid to problem Identification in Urban River Recreation Development, pp. 200-212, in Proceedings of Unified River Basin Management-Stage II, ( Allee, Dworsky and North Eds.). Steele, R. J.; Fletcher, J. E. (1991). Use of Paired Management Action Grids for Ease in Depicting Differences between Users and Managers Perceptions of Problems. pp. 122-126 in Proceedings of the 1991 Northeastern Recreation Research Symposium. USDA, Forest Service Gen. Tec. Rep. NE-160, 234 p.. Vaske, J.;Beaman, J.; Stanley, R.;Grenier, M. 1995. "P-I And Segmentation: Where Do We Go From Here?". Journal Or Tourism And Marketing Research, 5(3):225-40.

ABSTRACTS of Papers Presented at the Eleventh Canadian Congress on Leisure Research May 17 20, 2005 Hosted by Department of Recreation and Tourism Management Malaspina University-College Nanaimo, B.C. Abstracts compiled and edited by Tom Delamere, Carleigh Randall, David Robinson CCLR-11 Programme Committee Tom Delamere Dan McDonald Carleigh Randall Rick Rollins and David Robinson

Copyright 2005 Canadian Association for Leisure Studies ISBN 1-896886-01-9