ASSESSING THE IMPACT OF ELECTRONIC COMMERCE ON BUSINESS PERFORMANCE: A SIMULATION EXPERIMENT

George M. Giaglis, Ray J. Paul

Department of Information Systems and Computing Brunel University, UK

Georgios I. Doukidis
Department of Informatics Athens University of Economics and Business, Greece

ABSTRACT

Before committing resources in developing Electronic Commerce (E-Comm) applications, organisations need to assess the bottom-line business benefits that E-Comm can deliver and compare them to the costs of the associated investments. However, the intangible nature of most E-Comm benefits may render the development of a business case very difficult in practice. In this paper we argue that computer-based simulation can assist companies in gaining insight on the real benefits and dangers associated with the adoption of E-Comm. Drawing on the results of a real-life case study, we discuss the potential of simulation modelling to support assessment of E-Comm business value.

INTRODUCTION

Despite the great publicity, even hyperbole, surrounding E-Comm, existing practical applications have not always been able to deliver in practice the business benefits they promise in theory (Nath et al 1998). This

has resulted in lower adoption rates than initially expected. Historically, the growth of E-Comm has been more an evolution rather than the much-anticipated revolution (DosSantos and Peffers 1998).

One of the main reasons that may explain the reluctance of organisations to adopt E-Comm on a great scale may be the significant amount of organisational change required (Costello and Tuchen 1998). Kalakota and Whinston (1996) point out that the broad goals of reengineering and electronic commerce are remarkably similar: reduced costs, lower product cycle times, faster customer response, and improved service quality . Such a radical change will necessarily pose a fundamental question to managers and decision-makers: can the benefits achieved by employing E-Comm outweigh the costs needed for setting up and maintaining the necessary infrastructure and applications?

In this paper we present a real-life case study where discrete-event simulation models of business processes were developed to assist two companies to identify the problems faced in their co-operation, formulate appropriate solutions based on E-Comm applications, and realise the expected impacts of these solutions on key business performance indicators. Drawing on the study results, we discuss the potential of simulation to support assessment of E-Comm business impact on organisations.

THE BUSINESS VALUE OF ELECTRONIC COMMERCE

Investments in E-Comm are equally as difficult to evaluate as any other type of investments in Information Technology (Smithson and Hirschheim 1998). In order to evaluate an E-Comm investment, the costs and benefits associated with it should be measured and compared. Costs are relatively easier to measure, at least the direct ones, usually during the feasibility study and the development of specific project proposals. However, in comparative terms, it is significantly more difficult to obtain hard evidence of the expected benefits (Weill and Olson 1989).

The business benefits of E-Comm can be classified into four categories (Brown 1994):

a)

Hard benefits, which can be realised as a direct result of the introduction of E-Comm. An example of hard benefit is the reduction in data-entry staff made possible by the introduction of an electronic ordering system. Such benefits are easily measured but do not usually contribute significantly to the overall benefit an organisation might expect from E-Comm.

b) Intangible benefits, which can be attributed directly to E-Comm but cannot be easily expressed in quantitative terms. Benefits of this type arise, for example, with the introduction of an EDI system to allow a company s customers to place their orders electronically. A second-order effect of such a system might be locking-in customers by making it more difficult to them to switch to another nonEDI capable supplier. This is a clear business benefit, but how can the company predict the real effect of the EDI system on customer loyalty and its bottom line implications on revenue generation? c) Indirect benefits, which are potentially easy to measure but cannot be wholly attributable to E-Comm and can only be realised as a result of further investments, enabled by the introduction of the E-Comm infrastructure. For example, the implementation of the EDI system mentioned above would provide the infrastructure on which further E-Comm applications can be subsequently built at reduced costs. Although this is a potential benefit made possible by the initial EDI system, it cannot be realised (and measured) unless further applications are also successfully introduced. d) Strategic benefits, which refer to positive impacts that are realised in the long run and usually come as a result of the synergistic interaction among a number of contributing factors. They are the outcome of, for example, a new business strategy or a better market positioning of the organisation, which can only be partially attributed to a given E-Comm application. Such benefits are notoriously difficult to quantify in advance due to their very nature and to the risk associated with their realisation.

The majority of benefits that an organisation might expect from an investment of E-Comm will be of an intangible, indirect, or strategic nature. Such benefits cannot be easily quantified and measured to provide the basis of a formal cost-benefit analysis of the investments needed to implement E-Comm.

The problem of measurement can be overcome by employing some quantitative technique that will allow for studying the business processes affected by an E-Comm investment, rather than narrowly concentrating

on the investment itself (Hochstrasser 1993). Discrete-event Business Process Simulation (BPS) offers a theoretically attractive mechanism for modelling and studying complex phenomena in quantitative terms. Simulation of business processes constitutes one of the most widely used applications of operational research as it allows for understanding the essence of business systems, identifying opportunities for change, and evaluating the impact of proposed changes on key performance indicators (Law and Kelton 1991, Warren et al 1995, Hansen 1994). Perhaps more importantly, it allows decision-makers to experiment with alternative business configurations without the need for disrupting the actual system operations.

Simulation models have been used in various practical business modelling applications, including E-Comm modelling. For example, Yarden (1997) discusses a simple simulation meta-model structure for modelling Electronic Commerce payment transactions over the Internet. Along the same lines, simulation models have been used (Mylonopoulos et al 1995, Giaglis 1996) to assess the expected benefits of interorganisational changes made possible by the use of EDI at an industry-wide scale. Such simulation models work by incorporating the effects of IT applications (such as E-Comm) in models that depict the business operations that are expected to be affected by the introduction of a new system. Other practical examples of business process modelling applications can be found in Pruett and Vasudev (1990), Lee and Elcan (1996), Ninios et al (1995), and Kim and Kim (1997).

A CASE STUDY

We will present a case study where simulation was used to evaluate an inter-organisational investment in a new EDI system by two collaborating organisations in Greece. One company is the national branch of a major multinational pharmaceuticals organisation (we will henceforth refer to this company as PHARMA S.A. ), while the other is a small-sized regional distributor of Pharma s products (we will refer to this company as TRANSPHARM Ltd. ). The project aimed at assessing the potential of redesigning the trading communications scheme between the two companies and evaluating the possibility of introducing E-Comm applications for supporting the redesigned processes.

The case study presented herein was carried out within the Medical Division of PHARMA. This division trades hospital consumables and medical devices to hospitals, health care organisations, networks of physicians, and the government. On the other hand, TRANSPHARM is a small company (employing at the time of the case study nine people, of which five are the company s drivers) that acts as PHARMA s exclusive distributor of Medical Division products for the whole of Northern Greece.

Due to the special nature of the health care business and the subsequent urgency of most customer demands, TRANSPHARM has to operate within strict deadlines regarding deliveries. Indeed, each order has to be fulfilled with 24 hours if the products are to be delivered within Thessaloniki (the city where the company is based) or within 48 hours for the rest of Northern Greece.

However, since the beginning of the collaboration between the two companies in 1993, it has been noted by management that the aforementioned targets are virtually never met in practice. During preliminary discussions the two companies agreed that the problems seemed to be arising from inefficiencies in the ordering process as well as due to the inability of TRANSPHARM to maintain an optimal level of product inventory to support order fulfilment. Moreover, the extant communication and information exchange scheme between the two companies was deemed to be cumbersome and inflexible.

In this context, the study presented here was conducted. The objectives of the study were: (a) To examine in detail the existing processes of customer order fulfilment, invoicing, and warehouse management, where co-operation between PHARMA and TRANSPHARM is taking place; (b) To propose alternative processes by which the existing problems could be alleviated; and (c) To evaluate the potential of introducing appropriate E-Comm applications to facilitate the communication between the two companies.

The Business Processes

The overall process to be considered in this case study is the Order Fulfilment Process (OFP). This can be thought of as the collection of activities that occur from the time a customer places an order until the time

this order is fulfilled (i.e. the customer has received the products and the corresponding invoice). Figure 1 shows the parties involved in the OFP as well as the existing communication between the participants (both physical and informational exchanges).

======================================== Insert Figure 1 here ========================================

Problems of the existing situation

The major problems identified by both PHARMA and TRANSPHARM management during preliminary discussions on the performance of the aforementioned business processes can be summarised as follows: a) Excessive Order Lead Times. Customer orders are usually fulfilled much slower than the stated targets (24 to 48 hours). b) Poor internal and external customer service. Long lead-times result in customer dissatisfaction that is expressed through a growing number of complaints about delivery delays. c) Excessive Invoice Lead Times. The time it takes for the invoice to reach a customer is unnecessarily long, resulting in poor cash-to-cash cycle for PHARMA. d) Out of balance inventory and excessive stock levels. In order to ensure that orders could be fulfilled without any need for backordering, PHARMA warehouse managers have followed a policy of overstocking TRANSPHARM warehouse. However, this has caused considerable costs to TRANSPHARM due to the need to maintain large inventories. Furthermore, it is questionable whether this policy has managed to achieve much more than camouflaging the internal process inefficiencies. e) Information Sharing. Because of incompatibilities between PHARMA and TRANSPHARM information infrastructure, the companies have been relying in paper forms for exchange of information. Apart from duplication of effort and slow processing times, this has resulted in building a culture of limited information sharing between the two partners.

Poor, incompatible, sometimes non-existent IT infrastructure has been identified as one major contributor to the problems faced by PHARMA and TRANSPHARM. It was therefore decided to examine the potential of adopting electronic messaging applications to facilitate the exchange of information between the two companies. The companies management wanted to introduce EDI to support exchange of data between the firms. It was also believed that such a system could be the starting point for building a wider Inter-Organisational Information System (IOS) that will strengthen the links between the companies and effectively make them work as a single virtual enterprise. PHARMA also wanted to use this project as a pilot for introducing similar systems to foster closer relationships with their other distributors across Greece.

However, such a change would necessarily involve significant expenditure on behalf of both firms. The main problem facing the management was to evaluate the magnitude of benefits that could be achieved by the proposed change in order to assess whether it would surpass the associated investment costs. It was decided to pursue a more in-depth study of the processes and to employ Business Process Modelling and Simulation.

Simulation Model Development

A four-phase approach was followed for modelling and analysing the processes under consideration. The approach is presented in Figure 2.

======================================== Insert Figure 2 here ========================================

During the first phase, interviews with key process participants (management and employees) of both PHARMA and TRANSPHARM were conducted in order to capture the process essence and decompose the

OFP into its component activities. The knowledge elicited by the interviews was used to define the boundaries of the process and the models to be developed.

The simulation model depicting the existing situation (AS-IS model) was developed using an off-the-shelf, packaged software platform (Process Charter, by Scitor Corporation). The model was then run for a (simulated) 6-month period. The results from the simulation runs are summarised in Table 1 (only the key performance indicators are included due to space constraints). It is clear that the existing processes are far from producing results within the stated management targets. Orders are fulfilled in around 60 hours (2 days), while backorders need an average of nine days to reach the customers. It is worth noting that 95% of the order lead-time and 70% of the backorder lead-time are actually waiting times. In order words, orders and backorders spend more of their time waiting for something to happen rather than being processed. The same also holds true for invoices, where the lead-time is over 11 days with over the half being idle waiting time.

======================================== Insert Table 1 here ========================================

The objectives of the third phase were to analyse the AS-IS model, develop alternative process structures to alleviate the problems identified, and transform these structures into process models to identify the impact of each solution on key performance indicators. The model analysis brought to light a number of reasons that contribute to the inefficiencies identified. The major problems identified by AS-IS model analysis are shown in Table 2.

======================================== Insert Table 2 here ========================================

Based on the results of the AS-IS modelling phase, alternative process configurations were developed and discussed with PHARMA and TRANSPHARM management for acceptance and feasibility. Four alternative scenarios were developed and modelled with Process Charter. The scenarios were then simulated and results were compared with the AS-IS model to evaluate the impact of changes on key performance indicators. Table 3 shows the scenarios modelled and their relation to the problems of the existing processes.

======================================== Insert Table 3 here ========================================

The results from the scenario runs are summarised in Figure 3. The simulations produced a number of quantitative data but, due to space constraints, only the results regarding the three main key performance indicators (average waiting times for orders, backorders, and invoices) are presented here.

======================================== Insert Figure 3 here ========================================

The purpose of the final phase was to evaluate the results of the experimentation and develop a set of options that would be acceptable by the decision-makers. Simulation provided valuable insight into the ability of the proposed solutions to alleviate the problems faced by the two companies. In terms of the key performance indicators we can note the following: a) Orders: The results were somewhat surprising. Contrary to what was expected, the adoption of EDI did not result in the time savings for order fulfilment initially envisaged by PHARMA and TRANSPHARM. The adoption of EDI alone (scenario A) resulted in only a 17.2% reduction of average order fulfilment time. The resulted 48 hours on average needed to fulfil an order under Scenario A only marginally meet the set targets. However, simulation made it possible to realise that

other (non technology-supported) changes could provide a solution to the inefficiencies of the process. Thus, the modification of the order authorisation policy and employing an extra employee at the TRANSPHARM warehouse, when combined with the EDI system (scenario D), resulted in a 48.3% saving of order fulfilment time, resulting in an acceptable average of 30 hours to fulfil an order. b) Backorders: EDI here provided an efficient mechanism for overcoming the obstacles raised by poor co-ordination and information exchange between PHARMA and TRANSPHARM. The adoption of EDI (scenario A) reduced the average backorder fulfilment time from 215 to 122 hours (43.3%). Although this time is not within the specified targets, it certainly provides a huge relief to PHARMA and TRANSPHARM management. c) Invoices: Again EDI proved, as expected, to be instrumental in substantially reducing the invoice delivery time. Scenario A resulted in a 74.7% reduction in average invoicing time (from 269 to 68 hours). Like backordering, the remaining scenarios B to D only marginally affect invoicing time, as expected.

Further to simulation analysis, the four scenarios were further scrutinised in order to develop a detailed understanding of implementation challenges and transform hypotheses into detailed implementation plans. The requirements of each option regarding technology, people and skills were assessed and a formal costbenefit analysis was conducted (on the basis of the simulation results) in order to evaluate the proposed investments. Based on the results of the analyses, detailed recommendations for change and implementation plans were proposed.

CONCLUDING REMARKS

Discrete-event simulation proved to be valuable mechanisms for realising the real business value of EDI. Both PHARMA and TRANSPHARM management were able to see for themselves and assess the costs and benefits associated with various proposed options. This hands-on experience helped them to overcome their doubts about adopting EDI and build their confidence in the technology, without bearing the risk and cost of developing prototype applications and disrupting the operation of their businesses.

It was further appreciated how simulation proved that the adoption of EDI alone would only marginally improve the performance of the main process (order fulfilment time), contrary to what was initially expected. Management was able to identify, propose, and experiment with other options that would complement the EDI investment to achieve the desired results. If simulation had not been employed and the EDI application was adopted in the hope that order fulfilment times would be substantially decreased, it is very likely that the management of both companies would be disappointed. Thus, they could develop a negative perception of the value of E-Comm in general and hence be unwilling to invest further in similar applications. Thus, the case study provided empirical evidence to support the argument that the application of simulation can provide an efficient mechanism for allowing organisations to assess the real business value of EDI. By assisting managers to overcome their hesitance about E-Comm and understand the benefits it can bring to their businesses, it is believed that critical user masses can be more easily built, thereby resulting in better adoption rates for E-Comm in general.

Many authors argue that one of the major problems that contributes to a high failure rate in many real-life business change projects is a lack of tools for evaluating the effects of designed solutions before implementation (Paolucci et al 1997, Tumay 1995). This is be even more true in the case of the introduction of innovative business and technological applications, like E-Comm, for which there is relatively less experience available in the market. Mistakes brought by business change can only be recognised once the redesigned processes are implemented, when it is usually difficult and costly to correct wrong decisions. Although the evaluation of alternative solutions is usually difficult, it is essential in order to reduce some of the risks associated with business change projects. Business Process Simulation can be of great assistance in facilitating increased understanding and organisational visibility of changes.

Despite the potential of simulation to address the problem of E-Comm investment appraisal, the road to successfully applying simulation in business process design is not trouble-free. In order to be effective vehicles for investment evaluation, Business Process Simulation models should explicitly incorporate the expected effects of E-Comm adoption on business performance. In the case study presented in this paper,

these effects were confined to lead-time reductions expected in the execution of business processes. However, the effects that E-Comm can have on business operations can be very diverse, ranging from hard, operational benefits (e.g. error reduction, less work duplication) to indirect and strategic ones (e.g. lock-in customers, better market positioning). Further research can be directed towards investigating and articulating the exact ways by which E-Comm can influence business performance (positively or negatively). These ways can then be the matter of detailed analysis in Business Process Simulation models, depending on the nature, scope, and objectives of individual E-Comm evaluation projects.

ACKNOWLEDGEMENTS

Earlier versions of this case study have been published in the Proceedings of the 1998 Americas Conference on Information Systems, Baltimore, MA (August 1998) and the 11th International Bled Electronic Commerce Conference, Bled, Slovenia (June 1998).

REFERENCES

Brown, A. (1994) Appraising Intangible Benefits from Information Technology Investment. In the Proceedings of the First European Conference on IT Investment Evaluation, Henley, England, September, pp. 187-199. Costello, G.I. and Tuchen, J.H. (1998) A Comparative Study of Business to Consumer Electronic Commerce within the Australian Insurance Sector, Journal of Information Technology, 13, 3, pp. 153-167. DosSantos, B.L. and Peffers, K. (1998) Competitor and Vendor Influence on the Adoption of Innovative Applications in Electronic Commerce, Information and Management, 34, 3, pp. 175-184. Giaglis, G.M. Modelling Electronic Data Interchange Through Simulation: An Industry-Wide Perspective. In the Proceedings of the 8th European Simulation Symposium, Genoa, Italy, October 1996. Hansen, G.A. (1994) Automating Business Process Reengineering: Breaking the TQM Barrier, PrenticeHall, Englewood Cliffs, New Jersey.

Hochstrasser, B. (1993) Quality Engineering: A New Framework Applied to Justifying and Prioritising IT Investments, European Journal of Information Systems, 2, 3, pp. 211-223. Kalakota, R. and Whinston, A. (1996) Frontiers of Electronic Commerce, Addison-Wesley, Reading, MA. Kim, H.W. and Kim, Y.G. (1997) Dynamic Process Modelling for BPR: A Computerised Simulation Approach, Information and Management, 32, 1, pp. 1-13. Law, A.M. and Kelton, D.W. (1991) Simulation Modelling and Analysis, 2nd edition, McGraw-Hill, New York. Lee, Y. and Elcan, A. (1996) Simulation Modeling for Process Reengineering in the Telecommunications Industry, Interfaces, 26, 3, pp. 1-9. Mylonopoulos, N.A., Doukidis, G.I. and Giaglis, G.M. Information Systems Investments Evaluation through Simulation: The Case of EDI, In the Proceedings of the 8th International Conference on EDI and Interorganisational Systems, Bled, Slovenia, June 1995, pp.12-26. Nath, R., Akmanligil, M., Hjelm, K., Sakaguchi, T. and Schultz, M. (1998) Electronic Commerce and the Internet: Issues, Problems, and Perspectives, International Journal of Information Management, 18, 2, pp. 91-101. Ninios, P., Vlahos, K. and Bunn, D.W. (1995) Industry Simulation: System Modelling With an Object Oriented / DEVS Technology, European Journal of Operational Research, 81, pp. 521-534. Paolucci E., Bonci, F. & Russi, V. Redesigning Organisations Through Business Process Re-engineering and Object-orientation. In Proceedings of the 5th European Conference on Information Systems, Cork, Ireland, June 1997, pp. 587-601. Pruett, J.M. and Vasudev, V.K. (1990) MOSES: Manufacturing Organisation Simulation and Evaluation System, Simulation, 54, 1, pp. 37-45. Smithson, S. and Hirschheim, R. (1998) Analysing Information Systems Evaluation: Another Look at an Old Problem, European Journal of Information Systems, 7, 3, pp. 158-174. Tumay, K. Business Process Simulation. In Alexopoulos, C., Kang, K., Lilegdon, W.R. and Goldsman, D. (Eds.), Proceedings of the 1995 Winter Simulation Conference, Arlington, VA, December 1995, pp. 55-60.

Waren, J.R., Crosslin, R.L. and MacArthur, P.J. (1995) Simulation Modelling for BPR: Steps to Effective Decision Support, Information Systems Management, 12, 4, pp. 32-42. Weill, P. and Olson, M.H. (1989) Managing Investment in Information Technology: Mini Case Examples and Implications, MIS Quarterly, pp. 3-17. Yarden, S. Evaluating the Performances of Electronic Commerce Systems. In Andradottir, S., Healy, K.J., Withers, D.H. and Nelson, B.L. (Eds.), Proceedings of the 1997 Winter Simulation Conference, Atlanta, GA, December 1997, pp. 1053-1056.

Orders Invoices

Customers THESSALONIKI & NORTHERN GREECE

Deliveries Orders Orders

PHARMA Salesmen
Orders

TRANSPHARM Office and Warehouse THESSALONIKI

Order Confirmation

PHARMA Sales and Marketing THESSALONIKI

Backorders

Stock Replenishment

Invoices Copies of Despatch Notes

Out of Scope

PHARMA Warehouse and Plant ATHENS

Documents, Information

Out of Scope

PHARMA HQ ATHENS
Products Out of Scope

Figure 1. Information and material exchanged in the OFP

Phase 1 Data Collection

Demand Patterns Activity Schedules Activity Lead-Times Activity Priorities Entity Life-Cycles Resource Schedules

Phase2 AS-IS Modelling

Phase 3 Option Development & Experimentation

Feasible Options (Brainstorming)

Phase 4 Decision Making & Recommendations

New Business Processes Static Process Modelling (Flowcharting) TO-BE Model Development IT Infrastructure

Organisational Impact (costs & benefits) Dynamic Process Modelling (Process Simulation) Experimentation (Test against criteria) Implementation Schedule

Feasibility

Impact

Figure 2. The Four-Phase Modelling Approach

Average Time Order Backorder Invoice 58h 39min 215h 23min 269h 38min

Average Wait 55h 31min (95%) 150h 42min (70%) 153h 41min (57%)

Max Time 142h 06min 334h 50min 602h 50min

Table 1. Key Performance Indicators (AS-IS Model)

Problem I

Description The existing policies for forwarding backorders and Despatch Notes to PHARMA are cumbersome and inflexible. If the information systems of the two companies were able to communicate electronically there would be no obstacle to forwarding these documents as soon as they are created (possibly in the form of an EDI message), thereby avoiding two major process bottlenecks.

II

There is a bottleneck of orders in TRANSPHARM warehouse. The reason behind this may be that the existing one employee in the TRANSPHARM warehouse may be unable to cope with the existing demand.

III

The order authorisation policy results in unnecessary delays as orders have to be exchanged between PHARMA and TRANSPHARM in a time consuming fashion that results in little or no added value to the process.

Table 2. Major Problems of the Existing Process

Scenario

Description

Problems Tackled

EDI is used to facilitate exchange of backorders, despatch notes, and invoices between PHARMA and TRANSPHARM.

Same as Scenario A, plus TRANSPHARM employs two employees at the warehouse.

I+II

Same as Scenario A, plus TRANSPHARM is also empowered to authorise the orders they receive.

I+III

Same as Scenario A, plus TRANSPHARM employs two employees at the warehouse and is also empowered to authorise the orders they receive.

I+II+III

Table 3. TO-BE Scenarios

ORDERS Average Time 60 50 37 40 30 20 10 0 AS-IS Sc A Sc B Sc C Sc D 58 48 41 30

BACKORDERS Average Time 250 200 150 100 50 0 AS-IS Sc A Sc B Sc C Sc D 122 120 122 119 215

INVOICES Average Time 300 250 200 150 100 50 0 AS-IS Sc A Sc B Sc C Sc D 68 59 62 53 269

Figure 3. Key Performance Indicators in AS-IS and TO-BE models