Approaches to the Ex-arte Evaluation of Information Systems

Walter, Sascha

Approaches to the Ex-arte Evaluation of Information Systems

A critical Review

von Sascha Walter (Autor:in)

BWL - Controlling

Zusammenfassung

Inhaltsangabe:Summary:
This paper critically reviews approaches for the evaluation of investments in information systems prior to their implementation. First, the ground for the review is prepared by examining characteristics of evaluation, information systems and value. A classification of 54 evaluation approaches identified in English and German literature is then presented. Examples of each class are reviewed and their advantages and drawbacks are discussed. Their use in evaluation practice is analysed through the examination of empirical studies and directions for future research are given. A complete overview of state-of-the-art evaluation approaches is given at the end of the paper.

Inhaltsverzeichnis:Table of Contents:
1.INTRODUCTION1
2.THEMATIC BACKGROUND3
2.1Information Systems3
2.2Evaluation8
2.3Value13
3.CRITICAL REVIEW OF EVALUATION APPROACHES17
3.1Antecedents to the Review17
3.2Effect-Assessing Approaches20
3.3Effect-Locating Approaches33
3.4Discussion of Usability of Approaches41
4.EVALUATION IN PRACTICE51
4.1Review of Empirical Studies51
4.2Interpretation of Empirical Studies55
4.3Discussion and Future Research57
5.CONCLUSIONS60
APPENDIX61
Overview of Evaluation Approaches61
Literature Search Strategy64
REFERENCES65

Leseprobe

Inhaltsverzeichnis

ID 7027

Sascha Walter

Approaches to the Ex-arte

Evaluation of Information

Systems

A critical Review

Diplomarbeit

an der Universität Bielefeld

Fachbereich Wirtschaftswissenschaften

Mai 2003 Abgabe

ID 7027

Walter, Sascha: Approaches to the Ex-arte Evaluation of Information Systems - A critical

Review

Hamburg: Diplomica GmbH, 2003

Zugl.: Fachhochschule Südwestfalen, Universität, Diplomarbeit, 2003

Dieses Werk ist urheberrechtlich geschützt. Die dadurch begründeten Rechte,

insbesondere die der Übersetzung, des Nachdrucks, des Vortrags, der Entnahme von

Abbildungen und Tabellen, der Funksendung, der Mikroverfilmung oder der

Vervielfältigung auf anderen Wegen und der Speicherung in Datenverarbeitungsanlagen,

bleiben, auch bei nur auszugsweiser Verwertung, vorbehalten. Eine Vervielfältigung

dieses Werkes oder von Teilen dieses Werkes ist auch im Einzelfall nur in den Grenzen

der gesetzlichen Bestimmungen des Urheberrechtsgesetzes der Bundesrepublik

Deutschland in der jeweils geltenden Fassung zulässig. Sie ist grundsätzlich

vergütungspflichtig. Zuwiderhandlungen unterliegen den Strafbestimmungen des

Urheberrechtes.

Die Wiedergabe von Gebrauchsnamen, Handelsnamen, Warenbezeichnungen usw. in

diesem Werk berechtigt auch ohne besondere Kennzeichnung nicht zu der Annahme,

dass solche Namen im Sinne der Warenzeichen- und Markenschutz-Gesetzgebung als frei

zu betrachten wären und daher von jedermann benutzt werden dürften.

Die Informationen in diesem Werk wurden mit Sorgfalt erarbeitet. Dennoch können

Fehler nicht vollständig ausgeschlossen werden, und die Diplomarbeiten Agentur, die

Autoren oder Übersetzer übernehmen keine juristische Verantwortung oder irgendeine

Haftung für evtl. verbliebene fehlerhafte Angaben und deren Folgen.

Diplomica GmbH

http://www.diplom.de, Hamburg 2003

Printed in Germany

TABLE OF CONTENTS

1. INTRODUCTION ... 1

2. THEMATIC BACKGROUND... 3

2.1. Information Systems... 3

2.2. Evaluation... 8

2.3. Value ... 13

3. CRITICAL REVIEW OF EVALUATION APPROACHES ... 17

3.1. Antecedents to the Review ... 17

3.2. Effect-Assessing Approaches... 20

3.3. Effect-Locating Approaches... 33

3.4. Discussion of Usability of Approaches ... 41

4. EVALUATION IN PRACTICE... 51

4.1. Review of Empirical Studies... 51

4.2. Interpretation of Empirical Studies ... 55

4.3. Discussion and Future Research... 57

5. CONCLUSIONS... 60

APPENDIX ... 61

A. Overview of Evaluation Approaches... 61

B. Literature Search Strategy ... 64

REFERENCES ... 65

LIST OF FIGURES

Fig. 1: Model of IS life cycle ... 5

Fig. 2: Classification of evaluation approaches ... 18

Fig. 3: Generic types of indicators ... 24

Fig. 4: Basic concept of the BSC ... 26

Fig. 5: Example of a BSC modified for IS evaluation ... 27

Fig. 6: IE factors for computing the project score... 30

Fig. 7: The three stages of the NNC model... 34

Fig. 8: Process chain map... 36

Fig. 9: Example of effect chain map ... 36

LIST OF TABLES

Tab. 1: Overview of application systems ... 4

Tab. 2: Ideal type characterisations of approaches to IS evaluation ... 10

Tab. 3: Components of value ... 14

Tab. 4: Criteria for a characterisation of evaluation approaches... 19

Tab. 5: Basic structure of a SMART table ... 29

Tab. 6: Description of IE factors... 31

Tab. 7: Example of budget allocation ... 34

Tab. 8: Stages of the customer resource life cycle... 38

Tab. 9: Overview of reviewed evaluation approaches

... 43

Tab. 10: General problems of IS evaluation ... 46

LIST OF ABBREVIATIONS

BSC balanced

scorecard

CAD computer-aided

design

CAM computer-aided

manufacturing

CBA cost-benefit

analysis

CRLC

customer resource life cycle

CRM

customer relationship management

IE information

economics

IS information

system

IT information

technology

NPV

net present value

ROI

return on investment

SMART

simple multi-attribute rating technique

SWIFT

Society for Worldwide Interbank Financial Telecommunication

UK United

Kingdom

US United

States

III

ABSTRACT

This paper critically reviews approaches for the evaluation of investments in information

systems prior to their implementation. First, the ground for the review is prepared by

examining characteristics of evaluation, information systems and value. A classification of 54

evaluation approaches identified in English and German literature is then presented. Examples

of each class are reviewed and their advantages and drawbacks are discussed. Their use in

evaluation practice is analysed through the examination of empirical studies and directions for

future research are given.

1. INTRODUCTION

Today, a significant share of corporate funds is spent on the implementation, upgrading and

maintenance of an information system (IS). Recent studies show that in 2001 the IS budget of

companies worldwide accounted for an averaged 8.8% of total corporate revenues (cf. CSC,

2001). Consequently, a thorough evaluation of investments in information systems before,

during and after the implementation of a project is important. However, the normative

literature reports a great deal of difficulty in the appraisal of these investments (cf. Irani,

2002:11). Although IS evaluation has been an issue for both academics and managers for

more than three decades now, there are still serious concerns about how to select projects for

investments, how to control the development and how to measure benefits after the

implementation (cf. Farbey, 1999:189). This concern has been matched by increased research

activity which prevailed through two broad streams. The first stream aimed to directly

measure the payoff of IS investments for companies and came to mixed conclusions (cf.

Dehning and Richardson, 2002:8). The second stream addressed the question of how IS

investments can actually be assessed by decision-makers and particularly focussed on the

research of evaluation criteria, evaluation methods and the very nature of the evaluation

process (cf. Avgerou, 2000:570).

Of late, several deficiencies in the field of evaluation methods have induced calls for in-depth

research. Academics have criticised the current state of the field as being immature and

fragmented (cf. Mahmood and Szewczak, 1999:491) and have thus demanded "an overview

of the whole panoply of evaluation methods, together with ... the assumptions they depend on

...[in order to enable]... the identification of gaps." (Farbey, Land and Targett, 1999: 205).

Another concern is a growing mismatch of theory and practice of IS evaluation (cf. Arribas

and Inchusta, 1999:151). Although more than 50 techniques for the evaluation prior to

implementation (ex-ante evaluation) have been suggested by the academic literature,

managers still draw some of their IS investment decisions on so-called "acts of faith", i.e. on

their intuition and instincts (cf. Renkema and Berghout, 1997:1; Fitzgerald, 1998:16). These

are worrying phenomena for proponents of rational decision-making.

Therefore, a critical analysis on the potential of the proposed methods, also with regard to

their role in evaluation practice, is desirable. This is the purpose of this thesis which

particularly addresses the following research problems:

(1) What approaches are available for the ex-ante evaluation of investments in

information systems? What are their strengths and weaknesses and to what extent are

they usable in practice?

(2) What role do evaluation approaches play for practical IS investment appraisal?

Essentially, answers to these questions are sought in a comprehensive review of English and

German literature. The practical side of IS evaluation is explored through the analysis of

published empirical studies. Thereby, this paper attempts to reflect the state-of-the-art of the

literature on IS evaluation approaches. Investments into information systems, instead of entire

information systems, are analysed due to the broad definition of such systems in this thesis.

This thesis is organised as follows. In Chapter 2, the ground is prepared through a delimitation

and discussion of the central concepts "evaluation" as general activity, "information system"

as evaluation object and "value" as key evaluation criterion. In particular, evaluation

problems associated with each concept are identified. Chapter 3 presents the criteria for the

selection, classification and characterisation of evaluation approaches within this paper. A

review of representative approaches is given, their individual advantages and drawbacks are

discussed and the usability of all reviewed techniques is examined on a general level. In

Chapter 4, empirical studies on IS evaluation in practice are presented, interpreted and

discussed with respect to the role of evaluation approaches. This reveals several directions for

future research. Finally, conclusions are given in Chapter 5.

2. THEMATIC BACKGROUND

2.1. Information

Systems

In the previous chapter, a brief introduction stated and justified the focal point of analysis in

this thesis. On this ground, the paper continues with examining the background before which

evaluation approaches are applied. Therefore, aspects of information systems, evaluation and

value are subsequently discussed and especially problems associated with IS evaluation are

identified.

The following working definition reflects the fundamental understanding of an IS within this

thesis: an information system is an integrated socio-technical system with a certain life cycle

that aims to provide information within organisations (cf. Krcmar, 2000:20; Alpar, 2002:28;

Stickel, Groffmann and Rau, 1997:336). Being a part of this definition, the concepts of

"information" and "system" require further delimitation due to their ambiguity. Information

encapsulates knowledge for the purpose of taking effective action (cf. Stickel, 2001:2) and

possesses five main attributes: actuality, accuracy, degree of aggregation, form of presentation

and costs (cf. Alpar, 2002:10-11). A system is generally referred to as a "set of interrelated

elements" (Ackoff, 1971:662) while an IS as a special type of system can be characterised as

an open, dynamic and complex system: open, because its elements interact with their

environment; dynamic, because this interaction can yield alterations of the elements' features;

complex, because of the high number of elements and their respective relations (cf. Krcmar,

2000:20).

The above definition raises the question of what the actual elements of an IS are. An answer

to this question might be sought by looking at the disciplinary bases of this work:

organisational theory and computer science. In organisational theory, an organisation

possesses one IS (organisational IS) that comprises all organisational activities and processes

related to information (cf. Krcmar, 2000:20). Information is seen as the foundation for

organisational decision-making processes and it is therefore the system's main function to

enable and to support organisational planning and control (cf. Gutenberg 1983:268).

An organisational IS comprises a formal and an informal side. The formal side is designed by

the organisation and includes for instance the official reporting system whereas the informal

side refers to the informal actions of organisational members such as a phone call to obtain

urgent information (cf. Zmud, 1983:62; Davis, 1974:197-200). In contrast, computer science

has developed a more technical understanding of information systems as a super-class of

application systems (cf. Krcmar, 2000:20). The function of these systems is to automate parts

of the formal organisational IS, thus they can be understood as all hardware and software

components in an organisation (cf. Stickel, 2001:4). As exhibited in Table 1, they address

differently structured problems at different levels of an organisation.

Management level

Problem structure

Transaction level

Operational

Tactical

Strategic

Well-structured

Transaction

processing systems

(TPS)

Office automation

systems (OAS)

Planning systems

(PS)

Management

information systems

(MIS)

Executive

information systems

(EIS)

Semi-structured

Expert systems (ES)

Decision support

systems (DSS) ES

DSS

Unstructured -

Knowledge

discovery systems

(KDS)

KDS -

Tab. 1: Overview of application systems (Adapted from: Alpar, 2002:31)

In this thesis, elements of an IS are understood as the technological infrastructure, i.e.

hardware and software, as well as manual procedures for development and maintenance of the

IS (cf. Davis, 1974:251). Thus, formal organisational IS based on application systems are

referred to as information systems.

Another feature of an IS mentioned in the definition above is its life cycle character. Life

cycle models were suggested in the IS literature for software applications (cf. Krcmar,

2000:110-114) and for entire systems (cf. Farbey, Land and Targett, 1993:16-21). Figure 1

shows a phase model of the IS life cycle that builds on prior work (cf. Farbey, Land and

Targett, 1993:16-21; Boehm, 1976:1227; Macharzina, 1999:306). Since it primarily serves as

See Alpar (2002:31-41) for a detailed description of each application system.

a framework for discussing different functions of evaluation in Chapter 2.2, problems of

identifying the present phase of an IS or the end of the life cycle are not further discussed.

The model depicts the IS lifetime as a repeating cycle comprising six phases. In the strategy

development phase, an IS strategy is being developed using either a bottom-up, a top-down or

a mixed approach and general objectives for the IS sector are formulated. For achieving these

objectives, several investment opportunities, referred to as projects, might exist from which

one or more are chosen in the project selection phase. After that, a detailed planning of the

previously selected project (portfolio) takes place in the project specification phase and leads

to a concrete project plan. In the project implementation phase, this plan is executed and an IS

is developed, extended or modified. Before setting it into operation, outcomes of the project

are tested. The final introduction and operation of the new or changed IS follows in the IS

usage phase. Throughout this phase, the system is subject to alterations due to faults,

insufficient capacity or changed requirements, so that follow-up projects might become

necessary and the cycle might re-start at the project selection phase. The life cycle comes to

an end when changes become overwhelming and replacement is needed. Then, the cycle is

repeated starting either at the first or second stage.

Strategy

development

Project

selection

Project

specification

Project

implementation

IS usage

IS life cycle

end

IS objectives

Project (mix)

Project

plan

New, modified

or extended IS

Required

modifications

or extensions

Overwhelming

change

requirements

Replacement

requirements

Fig. 1: Model of IS life cycle

Within its life cycle, an IS can be subject to different types of investments. An investment in

general is characterised through the allocation of scarce resources, the expectation of a

Such problems are reported, for instance, by Krcmar (2000:113) for a similar software life cycle model.

positive return during a certain period of time and a certain lifetime (cf. Renkema, 2000:100).

Basically, three categories of IS investments can be distinguished: investments to set up a new

system (installation investments), to extend (add-on investments) or to replace (replacement

investments) an existing system (cf. Perridon and Steiner, 1999:29-30). Alternatively, IS

investments can be classified according to the degree of initiated change, ranging from pure

automation of existing routines to business transformations (cf. Farbey, Land and Targett,

1993:121-131). In a synthesis of both categorisations, substitutive, complementary and

innovative IS are distinguished. Substitutive IS replace human labour in order to achieve

calculable cost-savings whereas complementary IS support human labour to generate

estimatable productivity and effectiveness increases. Finally, innovative IS aim to realise

competitive advantage (cf. Parker and Benson, 1988:103; Alpar, 2002:77).

The different roles of information systems in corporations are another point to consider in IS

investment appraisal. A study by Ragowsky et al. (1996:98) confirmed that different

organisations can gain different benefits from the same IS application. Therefore, the

normative literature increasingly recommends to adjust evaluation procedures to

organisational characteristics (cf. Premkumar and King, 1992:101). In particular, the roles of

information systems can differ with regard to two aspects. First, information systems can be

of different strategic relevance for an organisation. While for some companies such as

Amazon and Ebay IS activities are a part of crucial business processes, they play a mostly

supportive role in other organisations e.g. in the handicraft sector (cf. Avison, Cuthbertson

and Powell, 1999:422). Second, the degree of system integration varies across organisations

and can range from stand-alone systems with no integration to sector-wide integration such as

realised through the S.W.I.F.T. network in the banking sector (cf. Stickel, 2001:140-142).

Both aspects directly affect IS evaluation because they determine the reach of potential

investment impacts and the locus of responsibility for system planning. However, as a general

trend of the last few decades, computer-based information systems have penetrated

organisations more and more due to the rising potential and shrinking prices of IS technology

(cf. Premkumar and King, 1992:100). Simultaneously, a trend to the integration of isolated

systems in and across organisations developed (cf. Smithson and Hirschheim, 1998:164).

S.W.I.F.T. (Society for Worldwide Interbank Financial Telecommunication) runs an international network for the electronic data-exchange

between banks.

A great deal of the difficulties associated with IS evaluation stems from characteristics of

information and information systems. With regard to information, the determination of its

value is particularly troublesome. One reason is that information has no intrinsic value

because its value depends on the associated purpose (cf. Droste, 1986:90) and varies inter-

subjectively. Defining the value of information in relation to the decisions that can be based

upon it is also problematic due to intervening factors. For instance, the quality of managerial

decision-making does not only depend on the quality of the underlying information but also

on the individual competences of a manager to make use of this information (cf. Farbey, Land

and Targett, 1993:13-14). Another reason is a fact known as "information paradox": in order

to assess ex-ante an information's worth, its content has to be disclosed to determine the

worth of the activities based on it after which there is no need to acquire the information

anymore (cf. Walter, 1995:203).

Features of an IS account for further problems. First, the returns of investments in such

systems can seldom be completely expressed in monetary terms because, being often of

supportive nature, the system has a great distance to the cash flow generating activities of an

organisation (cf. Huber, 1999:111). Second, the IS lifetime is hard to anticipate due to threats

of technological obsolescence and changed organisational requirements (cf. Farbey, Land and

Targett, 1993:13). Third, information systems are often difficult to distinguish from each

other. Reasons for this are the growing integration of application systems (cf. Smithson and

Hirschheim, 1998:164) and the evolution of systems through modifications and extensions

over time (cf. Clegg et al., 1997:857). Fourth, the realised value after an IS project is seldom

exclusively attributable to a particular IS investment, but is due to compound effects (cf.

Anselstetter, 1984:10-11; Huber, 1999:112). For instance, major benefits and costs could not

emerge from the IS investment per se, but more from the reorganisation induced by it (cf.

Stefanou, 2001:206; von Dobschütz, 2000:446). Finally, IS investments often trigger not

entirely predictable, organisation-specific processes of change (cf. Clegg et al., 1997:857).

These processes complicate reliable estimations because they can prevent the realisation of

expected benefits or can cause unforeseen costs (cf. Chircu and Kauffman, 2000:66-67). For

instance, sales personal could hinder the value realisation of a CRM system by resisting to

maintain the necessary data.

2.2. Evaluation

For the purposes of this thesis, the following working definition of IS evaluation is

suggested: a process of determining by quantitative and/or qualitative means the value of an

information system investment to an organisation (cf. Doherty and King, 2001). This

definition implies that two fundamental activities of IS evaluation are the collection of

evaluation-specific data and, based on this, the ascertainment of value (cf. Wittmann,

1985:263). It also indicates that on the whole the organisation as total, not individuals or

groups alone, is supposed to be the primary beneficiary of any IS investment. As a

shortcoming in the above definition, the "nicely ambiguous" (Veryard, 1991:3) character of

the concept "value" necessitates further delimitations that are presented in detail in Chapter

2.3. Essentially, it is understood as the sum of positive ("benefits") and negative

consequences ("sacrifices") of an IS investment that can be of both a financial and non-

financial nature (cf. Renkema and Berghout, 1997:2).

Several taxonomies of IS evaluation are suggested in the academic literature. According to

the timing, evaluation before (ex-ante evaluation) and after (ex-post evaluation) the

implementation of an IS project is generally differentiated (cf. Remenyi, Money and

Sherwood-Smith, 2000:25). A categorisation into formative and summative evaluation is also

proposed. The former involves monitoring the processes and products of system development

and gathering user feedback for use in the refinement and further system development

whereas the latter focuses on assessing the impact, usability and effectiveness of the system

(cf. Remenyi, Money and Sherwood-Smith, 2000:27).

Research and practice have developed a variety of evaluation approaches. To delimitate an

evaluation approach from mere planning techniques, the following definition of ex-ante

evaluation approaches is proposed. An ex-ante evaluation approach describes a procedure

prior to the implementation of an IS in which criteria for the evaluation (as defined above) of

an IS investment are prescribed or can be generated. Several requirements of evaluation

approaches have been suggested in the IS literature (cf. Schumann, 1993:168-169). They

should facilitate an analysis of cost-savings, productivity increases and competitive advantage

associated with an IS investment, in order to make the approach applicable for appraising

substitutive, complementary and innovate information systems. Different types of investment

impacts should be envisaged including quantitative financial, quantitative non-financial and

qualitative factors. In addition, risk as well as the indirect effects involved in IS investments

should be considered.

Academics have approached the topic of IS evaluation from diverse research perspectives,

ranging between two extreme positions: the formal-rational perspective and the interpretive

perspective (cf. Serafeimidis, 2001:66-67). The formal-rational perspective seeks the worth of

an IS in a system's performance and financial profitability whereby it emphasises the

economic and technical aspects. In contrast, the interpretive perspective focuses on the

analysis and understanding of the social and subjective nature of IS evaluation. Particularly,

the interaction of the technology with organisational structures, culture and stakeholders is

addressed. Both perspectives correspond to distinct models of organisational decision-making

(Daft, 1989 in Renkema, 2000:137; Schäfer, 1988:192). In the rational model, evaluation is

regarded as a means to optimise corporate actions by selecting efficient and effective choice

options. The members of the organisation are commonly pursuing organisational goals and

decision power is centralised. In contrast, the political model attributes pluralistic objectives

to the organisational members and power is distributed across shifting interest groups.

Evaluation is viewed as a disorderly process of mediating between heterogeneous coalitions

that tend to use and withheld information strategically. As key difference, evaluation aims to

determine objective value irrespective of context in the first model and context-specific

understanding by the exploration of different views in the second model.

As Table 2 illustrates, different types of evaluation approaches are related to both

perspectives. It should be noted that the views are given as ideal types for illustrative purposes

and to draw the attention to their contrasts. In practice, IS evaluation may take place between

both extremes, reflecting in varying proportions their characteristics.

Overall, this dichotomy of perspectives reflects in a sense the influence of two research

paradigms on the field: the positivistic and the interpretive paradigm (cf. Serafeimidis,

2001:66).

While the interpretive perspective follows the interpretive paradigm and considers

Risk (incertitude) refers to situations in which decision-makers know (do not know) the probabilities for the occurrence of certain events

(cf. Bamberg and Coenenberg, 2002:19).

The interpretive position assumes, in contrast to the positivistic side, that our knowledge of reality is a social construction of human actors

and that value-free data cannot be obtained. Thus, it denies the assumption that hypotheses or theories can be tested by "objective" data (cf.

Walsham, 1995:376).

the partly political context of evaluation, the formal-rational perspective disregards, like the

positivistic paradigm, the context of evaluation.

Formal-rational approaches

Interpretive approaches

Concerned with IS evaluation methods irrespective

of context

Concerned with context in which IS evaluation takes place

Traditional mechanistic methods prevail

New interpretive methods can be applied

Economic factors dominate

Social factors dominate

Legitimises IS evaluation process

Engages with stakeholders in process to understand

assumptions and views

Espouses single objective view

Seeks multiple-stakeholder subjective views

Claims to be apolitical

Recognises IS evaluation as a political process

Official, organisational view of IS evaluation

process

Unofficial, stakeholder view of IS evaluation process

Tab. 2: Ideal type characterisations of approaches to IS evaluation (Adapted from: Jones and Hughes,

2001:193)

The usage of evaluation approaches induces the question of who to assign as evaluators.

Several authors argue for an active participation of key stakeholders in the IS evaluation

process (cf. Avgerou, 1995:435) by asserting that their involvement would create the right

level of commitment needed for a successful project implementation (cf. Renkema, 2000:138;

Markus, 1983:441).

Essentially, three classes of key stakeholders are distinguished in the IS

literature (cf. Remenyi, Money and Sherwood-Smith, 2000:17-19; Buss, 1983:120; Khalifa et

al., 2001; Fitzgerald, 1998:25): (a) information technology (IT) professionals including

internal staff, contractors and consultants, (b) business managers, and (c) end-users. However,

the actual selection of evaluators is a research field in its own rights which is for reasons of

space and time not further regarded.

The literature has acknowledged several reasons for evaluating IS investments prior to their

implementation. From an formal-rational standpoint, it should be, at least in theory, a natural

concern of management to estimate the worth of an investment in order to enable an optimal

Stakeholders can be defined as "all those parties who affect or are affected by a corporation's actions, behavior, and policies" (Jurison,

1996:266).

allocation of scarce resources (cf. Smithson and Hirschheim, 1998:160; Farbey, Land and

Targett, 1993:12). With regard to the large amounts of organisational funding consumed by IS

investments (cf. Farbey, Land and Targett, 1999:203), the determination of the relative merits

of alternative projects competing for resources becomes even more important (cf. Ballantine,

Levy and Powell, 1998:242). Furthermore, situation-specific evaluation is required because

research has so far failed to provide general evidence that organisations can gain productivity

increases through the use of information systems. Instead, empirical findings are very mixed,

leading to the "productivity paradox" debate, where in some cases productivity increases,

while in others it declines (cf. Smithson and Hirschheim, 1998:161). Moreover, ex-ante

evaluation can provide insight to the interface between the technology and fundamental

organisational processes (cf. Hirschheim and Smithson, 1988:21). It generates a set of

measures to exercise control over the implementation and usage of an IS by estimating

resource requirements and benefits to be gained (cf. Farbey, Land and Targett, 1992:110).

Furthermore, measures for the degree of goal achievement by an IS investment can be

produced through evaluation procedures (cf. Irani and Love, 2001:184). Additionally, it

facilitates feedback and learning in order to improve IS evaluation practices and IS

development capabilities (cf. Irani and Love, 2001:184; Farbey, Land and Targett, 1992:110).

Regarded from the interpretive perspective, evaluation can act as a mechanism for gaining

commitment and, in highly politically influenced environments, for legitimisation (cf. Powell,

1992:33). The achievement of understanding and consensus among decision-makers can be

promoted by making often implicit evaluation criteria explicit (cf. Farbey, Land and Targett,

1993:12). Evaluation procedures can also be a part of a regular justification process for

investments (cf. Farbey, Land and Targett, 1992:110) and can function as a political

instrument to rationalise already-made decisions (cf. Schäfer and Wolfram, 1987:37).

Furthermore, having a demonstrable record that a "legitimate" ex-ante evaluation procedure

has been followed can mitigate the people involved if the system does not live up to

expectations.

Evaluation can fulfil diverse functions within the IS life cycle presented in Figure 1 (p.5).

While ex-ante evaluation takes place in the first three phases of the cycle, ex-post evaluation

is conducted in the last three.

In each phase, the focus of appraisal activities differs in terms

of evaluation objectives, objects and criteria (cf. Farbey, Land and Targett, 1993:16-21;

Ex-post evaluation is considered here to show the complete life cycle and to indicate links between ex-ante and ex-post evaluation.

Details

Seiten
Erscheinungsform: Originalausgabe
Jahr: 2003
ISBN (eBook): 9783832470272
ISBN (Paperback): 9783838670270
DOI: 10.3239/9783832470272
Dateigröße: 968 KB
Sprache: Englisch
Institution / Hochschule: Universität Bielefeld – Wirtschaftswissenschaften
Erscheinungsdatum: 2003 (Juli)
Note: 1,0
Schlagworte: iv-controlling it-controlling informationssystem wirtschaftlichkeit nutzen

Autor

Sascha Walter (Autor:in)