Risk management of Information Technology  implementation projects

Software project management is one of the most relevant topic in the study and practice of risk management. Despite relative short life of the area, his complexity and difficulties, recent literature survey (Ropponen, 1999) identified over 34 empirical studies on software project risk.

Special issues of Journal of Information Technology ( no. 11,4/1996) and IEEE Software (no. 14,3/1997) are devoted to the software project risk management.

An interesting paper (Henry Barki, Suzanne Rivard, Jean Talbot, Contingency Model of Software Project Risk Management, HEC, Montreal, Canada, develops an integrative contingency model of software project risk management centered on the fitting between the project's degree of risk exposure and its risk management profile, the extent to wich risk management matches its level of risk exposure.

An overview of main model variables structure is drawn in fig. 1.

The following summarized table offers details about variables, time of measurement and respondent for each task. The conceptualization and the assessing proposals can stay always for a good support in a practical approach.

RISK MANAGEMENT

INTERNAL INTEGRATION. 7-point Likert scale.

Respondent: project leader.

Time of measurement: during development.

For each item the respondent indicated the extent to which the contents of the statement corresponds/does not correspond to what transpired in the project. (Standardized Cronbach alpha =.67)

The project team meets frequently.

Project team members are kept informed about major decisions concerning the project.

Every effort is made to keep project team turnover at a minimum.

Project team members actively participate in the definition of project goals and schedules.

EXTERNAL INTEGRATION (USER PARTICIPATION) 7-point Likert scale.

Respondent: key user.

Time of measurement: three months after project implementation completed.

For each item the respondents indicated the extent to which the contents of the statement completely/not at all describes what transpired in the project (Cronbach alpha = .88).

Users took on the leadership role in the development of the system.

Estimating development costs was users' responsibility.

Evaluating system benefits was users' responsibility.

Covering unforeseen budget increases in the project was users' responsibility.

Selecting the hardware/software was users' responsibility

Users played a major role in the system analysis phase of the project.

Users played a major role in the system design phase of the project.

Users played a major role in the implementation phase of the project.

One or more users acted as liaison between the users and the project team.

Ensuring project success was users' responsibility.

The project team drew up a formalized agreement of the work to be done.

Users were able to make changes to the formal agreements of the work to be done.

The project team kept users informed concerning project progress and/or problems.

Users formally evaluated the work done by the project team.

Users formally approved the work done by the project team.

FORMAL PLANNING 7-point Likert scale.

Respondent: project leader.

Time of measurement: during development.

For each item the respondent indicated the extent to which the contents of the statement corresponds/does not correspond to what transpired in the project (Standardized Cronbach alpha = .82)

Tools such as PERT or CPM are used to closely follow the project's status.

Special attention is being paid to project planning.

Significant resources were allocated to estimate project times and budgets.

SYSTEM QUALITY. 7-point Likert scale.

Respondent: project leader.

Time of measurement: three months after project implementation completed. For each item the respondents indicated the extent to which the contents of the statement completely/not at all describes the system that was developed (Standardized Cronbach alpha = .88).

Reliable (the system runs without errors, does what it is supposed to do, and the information it produces is error-free and accurate).

Ease of use (the system is easy to use).

Secure (the system enables recovery from errors, accidents, and intrusions while maintaining data security and integrity).

Easy to maintain (programming errors can be easily corrected).

Flexible (the system can easily be modified to meet changing requirements).

Technically simple (the programs, the data base structure, and the technical documentation are easy to understand).

Portable (the system can easily be adapted to a new technical or organizational environment).

Efficient in its usage of resources (The system performs its different functions without wasting technical resources).

Testable (it is easy to test whether the system is functioning correctly).

Meets initial objectives (the system conforms to the specifications established at the start of the project).

Advantageous from a cost / benefit point of view (the benefits that will be derived from the system exceed its cost).

Understandable (the system is easy to understand).

Documented (documentation exists describing how the system functions and its structure).

Quick (the system performs its functions within acceptable delays).

Precise (the information produced by the system is precise).

Complete (the range of functions offered by the system is adequate).

Relevant (the information produced by the system is useful for the users).

Recent (the information produced by the system is up to date).

RISK EXPOSURE. From Barki, Rivard and Talbot, (1993).

Respondent: both project leader and key user (as indicated below).

Time of measurement: during development.

Items indicated with an asterisk are reverse coded. Project Risk Exposure score was calculated as Overall Uncertainty (an average of 23 variables, after conversion to same 0 to 1 scale) multiplied by Magnitude of Potential Loss (see Barki et al, 1993, p. 215-216)

TECHNOLOGICAL NEWNESS

NEED FOR NEW HARDWARE (1-item binary scale, respondent: project leader): The new system will require the acquisition and installation of new hardware

NEED FOR NEW SOFTWARE (1-item binary scale, respondent: project leader): The new system will require the acquisition and installation of new software

NUMBER OF HARDWARE SUPPLIERS (1-item ratio scale, respondent: project leader): How many hardware suppliers are involved in the development of this system?

NUMBER OF SOFTWARE SUPPLIERS (1-item ratio scale, respondent: project leader): How many software suppliers are involved in the development of this system?

NUMBER OF USERS OUTSIDE THE ORGANIZATION (1-item ratio scale, respondent: project leader): Approximately how many people external to the organization will be using this system (examples of external users would be customers using an automated bank teller machine, or an airline reservation system)?

APPLICATION SIZE

NUMBER OF PEOPLE ON TEAM (I-item ratio scale, respondent: project leader): How many people are there on the project team?

RELATIVE PROJECT SIZE (3-item Much Lower Than Average/Much Higher Than Average 7-point semantic-differential scale, respondent: project leader):

7a. Compared to other information system projects developed in your organization, the scheduled number of person-days for completing this project is:

7b. Compared to other information system projects developed in your organization, the scheduled number of months for completing this project is:

7c. Compared to other information system projects developed in your organization, the dollar budget allocated to this project is:

TEAM DIVERSITY (1-item 4-point interval scale, one point added for each category checked, respondent: project leader): The project team members fall into which of the following groups (you can check more than one)

Information system or data processing staff,

Outside consultants

Users

Other

NUMBER OF USERS IN THE ORGANIZATION (1-item ratio scale, respondent: project leader): Once it is implemented, how many employees of this organization will be using this system?

NUMBER OF HIERARCHICAL LEVELS OCCUPIED BY USERS (1-item ratio scale, respondent: project leader): What is the total number of different hierarchical levels occupied by the employees who will be using this system (for example, office clerks, supervisors, and managers each occupy different hierarchical levels in an organization)

EXPERTISE

11 LACK OF DEVELOPMENT EXPERTISE IN TEAM (4-item 7-point No Expertise/Outstanding Expertise Likert scale, respondent: project leader): Please evaluate the team's level of expertise in terms of the following:

11 a. Development methodology used in this project

11 b. Development support tools used in this project (e.g. DFD, flowcharts, ER model, CASE tools)

ll c. Project management tools used in this project (e.g. PERT charts, Gantt diagrams, walkthroughs, project management software)

11 d. Implementation tools used in this project (e.g. programming languages, data base inquiry languages, screen generators)

12 TEAM'S LACK OF EXPERTISE WITH APPLICATION (1-item 7-point semantic-differential scale, respondent: project leader): The members of the development team are: Very familiar with this type of application/ Unfamiliar with this type of application

13 TEAM'S LACK OF EXPERTISE WITH TASK (4-item 7-point No Expertise/Outstanding Expertise Likert scale, respondent: project leader): Please evaluate the team's level of expertise in terms of the following:

13a. Overall knowledge of organizational operations

13b. In-depth knowledge of the functioning of user departments 13c. Overall administrative experience and skill

13d. Expertise in the specific application area of this system

TEAM'S LACK OF GENERAL EXPERTISE (6-item 7-point Low/Outstanding Likert scale, respondent: project leader): Please evaluate the overall ability of the development team in terms of:

14a. Ability to work with undefined elements and uncertain objectives lob. Ability to work with top management

14c. Ability to work effectively in a team 14d. Ability to successfully complete a task

14e. Ability to understand the human implications of a new information system 14f. Ability to carry out tasks quickly

15 LACK OF USER EXPERIENCE AND SUPPORT (15-item 7-point Strongly Disagree/Strongly Agree Likert scale, respondent: project leader): Generally speaking, the users of this application:

* 15a. Have a positive opinion regarding the way in which the system can meet their needs

15b. Feel they need computerized support in carrying out the tasks for which the system is developed

15c. Are not enthusiastic about the project

15d. Have negative attitudes regarding the use of computers in their work

* 15e. Are ready to accept the various changes the system will entail

15f. Do not actively participate in requirement definition

15g. Are available to answer the development team's questions

15h. Are aware of the importance of their role in successfully completing the project

15i. Are not very familiar with information system development tasks and life cycle stages

* 15j. Are an integral part of the development team

15k. Are not very familiar with data processing as a working tool

15l. Have little experience with the activities to be supported by the future application

15m. Quickly respond to development team requests (for information, comments, approvals)

* 15n. Will have no constraints in fulfilling their development responsibilities for this system

15o. Are not very familiar with this type of application

APPLICATION COMPLEXITY

16 TECHNICAL COMPLEXITY (3-item Slightly Complex/Highly Complex 7-point semantic-differential scale, respondent: project leader): Referring to the application currently being developed, how would you evaluate the technical complexity of each of the following elements:

16a. The hardware (computers, networks) 16b. The software

16c. The data base

17 NUMBER OF LINKS TO EXISTING SYSTEMS (I-item ratio scale, respondent: project leader): How many existing information systems will be linked to this system?

18 NUMBER OF LINKS TO FUTURE SYSTEMS (1-item ratio scale, respondent: project leader): How many information systems currently under development will be linked to this system?

ORGANIZATIONAL ENVIRONMENT

19 EXTENT OF CHANGES BROUGHT (4-item 7-point semantic-differential scale, respondent for 3a and 3b: project leader, for 3c and 3d: user representative)

19a. The development of this system will require that user tasks be modified: Slightly/ A great deal

19b. In general, this system will lead to: Few organizational changes/Major organizational changes

19c. The development of this system will require that user tasks be modified: Slightly/ A great deal

19d. In general, this system will lead to: Few organizational changes/Major organizational changes

20 RESOURCE INSUFFICIENCY (3-item More Than Enough/Extremely Insufcient 7-point semanticdifferential scale, respondent: project leader)

20a. In order to develop and implement this system, the scheduled number of person-days is: 20b. In order to develop and implement this system, the scheduled number of months is:

20c. In order to develop and implement this system, the dollar budget provided is:

21 INTENSITY OF CONFLICTS (6-item 7-point semantic-differential scale, respondent for 21 a, 21 b, and 21c: project leader, for 21d, 21e, and 21f. user representative): In this project, conflicts between team members:

21a. Rarely occur/ Frequently occur

21b. Are not very serious/ Are very serious

21c. Concern relatively unimportant matters/ Concern very important matters

In this project, conflicts between the users and the team members:

21 d. Rarely occur/ Frequently occur

21e. Are not very serious/ Are very serious

21 f. Concern relatively unimportant matters/ Concern very important matters

22 LACK OF CLARITY OF ROLE DEFINITIONS (3-item 7-point semantic-differential scale, respondent: project leader)

22a. The role of each member of the project team is: Clearly Defined/ Not Clearly Defined

22b. Communications between those involved in the project are: Pleasant/ Unpleasant

22c. The role of each person involved in the project is: Clearly Defined/ Not Clearly Defined

23 TASK COMPLEXITY (20-item 7-point semantic-differential scale, respondent: user representative)

23a. The sequence of steps to be carried out to successfully complete these activities is: Easy to Identify/ Hard to Identify

23b. While the consequences of some activities are easy to predict, others are often unpredictable. The consequences of the activities in question are: Easy to Predict/ Hard to Predict

23c. A well-defined body of knowledge on which to base the execution of these activities: Exists/ Does Not Exist

23d. In general, one can determine whether or not the activities were successfully performed: Immediately/After a long period of time

23e. When problems arise in carrying out these activities, getting help is: Easy/ Difficult

23f. When carrying out these activities, problems which cannot be immediately resolved arise: Rarely/ Frequently

23g. Solving these problems typically requires: Little time/ A Lot of time

23h. In your opinion, these activities are: Routine/ Always new

23i. In general, carrying out these activities requires the use of A Large Number of Methods and Procedures / A Small Number of Methods and Procedures

23j. These rules and procedures are: Rarely Subject to Change/ Frequently Subject to Change

23k. Carrying out these activities requires: A Large Number of Different Steps/ A Small Number of Different Steps

23l. These activities can be performed in: Many Different Ways/ Only One Way

23m. Carrying out these activities generally involves: A Large Number of Repetitive Tasks/ A Small Number of Repetitive Tasks

23n. When carrying out these activities, the extent of variety with respect to situations, actors, and tasks is: Low/ High

23o. Regardless of the actors or the specific situations, the tasks and the procedures involved in carrying out these activities are: Always the Same/ Extremely Varied

23p. In carrying out these activities: There Is a Single Objective To Reach/ There Are Multiple Objectives To Reach

23q. When carrying out these activities all objectives: Can Be Reached/ Cannot Be Reached

23r. When choosing a specific way to proceed: One Knows What The Result Will Be/ One Does Not Know What The Result Will Be

23s. When evaluating the way in which all of these activities were carried out, the measure of their success is based on: One criterion/ Several criteria

23t. Carrying out these activities depends on the execution of Many Other Related Activities/ Only a Few Other Related Activities

24 MAGNITUDE OF POTENTIAL LOSS (11-item 7-point Little Impact/Large Impact Likert scale, respondent: project leader)

If, for some reason, the information system being developed is not implemented or if it has operational problems, what impact would this have on your organization in terms of the following:

24a. Customer Relations

24b. Financial Health

24c. Reputation of the Information System Department

24d. Profitability

24e. Competitive Position

24f. Organizational Efciency

24g. Organizational Image

24h. The Survival of The Organization

24i. Market Share

24j. Reputation of The User Department

24k. Ability to Carry Out Current Operations