Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘portfolio management’ Category

Symptoms, not causes: a systems perspective on project failure

with 20 comments

Introduction

A quick search reveals that the topic of project failure gets a fair bit of attention on the Internet. Many of the articles identify factors such as lack of executive support or incomplete/misunderstood requirements as the prime causes of failure. In this post I use concepts from systems theory  to argue that commonly identified “causes” of project failure – such as the ones noted above – are symptoms rather than causes.  I then  surface the real causes of project failure by taking a systems perspective –  a viewpoint that considers the project and the hosting organisation as a whole.

As I will argue, project failures can  often be traced back to dysfunctional structures and processes within the organisation.  These factors usually have little to do with the project directly and are therefore not always obvious at first sight.

Setting the scene

Readers who have waded through the vast literature on failed projects will have noted that there are diverse opinions on the prime reasons for failure. It would take me too long to wade through these articles, so I’ll just pick a source that is well known even  if not entirely credible: The Chaos Report by the Standish Group.  As per this summary  the Chaos Report 2009 claimed the following were the top three causes of project failure:

  1. Lack of user input.
  2. Incomplete or changing requirements/specifications.
  3. Lack of executive support.

Although the report lists the top ten factors, in the interests of space I’ll focus on just these three.

I should mention that the report refers to the above as  “Project Challenged Factors.” Grammar issues aside, this is a somewhat strange way of putting it. Anyway, I interpret this phrase to mean reasons (or driving factors) for failure.

Systems theory and projects

First up, what exactly is a system?

Here is a jargon-free definition from the wonderful book by Donella Meadows entitled, Thinking in Systems: A Primer. Incidentally, I highly recommend the book as an easy-to-read and engaging introduction to systems theory:

A system is a set of things interconnected in such a way that they produce their own pattern of behaviour over time. The system may be buffeted, constricted , triggered or driven by outside forces. But its response to these is characteristic of itself and is seldom simple in the real world.  (italics mine)

The word interconnected is  important because it tells us that when we study something from a systems perspective, we must identify all the important connections it has with its environment.  In the case of projects, the important connections are fairly obvious. A project is usually carried out within an organisational setting so it would have many connections to the hosting organisation. Chief amongst these is the fact that  a project is staffed and resourced by the hosting organisation (or by an organisation designated by it). Another important connection is that  a project will affect  different stakeholder groups within the hosting organisation in different ways.  However,  since all stakeholders have ongoing organisational roles  that go beyond the project, the project is not their only interest. This is a key point to which we will return later.

The phrases “own pattern of behaviour over time” and “characteristic of itself”  tell us that systems have a unique pattern of behaviour that is characteristic to them.  The important point to note is that characteristic behaviour implies that different systems  that have the same signature – i.e. identifying features – will tend to behave in the same way. From the perspective of projects  this tells us that projects within similar organisations will evolve in similar ways.

A systems view of the causes of project failure

With only this very brief look at systems theory we are now in a position to get some insights into  the real causes of project failure. As mentioned earlier, we will focus on the top three reasons ala Standish.

Lack of user input

First up, consider lack of user input. Systems  theory tells us that we need to look at the issue from the perspective of the project and the organisation. From this point of view it is clear that users who have been asked to work on the project in addition to their normal duties will view the project as a burden rather than something that may benefit them in the future.

This by itself is not a new insight. In fact, project management gurus have talked themselves hoarse about the need to free up resources etc. However, the point is that organisational structures typically work against this.  Firstly, people who are asked to work on a project know that it is an initiative that will end in a finite (usually, reasonably short) time. Therefore, their long terms interests lies in their ongoing roles rather than  in projects. Secondly, most organisations are  still structured along functional lines and people’s identities are anchored within their functional reporting lines rather than ephemeral project hierarchies.

Changing requirements

The issue of changing requirements and specifications can also be understood from a systems point of view.  A characteristic of many systems is that they are stable – i.e. they resist change.  Organisations are typically stable systems – they tend to retain their identity despite changes in their environment.  One of the characteristics of such organisations is that people in them tend to think and act in set ways – that is, their actions and thinking processes follow well worn patterns to the point where they do not need to think about what they do too deeply.

One of the consequences of this is that when they are asked for requirements users often provide incomplete description of what they do, leaving out significant items that are obvious to them (but not to the analysts who are gathering requirements!).  Although I don’t have the figures to back this – I speculate that a fair proportion of  changes in requirements  are the result of  inadequate detail or thought put into developing  initial requirements. The point is users and sponsors don’t necessarily  see these as changes, but project teams do.

Lack of executive support

Finally, let’s look at the the problem of lack of executive support.  Project sponsors usually hold important executive-level positions within the hosting organisation. By virtue of their positions,  they have a number of important things that compete for their attention. A project – even an important one – is only one of many things going on in an organisation at any one time.  Moreover,  organisational priorities do shift, perhaps more often than executives may want to admit.  So a project that was the key focus yesterday may be superseded by other priorities today.

There are of course many other ways in which project sponsors can be distracted, but I think I’ve made my point which is that lack of executive support is due to features that are inherent in organisations. So no amount of forcing executives to pay attention to their projects is going to work unless the entire system  (project + organisation) changes.  And this is difficult if not impossible to achieve because stable systems such as organisations tend to resist change, and therefore continue to display their characteristic patterns of behaviour.

Discussion

So we see that the causes of project failure can be traced back to the organisations in which they are embedded.  Specifically, they lie in unwritten norms and formal policies that dictate how the hierarchy operates and how things are done within the organisation. The most important consequence of this is that standard fixes (of encouraging user input and executive support, or instituting change management, say)   will not cure the problem of project failure because they do not address the dysfunctional norms and policies that are the root cause of failure.

The above is not news. In fact, the matrix organisation structure was proposed as a response to the need for “project friendly” organisations. I’m no expert on matrix organisations so I will leave it to others to comment on how successful they are. The only point I would make is that  in my experience multiple reporting lines (even if dotted and solid) do not work too well. There are always conflicting interests that cause divided loyalties and conflicting interests.

So,  the natural question is : what – if anything – can we do about this? The answer is implicit in the foregoing paragraphs. One has to align the project with the organisation, not just at the level of objectives or structure, but also in operational matters such as timelines, budget and resources –  the very things that make up the so-called  iron triangle.  The best time to do this is at the front-end of the project,   i.e. the start. At this time, the person(s) driving the project have to engage all stakeholders who will be affected by the initiative and find out their motivations, interests and – most importantly – their concerns regarding the project.  If this discussion happens in an open and frank manner,  it should surface the issues highlighted In the previous section.  Since the discussion takes place even before the project starts, there is at least some hope of addressing these concerns.

There are many ways to structure and facilitate such discussions. Check out this post for an introduction to one and have a look at my book co-authored with Paul Culmsee for much more. That said, one doesn’t need any particular technique – the willingness to discuss difficult matters openly and an openness to other points of view is all that’s needed. That, however, is not always easy to come by…

Conclusion

We have seen that the top causes of project failure can be traced back to the hierarchies and incentive systems of the hosting organisation. Therefore, superficial attempts to fix the problem at the level of individual projects (or even a PMO) will not work. The only hope of addressing the root causes of project failure is to focus on the systemic dysfunctions that cause them.

Written by K

June 18, 2013 at 9:18 pm

A visit from the methodology police – a PMO satire

with 15 comments

They came for me at 11:00 am.

I  was just settling down to finishing that damned business case when I heard the rat-a-tat-tat on my office door. “Come in,” I said, with a touch of irritation in my voice.

The door opened and there they were. They looked at me as though I was something that had crawled out from under a rock. “Mr. Hersey, I presume,” said the taller, uglier one.

“Yes, that’s me.”

“Joe  Hersey?” He asked, wanting to make sure before unloading on me.

“Yes, the one and only,” I said, forcing a smile. I had a deep sense of foreboding now: they looked like trouble;  I knew they couldn’t be enquiring after my welfare.

“You need to come with us,” said the shorter one. I did imply he was handsomer of the two, but I should clarify that it was a rather close call.

“I have better things to do than follow impolite summons from people I don’t know. I think you should talk to my manager. In fact, I will take you to him,” I replied, rising from my chair. “He won’t be happy that you’ve interrupted my business case. He wants it done by lunchtime,” I added, a tad smugly.

“We’ve already seen him. He knows. I would advise you to come with us. It would make life easier for everyone concerned,”  I forget which one of the two said this.

“What is going on?” I asked, toning down my irritation. To be honest, I had no clue what they were on about.

“We’re the methodology police,” they said in unison. I guess they’d had a fair bit of practice scaring the crap out of hapless project managers. “We’re from the PMO,” they added unnecessarily – I mean, where else could they be from.

“Holy s**t,” I said to myself. I was in big trouble.

“Well, Hersey,” said the short one, “I think you owe the PMO an explanation.” Ah, I loved his use of the third person– not “us” but “the PMO.”

We were seated at a table in a meeting room deep in the bowels of the PMO: windowless, with low wattage lighting sponsored by one of those new-fangled, energy-saving, greenie bulbs . The three chairs were arranged in interrogation mode , with the two goons on one side and me – Joseph M. Hersey, Project Manager Extraordinaire – on the other.

I was in trouble alright, but I have this perverse streak in me, “I don’t know what you are talking about,” I said, feeling a bit like a hero from a Raymond Chandler novel. I knew what I had done, of course. But I also knew that I was one of the good guys. The clowns sitting opposite me were the forces of evil…such thoughts, though perverse, lifted my spirits.

I must have smiled because the tall one said, “You think this is funny, do you? We have a direct line to the board and we could make life really unpleasant for you if you continue this uncooperative attitude.”

That was bad. I did not want to be hauled up in front of the big cheese. If I was branded a troublemaker at that level, there would be no future for me in the company. And to be absolutely honest, I actually enjoyed working here – visits from the methodology police excepted, of course.

“OK, tell me what you want to know,” I said resignedly.

“No, you tell us, Hersey. We want to hear the whole story of your subversion of process in your own words. We’ll stop you if we need any clarification.” Again, I forget which one of the two said this. Understandable, I think – I was pretty stressed by then.

Anyway, there is no sense in boring you with all the PMO  and process stuff. Suffice to say, I told them how I partitioned my big project into five little ones, so that each mini project would fall below the threshold criteria for major projects and thus be exempt from following the excruciating methodology that our PMO had instituted.

Process thus subverted, I  ran each of the mini projects separately, with deliverables from one feeding into the next. I’d got away with it; with no onerous  procedures to follow I was free to devise my own methodology, involving nothing more complicated than a spreadsheet updated daily following informal conversations with team members and stakeholders. All this held together – and, sorry, this is going to sound corny – by trust.

The methodology cops’ ears perked up when they heard that word, “Trust!” they exclaimed, “What do you mean by trust?”

“That’s when you believe people will do as they say they will,” I said. Then added, “A concept that may be foreign to you.” I regretted that snide aside as soon as I said it.

“Look, “ said the uglier guy, “I suggest you save the wisecracks for an audience that may appreciate them. “You are beginning to annoy me and a report to the board is looking like a distinct possibility if you continue in this vein.”

I have to say, if this guy had a lot of patience if he was only just “beginning to get annoyed.” I was aware that I had been baiting him for a while. Yes, I do know when I do that. My wife keeps telling me it will get me into trouble one day. May be today’s the day.

“…I do know what trust is,” the man continued, “but I also know that you cannot run a project on warm and fuzzy notions such as trust, sincerity, commitment etc. The only thing I will trust are written signed off project documents.”

Ah, the folly, the folly. “Tell me this, what would you prefer – project documentation as per the requirements of your methodology or a successful project.”

“The two are not mutually exclusive. In fact, methodology improves the chance of success.”

“No it doesn’t,” I retorted.

“It does,” he lobbed back.

Jeez, this was beginning to sound like recess in the local kindergarten. “Prove it,” I said, staking my claim to the title of King of Kindergarten Debates.

“There are several studies that prove the methodologies efficacy,” said the short one, “but that is not the point.”

“All those studies are sponsored by the Institute,” I said, referring to the August Body that maintains the standard. “so there is a small matter of vested interest….anyway, you say that isn’t the point. So what is your point then?.”

“The methodology is an internal requirement, so you have to follow It regardless. We could have a lot of fun debating it, but that is neither here no there. Compliance is mandatory, you have no choice.”

“I did comply,” I said, “none of my projects were over the threshold, so I did not need to follow the methodology.”

“That was subterfuge – it was one project that you deliberately divided into five so that you could bypass our processes.”

I was getting tired and it was close to my lunchtime. “OK, fair point“ I said, “I should not have done that. I will not do it again. Can I go now?”

“Hmm,” they said in unison.  I don’t think either of them believed me. “That’s not good enough.”

I sighed. “What do you want  then?” I asked, weary of this pointless drama.

“You will read and sign this form,” said the short one, “declaring you have been trained in the PMO processes – which you were last year, as you well know – and that you will follow the processes henceforth. I particularly urge you to read and digest the bit about the consequences of non-compliance.” He flicked the form in my direction.

I was not surprised to see that the form  was a multi-page affair, written in 8pt bureaucratese, utterly incomprehensible to mere mortals such as I. I knew I would continue to bypass or subvert processes that made no sense to me, but I also knew that they needed me to sign that form – their boss would be very unhappy with them if I didn’t.  Besides, I didn’t want to stay in that room a second longer than necessary.

“OK, where do I sign,” I said, picking up a pen that lay on the table.

“Don’t you want to read it.”

“Nah,” I said, “I have a pretty fair idea of what it’s about.”

I signed.

“We’re done, Hersey. You can go back to your business case now. But you can be sure that you are on our radar now. We are watching you.”

“Well Gents, enjoy the show. I promise, to lead a faultless life henceforth. I will be a model project manager,” I said as I rose to leave.

“We’re counting on it Hersey.  One more violation and you are in deep trouble.”

I refrained from responding with a wisecrack as I exited, leaving them to the paperwork that is their raison d’etre.

Projects as networks of commitments – a paper preview

with 8 comments

Mainstream project management standards and texts tend to focus on the tools, techniques and processes needed to manage projects. They largely ignore the fact that projects are conceived, planned, carried out and monitored by people. In a paper entitled, Towards a holding environment: building shared understanding and commitment in projects,  Paul Culmsee and I present a viewpoint that puts people and their project-relevant concerns at the center of projects. This post is a summary of the main ideas of the paper which is published in the May 2012 issue of the International Journal of Managing Projects in Business.

Conventional approaches to project management tend to give short shrift to the social aspects of projects –   issues such as stakeholders with differing viewpoints regarding the rationale and goals of a project. Our contention is that problems arising from stakeholder diversity are best resolved by helping stakeholders achieve a shared (or common) understanding of project goals and, based on that, a shared commitment to working towards them. Indeed, we believe this is  a crucial but often overlooked facet of  managing the early stages of a project.

One of the prerequisites to achieving shared understanding is open dialogue –dialogue that is free from politics, strategic behaviours and power games that are common in organisations. Details of what constitutes such dialogue and the conditions necessary for it are described by the philosopher Juergen Habermas in his theory of communicative rationality. Our paper draws extensively on Habermas’ work and also presents a succinct summary of the main ideas of communicative rationality.

The conditions required for open dialogue in the sense described by Habermas are:

  1. Inclusion: all affected stakeholders should be included in the dialogue.
  2. Autonomy: all participants should be able to present their viewpoints and debate those of others independently.
  3. Empathy: participants must be willing to listen to viewpoints that may be different from theirs and make the effort to understand them.
  4. Power neutrality: differences in status or authority levels should not affect the discussion:
  5. Transparency: participants must be completely honest when presenting their views or discussing those of others.

We call an environment which fosters open dialogue a holding environment.  Although a holding environment as characterised above may seem impossible to create, it turns out that an alliance-based approach to projects can approximate the conditions necessary for one. In brief, alliancing is an approach to projects in which different stakeholders agree, by contract, to work collaboratively to achieve mutually agreed goals while sharing risks and rewards in an equitable manner. There are a fair number of large projects that have been successfully delivered using such an approach (see the case studies on the Center for Collaborative Contracting web site).

Once such an approach is endorsed by all project stakeholders, most of the impediments to open dialogue are removed. In the paper we use a case study to illustrate how stakeholder differences can be resolved in such an environment. In particular we show how project-relevant issues and the diverse viewpoints on them can be captured and reconciled using the IBIS (Issue-based information system) notation (see this post for a quick introduction to IBIS).  It should be noted that our concept of a  holding environment does not require the use of IBIS;  any means to capture issues, ideas and arguments raised in a debate will work just as well.  The aim is to reach a shared understanding,  and once stakeholders do this –  using IBIS or any other means – they are able to make mutual commitments to action.

It should be emphasised that an alliance-based approach to projects  takes a fair bit of effort and commitment from all parties to implement successfully. In general such effort is justifiable only for very large projects,  typically public infrastructure projects (which is why many government agencies are interested in it). It is interesting to speculate how such an approach can be “scaled down” to smaller projects like the ones undertaken by corporate IT departments. Unfortunately such speculations are not permitted in research papers. However,  we discuss some of these at length in our book,  The Heretic’s Guide to Best Practices.

In their  ground-breaking book on design Terry Winograd and Fernando Flores describe organisations as networks of commitments.  We believe this metaphor is appropriate for  projects too. As we state in the paper,  “Organisations and projects are made up of people, and it is the commitments that people make (to carry out certain actions) that make organisations or projects tick. This metaphor – that projects are networks of commitments – lies at the heart of  the perspective we propose in this paper. The focus of project management ought to be on how commitments are made and maintained through the life of a project.

On the unintended consequences of organisational change

with 8 comments

Introduction

Change, as the cliché goes, is the only constant.  At any given time, most organisations are either planning or implementing changes of some kind.  Perhaps because of its ubiquity, the rationale and results of change are not questioned as deeply as they ought to be.  In this post I describe some unintended effects of organisational change, drawing on Barbara Czarniawska’s book, A Theory of Organizing and other sources. I also briefly discuss some ways in which these side effects can be avoided.

I’ll begin with a few words about terminology.  In this article planned changes (also referred to as reforms) are changes instituted in order to achieve specific goals. The goals of reforms are referred to as planned effects – that is, planned effects are intended results of change. As I discuss below, although planned effects may eventually be achieved, change initiatives have a host of unforeseen but significant consequences. These are referred to as unplanned, unintended or side effects.

This article is organised as follows: I’ll begin by describing some of the positive and negative side effects of change, following which I’ll discuss why side effects come about and how they can be managed.

Advantageous side effects of change

Although, the term side effect has a negative connotation, some side effects of change can actually be advantageous.  These include:

  1. Questioning of the status quo: In most organisations, processes and structures are taken for granted, rarely is the status quo questioned. Organisational change presents an opportunity to pose those “How can we do this better?” type questions that challenge the way things are done. Such questioning is unplanned in that it generally occurs spontaneously.
  2. Opportunities for reflection: This is a consequence of the previous point: questioning the status quo can cause people to reflect on how things can be done better. Again, this is an unintended consequence of a reform, not part of its planned goals. Also, it should be noted that although opportunities for reflection arise often, they are generally ignored because of time pressures.
  3. Spontaneous inventions: Finally, questioning of and reflecting on the status quo can trigger ideas for improvement.

Most people would agree that the above points are indeed Good Things that ought to be encouraged.  However, the important point is that people who are in the throes of a planned change seldom have the time or motivation to pursue these opportunities.

Harmful side effects of change

The negative side effects of planned changes are insidious because they tend to occur as a result of inaction – i.e. by not taking corrective actions to counter the detrimental effects of change. The following side effects serve to illustrate this point:

  1. The aims of reform become cast in stone: The objectives of a change initiative are formulated based on an understanding of a situation as it exists at a particular point in time. Problem is, as time evolves the original objectives maybecome irrelevant or obsolete. Yet, in many (most?) change initiatives, objectives are rarely reviewed and adjusted.
  2. The means get confused with the ends: Following from the previous point, a change initiative becomes pointless when its objectives are no longer relevant.  However, a common reaction in such situations is to continue the initiative, justifying it as a worthwhile end in itself. For example, if the benefits of, say, a restructuring initiative become moot,  the  restructuring itself  becomes the objective rather than the benefits that were supposed to flow from it.  This helps save face as the project can be declared a success once the restructuring is completed, regardless of whether or not the promised benefits are realised.
  3. Improvisations and spontaneous inventions are suppressed: As I have discussed at length in this post, planning and improvisation are complementary but contradictory aspects of organizational work. A negative aspect of planned change initiatives is that they are inimical to improvisations: those responsible for overseeing the change tend to ignore, even suppress any improvisations that arise because they are seen as getting in the way of achieving the objectives of the primary change.

Planned change initiatives are generally implemented through programs or projects.  In fact, most major projects in organisations – restructurings, enterprise system implementations etc – are aimed at implementing reforms of some kind. However, although the raison d’etre of such projects is to achieve the planned objectives, many suffer from the negative side effects mentioned above.   In her book Czarniawska states, “Planned change rarely, if ever, leads to planned effects.”  Although this claim may be a tad exaggerated, the significant proportion of large projects that fail suggests there is at least a whiff of truth about it.

In the next two sections I take a brief look at why planned changes fail and what can be done about it.

The origin of the side effects of change

Most structures and processes within organisations have a complex, path-dependent history. Among other things, they develop in ways that are unique to an organisation and are often deeply intertwined with each other.  As a result, it is impossible to be certain about the consequences of changing processes or structures – there are just too many variables and dependencies involved.

There are two related points that flow from this:

Firstly, those who plan changes need to have a good understanding of legacy: the history of the issues that the change aims to fix and those that it may create in the future. The problem is most of the people involved in planning, initiating and executing reforms have little appreciation of such issues.

Secondly, most major changes are conceived by a small number of people who hold positions of authority within organisations. These folks have a tendency to gloss over complexities, and often fail to involve those who have a detailed knowledge of the affected processes and structures. Consequently, their plans overlook dependencies and possible knock-on effects that can arise from them. This results in the negative side effects discussed in the previous section.

..and what can be done about them

Czarniawska recommends the following informal rules for successful change:

  1. Be willing to modify the objectives of the change and your path to get there as your understanding of it evolves.
  2. Implement lightweight processes, avoid bureaucratic procedures.
  3. Be open to improvisations.

This is good advice as it goes, but how exactly does one use it?

In our recently published book, The Heretic’s Guide to Best Practices, Paul Culmsee and I discuss how issues of legacy and lack of inclusiveness can be addressed.

Firstly, we suggest that apart from time, cost and scope (the classic iron triangle), project decision-makers would be well served by considering legacy as a separate variable in projects (also see this post on Paul’s blog for more on this point). More importantly, we describe techniques that can be used to surface hidden assumptions and aspects of history that could have a bearing on the project and those that might cause problems in the future.

Secondly, we discuss how one can work towards creating an environment in which a diverse group of stakeholders can air and reconcile their viewpoints. Such a discussion is a prerequisite to creating a plan that: a) considers as many viewpoints (variables) as possible and b) has the support of all stakeholders.  Without this, any implementation is bound to have side-effects because of overlooked variables and/or the actions (or non-actions) of stakeholders who do not support the plan.

Of course, inclusiveness sounds great but it can be difficult in practice, especially in large organisations. What can decision-makers do in such cases?  The answer comes from a slightly different, if rather obvious direction.

In his very illuminating book on decision-making, James March notes that organisations face messy and inconsistent environments. Given this, decisions made and implemented at lower levels have a better chance of success than those made in rarefied air of board-rooms.  Paraphrasing a statement from his book:

Since knowledge of local conditions and specialized competencies are both essential and more readily found in decentralized units, control over the details of policy implementation and adaptation of general policies to local conditions are [best] delegated to local units. From the standpoint of general management, the strategy is usually seen as one of gaining the informational and motivational advantages of using people with local involvement, [but] at the cost of accentuating problems of central coordination and control.

Indeed, most of the nasty side effects of planned change arise from over-centralisation of coordination and control.  The solution is to devolve control and decision-making authority down to the level at which the changes are to be implemented.

Conclusion

Planned change fails to achieve its goals because planners cannot foresee all the consequences of change or even know which factors may be important in determining these. Moreover, individuals will view changes through the lens of their background, biases and interests.  Since organisations consist of many individuals with different views, managing change is essentially a wicked problem.

To sum up, those who initiate large-scale changes should keep in mind the law of unintended consequencesany planned action will have consequences that are not intended, or even foreseen.  These consequences can be managed by getting a better appreciation of the factors that affect the processes and the structures to be changed.   One can gain an understanding of these factors through a consideration of legacy and/or via dialogue involving all those who work with the processes and structures that are to be changed. The simplest way to achieve both is by delegating decision making and implementation authority down to where it belongs – with the people who work at the coalface of the organisation.

The Labyrinths of Information – a book review

with 3 comments

 Introduction

Once implemented, IT systems can evolve in ways that can be quite different from their original intent and design.  One of the reasons for this is that enterprise systems are based on simplistic models that do not capture the complexities of real organisations. The gap between systems and reality is the subject of a fascinating book by Claudio Ciborra entitled, The Labyrinths of Information.  Among other things, the book presents an alternative viewpoint on systems development,  one that focuses on reasons for divergence between design and reality. It also discusses other aspects of system development that tend to be obscured by mainstream development methodologies and processes. This post is a summary and review of the book.

 Background

The standard treatment of systems development in corporate environments is based on the principles of scientific management. Yet, as Ciborra tells us,

…science-based, method-driven approaches can be misleading.  Contrary to their promise, they are deceivingly abstract and removed from practice. Everyone can experience this when he or she moves from the models to the implementation phase. The words of caution and pleas for ‘change management’ interventions that usually accompany the sophisticated methods and polished models keep reminding us of such an implementation gap. However, they offer no valid clue on how to overcome it…

Just to be clear, Ciborra offers no definitive solutions either. However, he offers  “clues on how to bridge the gap” by  looking into some of the informal techniques and approaches that people “on the ground” – users, designers, developers or managers –  use to work and cope with technology. He is not concerned with techniques or methodologies per se, but rather with how people deal with the messy day-to-day business of working with technology in organisations.

The book is organised as a collection of essays based on Ciborra’s research papers spanning a couple of decades – from the mid 1980s until a few years prior to his death in 2005. I discuss each of the chapters in order below, providing links to the original papers where I could find them.

 The divergence between models and reality

Most of the tools and techniques used in systems evaluation, design and development are based on simplified models of organisational reality. However,  organisations do not function according to organograms, data flow diagrams or entity-relationship models. Models used by systems professionals abstract away much of the messiness of real-life. The methods that come out of such simplifications cannot deal with the complexities of a real organisation.  As Ciborra states, “…concern with method is one of the key aspects of our discipline and possibly the true origin of its crisis…

Indeed, as any systems professional will attest to,  unforeseen occurrences and situations  inevitably encountered in real life  are what cause the biggest headaches in the implementation and acceptance of systems. Those on the ground deal with such exceptions by creative but essentially ad-hoc  approaches. Much of the book is a case-study based discussion of such improvised approaches to systems development.

 Making (do) with what is at hand

Ciborra argues that successful systems are invariably imitated by competitors, so any competitive advantage offered by such systems is, at best, limited. A similar argument holds for standards and best practices – they  promote uniformity rather than distinction. Given this, organisations should strive towards practices that cannot be copied. They should work towards inimitability.

In art, bricolage refers to a process of creating a work from whatever is at hand. Among other things it involves tinkering, improvising and generally making do with what is available. Ciborra argues that many textbook cases of strategic systems in fact evolved through bricolage, tinkering and serendipity, rather than plan. Some of the cases he discusses include Sabre Reservation System developed by American Airlines, and the development of Email (as part of the ARPANET project). Moreover, although the Sabre System afforded American Airlines a competitive advantage for a while, it soon became a part of the travel reservation infrastructure thereby becoming an operational necessity rather than an advantage. This is much the same point that Nicholas Carr made in his article, IT Doesn’t Matter.

The question that you may be asking at this point is: “All this is well and good, but does Ciborra have any solutions to offer?” Well, that’s the problem: Ciborra tells us that bricolage and improvisation ought to be encouraged, but offers little advice on how this can be done. For example, he tells  us to “Value bricolage strategically”, “Design tinkering” and “Establish systematic serendipity”  – sounds great in theory, but what does it really mean? It  is platitudinous advice that is hard to action.

Nevertheless his main point is a good one: that managers should encourage informal, creative practices instead of clamping down on them. This advice has not generally been heeded. Indeed, corporate IS practices have gone the other wa, down the road of standardisation and best practices. Ciborra tells us in no uncertain terms  that this is not a good thing.

 The enframing effect of technology

This part is, in my opinion, the most difficult chapter in the book. It is based on a paper by Ciborra and Hanseth entitled, From tool to Gestell: Agendas for managing the information infrastructure.   In German the term Gestell means shelf or rack.  The philosopher Martin Heidegger used the term to describe the way  in which technology frames the way we view (or “organise”)  the world.   Ciborra highlights the way in which existing infrastructure affects the success of  businesses processes and practices.  Ciborra emphasises that technology-based enterprise initiatives are doomed to fail unless they pay due attention to:

  1. Existing or installed infrastructure.
  2. Local needs and concerns.

Instead of attempting to oust old technology, system designers and implementers need to co-opt or cultivate the installed base (and the user community) if they are to succeed at all.  In this sense installed infrastructure is an actor (like an individual) with its own interest and agenda. It provides a context for the way people think and also influences future development.

The notion of Gestell thus reminds us of how existing technology influences and limits the way we think. To get around this, Ciborra suggests that we should:

  1. Be aware of technology and standards, but not be captive to them.
  2. Think imaginatively, but pay attention to the installed base (existing platforms and users).
  3. Remember that top down technology initiatives rarely succeed.

The drifting of information infrastructure

Ciborra uses Donald Schoen’s metaphor of the high ground and the swamp to highlight the gap between theory and practice of information systems (see this paper by Schoen, for a discussion of the metaphor). The high ground is the executive management view,where methodologies and management theories hold sway, while the swamp is the coalface where messy, day-to-day reality of organisational work unfolds. In the swamp of day-to-day work, people tend to use available technology in any way possible to solve real (and messy) problems. So, although a particular technology may have an espoused or intended aim, it may well be used in ways that are completely unforeseen by its designers.

The central point of this essay is that the full implications of a technology are often realised only after it has been implemented and used for a while. In Ciborra’s words, technology drifts – that is, it is put to uses that cannot be foreseen. Moreover, it may be never be used in ways that were intended by the designer.   Although Ciborra lists several cases that demonstrate this point, in my opinion, his blanket claim  that technology drifts is a bit over the top. Sure, in some cases, technologies may be used in unforeseen ways, but by and large they are used in ways that are intended and planned.

 The organisation as a host

Reactions to a  new technology in an organisation are generally mixed – some people may view the technology with some trepidation (because of the changes to their work routines, for instance) while others may welcome it (because of promised efficiencies, say).  In metaphorical terms, the new technology is a “guest,” whose “desires” and “intentions” aren’t fully known. Seen in this light of this metaphor, the notion of hospitality makes sense:  as Ciborra puts it, the organisation hosts the technology.

To be sure, the idea of hospitality applying to objects such as information systems will probably cause a few raised eyebrows. However it isn’t as “out there” as it sounds. Consider, for example, the following implications of the metaphor

  • Interaction between the host and guest can change both parties.
  • If the technology is perceived as unfriendly, it will be rejected (or even ejected!).
  • System development and operations methodologies are akin to cultural rituals (it is how we “deal with” the guest).
  • Technologies, like guests, stay for a while but not forever.

Ciborra’s intent  in this and most of the other essays is to make us ponder over the way we design, develop and run systems,  and possibly view what we do in a different light.

 The organisation as a platform

In this essay Ciborra looks at the way in which successful technology organisations adapt and adjust to rapidly changing environments. It is  based on his paper entitled, The Platform Organization: Recombining Strategies, Structures and Surprises, Using a case-study, he makes the point that the only way organisations can respond to rapidly evolving technology markets is to be open to recombining available resources in flexible ways: it is impossible to start from scratch; one has work with what is at hand, using it in creative ways.

Another point he makes is that the organisation of an organisation (hierarchy and structure) at any particular time is less important than how it gets there, where it’s headed and what are the obstacles in the way. To quote from the book:

 …analysing and evaluating the platform organisation at a fixed point in time is of little use: it may look like a matrix, or a functional hierarchy, and one may wonder how well its particular form fits the market for that period and what its level of efficiency really is. What should be appreciated, instead, is the whole sequence of forms adopted over time, and the speed and friction in shifting from one to the other.

However, the identification of such a trajectory can be misleading – despite after-the-fact rationalisations, management in such situations is often based on improvised actions rather than carefully laid plans.  Although this may not always be so, I suspect it is more common than managers would care to admit.

 Improvisation and mood

By now the reader would have noted that Ciborra’s focus is squarely on the unexpected occurrences in day-to-day organisational work. So it will come as no surprise that the last essay in the book deals with improvisation.

Ciborra argues that most studies on improvisation have a cognitive focus – that is, they deal with how people respond to emerging situations by “quick thinking.” In his opinion, such studies ignore the human aspect of improvised actions, the emotions and moods evoked by situations that call for improvisation. These, he suggests, can be the difference between improvised actions and panic.

As he puts it, people are not cognitive robots – their moods will determine whether they respond to a situation with indifference or interest and engagement. This human dimension of improvisation, though elusive, is the key to understanding improvisation (and indeed, any creative / innovative action)

He also discusses the relationship between improvisation and time – something I have discussed at length in an earlier post, so I’ll say no more about it here.

 A methodological postscript

In a postscript to the book, Ciborra discusses his research philosophy – the thread that links the essays in the book.. His basic contention is that methodologies and organisational models are based on after-the-fact rationalisations of real phenomena. More often than not such methods and models are idealisations that omit the messiness of real life organisations. They are abstractions, not reality. As such they can guide us, but we should be ever open to the surprises that real life may afford us.

Summarising

The essential message that Ciborra conveys is a straightforward one – that the real world is a messy place and that the simplistic models on which systems are based cannot deal with this messiness in full. Despite our best efforts there will always be stuff that “leaks out” of our plans and models. Ciborra’s book celebrates this messiness and reminds us that people matter more than systems or processes.

Written by K

September 8, 2011 at 10:41 pm

Processes and intentions: a note on cause and effect in project management

with 9 comments

Introduction

In recent years the project work-form has become an increasingly popular mode of organizing activities in many industries. As the projectization bandwagon has gained momentum there have been few questions asked about the efficacy of project management methodologies. Most practitioners take this as a given and move on to seek advice on how best to implement project management processes. Industry analysts and consultants are, of course, glad to oblige with reams of white papers, strategy papers or whatever else they choose to call them (see this paper by Gartner Research, for example).

Although purveyors of methodologies  do not claim their methods guarantee project success, they imply that there is a positive relationship between the two.  For example, the PMBOK Guide tells us that, “…the application of appropriate knowledge, processes, skills, tools and techniques can have a significant impact on project success”  (see page 4 of the 4th Edition).  This post is a brief exploration of the cause-effect relationship between the two.

Necessary but not sufficient

In a post on cause and effect in management, I discussed how  the link between high-level management actions and their claimed outcomes is tenuous. The basic reason for this is that there are several factors that can affect organizational outcomes and it is impossible to know beforehand which of these factors are significant. The large number of  factors, coupled with the fact that  controlled experimentation is impossible in organizations, makes it impossible to establish with certainty that a particular managerial action will lead to a desired outcome.

Most project managers are implicitly aware of this – they know that factors extrinsic to their projects can often spell the difference between success and failure. A mid-project change in organizational priorities is a classic example of such a factor. The effect of such factors can be accounted for using a concept of causality proposed by the philosopher Edgar Singer, and popularised by the management philosopher Russell Ackoff. In a paper entitled Systems, Messes and Interactive Planning, Ackoff had this to say about cause and effect in systems – i.e. entities that interact with each other and their environment, much like  organizations and projects do:

It will be recalled that in the Machine Age cause-effect was the central relationship in terms of which all actions and interactions were explained. At the turn of this century the distinguished American philosopher of science E.A. Singer, Jr.noted that cause-effect was used in two different senses. First… a cause is a necessary and sufficient condition for its effect. Second, it was also used when one thing was taken to be necessary but not sufficient for the other. To use Singer’s example, an acorn is necessary but not sufficient for an oak; various soil and weather conditions are also necessary. Similarly, a parent is necessary but not sufficient for his or her child. Singer referred to this second type of cause-effect as producer-product. It has also been referred to since as probabilistic or nondeterministic cause effect.

The role of intentions

A key point is that one cannot ignore the role of human intentions in management. As Sumantra Ghoshal wrote  in a classic paper:

 The basic building block in the social sciences, the elementary unit of explanation is individual action guided by some intention. In the presence of such intentionality functional [and causal] theories are suspect, except under some special and relatively rare circumstances, because there is no general law in the social sciences comparable to [say] the law of natural selection in biology

A producer-product view has room for human intentions and choices. As Ackoff stated in the his paper on systems and messes,

Singer went on to show why studies that use the producer-product relationship were compatible with, but richer than, studies that used only deterministic cause-effect. Furthermore, he showed that a theory of explanation based on producer-product permitted objective study of functional, goal-seeking and purposeful behavior. The concepts free will and choice were no longer incompatible with mechanism; hence they need no longer be exiled from science.

A producer-product view of cause and effect in project management recognizes that there will be a host of factors that affect project outcomes, many of which are beyond a manager’s ken and control. Further, and possibly more important, it acknowledges the role played by intentions of individuals who work on projects.  Let’s take a closer look at these two points.

Processes and intentions

To begin, it is worth noting that that project management lore is rife with projects that failed despite the use of formal project management processes. Worse, in many cases it appears that failure is a consequence of over-zealous adherence to methodology (see my post entitled The myth of the lonely project for a detailed example).  In such cases the failure is often attributed to causes other than the processes being used –   common reasons include lack of organizational readiness and/or improper implementation of methodology from which the processes are derived.   However, these causes  can usually be traced back to  lack of  employee buy-in,  i.e.  of getting front-line project teams and managers to believe in the utility of the proposed processes and to make them want to use them. It stands to reason that people will use processes only if they  believe they will  help. So the first action should be to elicit buy-in from people who will be required to use the proposed processes. The most obvious way to do this is by seeking their input in formulating the processes.

Most often though, processes are “sold” to employees in a very superficial way (see this post for a case in point).  Worse still, many times  organizations do not even bother getting buy-in: the powers that be simply mandate the process with employees having little or no say in how processes are to be used or implemented. This approach is doomed to fail because – as I have discussed in this post – standards, methodologies and best practices capture only superficial aspects of processes.  The details required to make processes work can be provided only by project managers and others who work at the coal-face of projects. Consequently employee buy-in shouldn’t be an afterthought, it should be the centerpiece of any strategy to implement a methodology. Buy-in and input are essential to gaining employee commitment, and employee commitment is absolutely essential for the processes to take root.

..and so to sum up

Project management processes are necessary for project success, but they are far from sufficient. Employee intentions and buy-in are critical factors that are often overlooked when implementing these.  As a first step to addressing this, project management processes should be considered and implemented in a way that makes sense to those who work on projects.  Those who miss this point are setting themselves up for failure.

Written by K

September 1, 2011 at 3:12 am

The ERP paradox

with 5 comments

“…strategic alignment flounders in never-ending tactics and compromises…”  Ole Hanseth et. al.  in The Control Devolution: ERP and the Side-effects of Globalization (The Database for advances in information systems, Vol. 32, pp. 34-36)

Introduction

Organisations implement Enterprise Resource Planning (ERP) systems for a number of reasons.  Some of the more  common ones are:

  • To gain control over processes within the organisation.
  • To make these processes more efficient.
  • To reduce the portfolio of applications that the IS department has to manage.

Yet, the end result of ERP implementations is often the opposite: less control, and efficiency; and even though the number of applications may be reduced, this advantage is often offset by the cost and effort of maintaining ERP systems. In this post I explore this paradox, drawing from the paper from which the quote at the start of this post was taken. In essence, the paper discusses- via a case study – how the implementation of an ERP system can actually increase organisational drift and reduce efficiency.

Globalisation and its effect on IT strategy

Those who have lived through an ERP implementation would be well aware of the some of the difficulties associated with these. This is no longer news: there is a fair bit of research done on the problems and pitfalls of ERP system implementations (see this paper, for example). The question, however, is why ERP implementations run into problems. To answer this, the authors of the paper turn to the notion of globalisation and how ERP systems can be seen as a reaction to it.

Globalisation is essentially the interaction and integration between people of different cultures across geographical boundaries, facilitated by communication, trade and technology.  The increasing number of corporations with a global presence is one of the manifestations of globalisation. For such organisations, ERP systems are seen as a means to facilitate globalisation and control it.

There are four strategies that an organisation can choose from when establishing a global presence. These are:

  • Multinational: Where individual subsidiaries are operated autonomously.  
  • International: Where work practices from the parent company diffuse through the subsidiaries (in a non-formal way).  
  • Global: Where local business activities are closely controlled by the parent corporation.  
  • Transnational: This (ideal) model balances central control and local autonomy in a way that meets the needs of the corporation while taking into account the uniqueness of local conditions.

These four business strategies map to two corporate IT strategies:

  •  Autonomous: where individual subsidiaries have their own IT strategies, loosely governed by corporate.
  •   Headquarters-driven: where IT operations are tightly controlled by the parent corporation.

Neither is perfect – both have downsides that start to become evident only after a particular strategy is implemented. Given this, it is no surprise that organisations tend to cycle between the two strategies, with cycle times varying from five to ten years; a trend that corporate IT minions are all too familiar with.  Typically, though,  executive management tends to favour the centrally-driven approach since it holds the promise of higher control and reduced costs.

Another consequence of globalisation is the trend towards outsourcing IT infrastructure and services. This is particularly popular for operational IT – things like infrastructure and support. In view of this, it is no surprise that the organisation discussed in the paper chose to outsource their ERP operations to an external vendor.  Equally unsurprising, perhaps, is that the quality of service did not match expectations.

The effect of modernity

The phenomenon of modernity forms an essential part of the backdrop against which ERP systems are implemented. According to a sociological definition due to Anthony Giddens,  modernity  is “associated with (1) a certain set of attitudes towards the world, the idea of the world as open to transformation, by human intervention; (2) a complex of economic institutions, especially industrial production and a market economy; (3) a certain range of political institutions, including the nation-state and mass democracy”  

Modernity is characterised by the following three “forces” that have a direct impact on our lives:

  • The separation of space and time: This refers to the ability to coordinate activities across the world – be they global supply chains or virtual project teams. The ability to coordinate work across space and time is made possible by technology. The important consequence of this ability, relevant to ERP systems,  is that it makes it possible for organisations to increase their level of surveillance and control of key business processes. 
  • The development of disembedding mechanisms: As I have discussed at length in this post, organisations often “import” procedures that have worked well in organisations. The assumption underlying this practice is that the procedures can be lifted out of their original context and implemented in another one without change. This, in turn, tacitly assumes that those responsible for implementing the procedure in the new context understand the underlying cause-effect relationships completely. This world-view, where organisational processes and procedures are elevated to the status of universal “best practices”  is  an example of a disembedding mechanism at work. Disembedding mechanisms are essentially processes via which certain facts are abstracted from their context and ascribed a universal meaning. 
  • The reflexivity of knowledge and practice: Reflexive phenomena are those for which cause-effect relationships are bi-directional – i.e. causes determine effects which in turn modify the causes. Such phenomena are unstable in the sense that they are continually evolving – in potentially unpredictable ways. Organisational practices (which are based on organisational knowledge) are reflexive in the sense that they are continually modified in the light of their results or effects.  This conflicts with the main rationale for ERP systems, which is to rationalise and automate organisational processes and procedures (most often in a completely inflexible manner) .

Implications for organisations

One of the main implications of globalisation and modernity is that the world is now more interconnected than ever before. This is illustrated by the global repercussions of the financial crises that have occurred in recent times. For globalised organisations this manifests itself in not-so-obvious dependencies – both on events internal to the organisation and those that take place in its business, political and social environment. The important thing to note is that these events are outside of the organisation’s control. At best they can be managed as risks –i.e. events that cannot be foreseen with certainty.

A standard response to risk is to increase control. Arguably, this may well be the single most common executive-level rationale behind many ERP implementations. Yet, paradoxically, the imposition of stringent controls can lead to less control. One of the main reasons for this is that strict controls can give rise to unanticipated side effects. A good example of this is when employees learn how to game performance metrics and service level agreements.

The gap between plan and reality

The authors use a case study to illustrate how ERP implementations can be subverted by the side effects of globalisation and modernity.  The organisation they studied was Norsk Hydro a Norwegian multinational which, at that time, was undergoing an organisation-wide consolidation of its IT infrastructure and services.  Up until then, the IT landscape within the organisation was heterogeneous, with individual business units and subsidiaries free to implement whatever systems they saw fit. The decision to implement a global ERP system (SAP R/3) was a direct consequence the drive to consolidate the IT portfolio.

To reduce risk, it was decided to develop and validate a pilot project in one site and manufacturing plant.  Problems started to emerge during the pilot validation. As the authors state:

When the pilot was installed, it took three months of extensive support to make it work properly. …The validation effort identified more than 1000 “issues,” each of them requiring changes in the system.

Understandably, business managers were not impressed:

Some managers also argued that the “final” version should be based on a complete redesign of the pilot, as the latter was not structured as well as the more complex “final” version would require.

Yet, this redesign never happened. One can only speculate why – but it is a pretty good guess that cost had something to do with it.

The SAP implementation unfolded against a backdrop of a large-scale business restructuring. One of the other sub-projects in this restructuring was a business re-engineering initiative. Quite logically, this was subsumed within the SAP project. One of the main outcomes of this was the establishment of “common” organisation-wide processes to replace myriad local processes.  These common processes were to be modelled on “best practices.” Although this made sense from a management perspective, implementation was difficult because just about every local procedure had quirks that could not be shoe-horned into standardised global processes.

Side effects

The authors list a number of unintended “side effects” of the implementation. I will describe just a couple of these, referring the reader to the original paper for others.

Homogeneity to heterogeneity

Ideally, an SAP implementation is intended to provide a single “harmonised” solution across an organisation.  In practice, however, local differences and the existence of legacy systems guarantees that this ideal will be compromised. This is exactly what happened at Norsk Hydro. In the authors’ words:

…differences in business cultures and market structures in nations and regions [had to be accounted for]. In this process locals played a key role. They, in fact, took over the design process and turned SAP into an ally helping them get control over the overall change process…the SAP solution was customised for each individual site. Slowly, but irreversibly, the SAP solution had changed from one coherent common system to a complex, heterogeneous infrastructure.

Those who have lived through an ERP implementation may recognise echoes of this in their own experiences.

Side effects of integration

Although the above example illustrates the integration was perhaps not as complete as was intended, the implementation was largely successful in rationalising the organisation’s IT landscape. For one, it  replaced several legacy systems, thus (in theory) reducing costs. However, as the authors’ point out, integration means interdependence, which can create significant maintenance problems. ERP systems are notoriously hard and expensive to maintain. Norsk Hydro’s experience was no different: SAP upgrades were horrendously expensive and time consuming. As the authors state:

Typically, SAP is subject to rapid change because the huge customer base generates lots of new requirements all the time. Moreover, as its integrated nature implies, when any module is changed, the whole system has to be modified. Thus, in spite of the fact that the number of interfaces to be maintained decreases when an organization installs SAP, their complexity and change rate increase so much that the overall maintenance costs reach very high levels.

In spite of the standard solutions applied, the upgrades of the SAP code itself are also very complex and time consuming. The last upgrade (at the time of writing) enforced the SAP application to be down for 9 days! Also here there are many explanations.

For example, when all the work processes are integrated it creates a complex production lattices. Because of many errors in the software all work processes have to be tested extensively, etc.

This side effect is, in fact, an unavoidable consequence of the complexity and interconnectedness of ERP systems

Conclusion

In closing, it is appropriate to return to the themes mentioned at the start of this post. The case study discussed by the authors highlights the fact that ERP systems can have effects that are exactly opposite to the ones intended. Specifically, they can lead to less rather than more control, less efficiency and addition of complexity to the IT portfolio. Moreover, seen in a broader context, ERP systems are a microcosm of modernity: they attempt to coordinate activities at a global scale, implement disembedding mechanisms in the form of best practices,  and  are reflexive in the sense that they change organisational practices but are also changed by them.  The  interconnectedness and uncertain cause-effect relationships lead to unanticipated side effects that can completely subvert the original intent of these systems.

The authors summarise this well in the last few lines of the paper:

ERP installations in global organizations conform pretty well to the image of the modern world as a juggernaut, i.e. a runaway engine of enormous power that, collectively as human beings, we can drive to some extent but that also threatens to rush out of our control in directions we cannot foresee, crushing those who resist it

In my opinion, those thinking of committing their organisations to implementing ERP systems would do well to read this paper in addition to vendor propaganda literature.

%d bloggers like this: