Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘portfolio management’ Category

On the unintended consequences of organisational change

with 8 comments

Introduction

Change, as the cliché goes, is the only constant.  At any given time, most organisations are either planning or implementing changes of some kind.  Perhaps because of its ubiquity, the rationale and results of change are not questioned as deeply as they ought to be.  In this post I describe some unintended effects of organisational change, drawing on Barbara Czarniawska’s book, A Theory of Organizing and other sources. I also briefly discuss some ways in which these side effects can be avoided.

I’ll begin with a few words about terminology.  In this article planned changes (also referred to as reforms) are changes instituted in order to achieve specific goals. The goals of reforms are referred to as planned effects – that is, planned effects are intended results of change. As I discuss below, although planned effects may eventually be achieved, change initiatives have a host of unforeseen but significant consequences. These are referred to as unplanned, unintended or side effects.

This article is organised as follows: I’ll begin by describing some of the positive and negative side effects of change, following which I’ll discuss why side effects come about and how they can be managed.

Advantageous side effects of change

Although, the term side effect has a negative connotation, some side effects of change can actually be advantageous.  These include:

  1. Questioning of the status quo: In most organisations, processes and structures are taken for granted, rarely is the status quo questioned. Organisational change presents an opportunity to pose those “How can we do this better?” type questions that challenge the way things are done. Such questioning is unplanned in that it generally occurs spontaneously.
  2. Opportunities for reflection: This is a consequence of the previous point: questioning the status quo can cause people to reflect on how things can be done better. Again, this is an unintended consequence of a reform, not part of its planned goals. Also, it should be noted that although opportunities for reflection arise often, they are generally ignored because of time pressures.
  3. Spontaneous inventions: Finally, questioning of and reflecting on the status quo can trigger ideas for improvement.

Most people would agree that the above points are indeed Good Things that ought to be encouraged.  However, the important point is that people who are in the throes of a planned change seldom have the time or motivation to pursue these opportunities.

Harmful side effects of change

The negative side effects of planned changes are insidious because they tend to occur as a result of inaction – i.e. by not taking corrective actions to counter the detrimental effects of change. The following side effects serve to illustrate this point:

  1. The aims of reform become cast in stone: The objectives of a change initiative are formulated based on an understanding of a situation as it exists at a particular point in time. Problem is, as time evolves the original objectives maybecome irrelevant or obsolete. Yet, in many (most?) change initiatives, objectives are rarely reviewed and adjusted.
  2. The means get confused with the ends: Following from the previous point, a change initiative becomes pointless when its objectives are no longer relevant.  However, a common reaction in such situations is to continue the initiative, justifying it as a worthwhile end in itself. For example, if the benefits of, say, a restructuring initiative become moot,  the  restructuring itself  becomes the objective rather than the benefits that were supposed to flow from it.  This helps save face as the project can be declared a success once the restructuring is completed, regardless of whether or not the promised benefits are realised.
  3. Improvisations and spontaneous inventions are suppressed: As I have discussed at length in this post, planning and improvisation are complementary but contradictory aspects of organizational work. A negative aspect of planned change initiatives is that they are inimical to improvisations: those responsible for overseeing the change tend to ignore, even suppress any improvisations that arise because they are seen as getting in the way of achieving the objectives of the primary change.

Planned change initiatives are generally implemented through programs or projects.  In fact, most major projects in organisations – restructurings, enterprise system implementations etc – are aimed at implementing reforms of some kind. However, although the raison d’etre of such projects is to achieve the planned objectives, many suffer from the negative side effects mentioned above.   In her book Czarniawska states, “Planned change rarely, if ever, leads to planned effects.”  Although this claim may be a tad exaggerated, the significant proportion of large projects that fail suggests there is at least a whiff of truth about it.

In the next two sections I take a brief look at why planned changes fail and what can be done about it.

The origin of the side effects of change

Most structures and processes within organisations have a complex, path-dependent history. Among other things, they develop in ways that are unique to an organisation and are often deeply intertwined with each other.  As a result, it is impossible to be certain about the consequences of changing processes or structures – there are just too many variables and dependencies involved.

There are two related points that flow from this:

Firstly, those who plan changes need to have a good understanding of legacy: the history of the issues that the change aims to fix and those that it may create in the future. The problem is most of the people involved in planning, initiating and executing reforms have little appreciation of such issues.

Secondly, most major changes are conceived by a small number of people who hold positions of authority within organisations. These folks have a tendency to gloss over complexities, and often fail to involve those who have a detailed knowledge of the affected processes and structures. Consequently, their plans overlook dependencies and possible knock-on effects that can arise from them. This results in the negative side effects discussed in the previous section.

..and what can be done about them

Czarniawska recommends the following informal rules for successful change:

  1. Be willing to modify the objectives of the change and your path to get there as your understanding of it evolves.
  2. Implement lightweight processes, avoid bureaucratic procedures.
  3. Be open to improvisations.

This is good advice as it goes, but how exactly does one use it?

In our recently published book, The Heretic’s Guide to Best Practices, Paul Culmsee and I discuss how issues of legacy and lack of inclusiveness can be addressed.

Firstly, we suggest that apart from time, cost and scope (the classic iron triangle), project decision-makers would be well served by considering legacy as a separate variable in projects (also see this post on Paul’s blog for more on this point). More importantly, we describe techniques that can be used to surface hidden assumptions and aspects of history that could have a bearing on the project and those that might cause problems in the future.

Secondly, we discuss how one can work towards creating an environment in which a diverse group of stakeholders can air and reconcile their viewpoints. Such a discussion is a prerequisite to creating a plan that: a) considers as many viewpoints (variables) as possible and b) has the support of all stakeholders.  Without this, any implementation is bound to have side-effects because of overlooked variables and/or the actions (or non-actions) of stakeholders who do not support the plan.

Of course, inclusiveness sounds great but it can be difficult in practice, especially in large organisations. What can decision-makers do in such cases?  The answer comes from a slightly different, if rather obvious direction.

In his very illuminating book on decision-making, James March notes that organisations face messy and inconsistent environments. Given this, decisions made and implemented at lower levels have a better chance of success than those made in rarefied air of board-rooms.  Paraphrasing a statement from his book:

Since knowledge of local conditions and specialized competencies are both essential and more readily found in decentralized units, control over the details of policy implementation and adaptation of general policies to local conditions are [best] delegated to local units. From the standpoint of general management, the strategy is usually seen as one of gaining the informational and motivational advantages of using people with local involvement, [but] at the cost of accentuating problems of central coordination and control.

Indeed, most of the nasty side effects of planned change arise from over-centralisation of coordination and control.  The solution is to devolve control and decision-making authority down to the level at which the changes are to be implemented.

Conclusion

Planned change fails to achieve its goals because planners cannot foresee all the consequences of change or even know which factors may be important in determining these. Moreover, individuals will view changes through the lens of their background, biases and interests.  Since organisations consist of many individuals with different views, managing change is essentially a wicked problem.

To sum up, those who initiate large-scale changes should keep in mind the law of unintended consequencesany planned action will have consequences that are not intended, or even foreseen.  These consequences can be managed by getting a better appreciation of the factors that affect the processes and the structures to be changed.   One can gain an understanding of these factors through a consideration of legacy and/or via dialogue involving all those who work with the processes and structures that are to be changed. The simplest way to achieve both is by delegating decision making and implementation authority down to where it belongs – with the people who work at the coalface of the organisation.

The Labyrinths of Information – a book review

with 3 comments

 Introduction

Once implemented, IT systems can evolve in ways that can be quite different from their original intent and design.  One of the reasons for this is that enterprise systems are based on simplistic models that do not capture the complexities of real organisations. The gap between systems and reality is the subject of a fascinating book by Claudio Ciborra entitled, The Labyrinths of Information.  Among other things, the book presents an alternative viewpoint on systems development,  one that focuses on reasons for divergence between design and reality. It also discusses other aspects of system development that tend to be obscured by mainstream development methodologies and processes. This post is a summary and review of the book.

 Background

The standard treatment of systems development in corporate environments is based on the principles of scientific management. Yet, as Ciborra tells us,

…science-based, method-driven approaches can be misleading.  Contrary to their promise, they are deceivingly abstract and removed from practice. Everyone can experience this when he or she moves from the models to the implementation phase. The words of caution and pleas for ‘change management’ interventions that usually accompany the sophisticated methods and polished models keep reminding us of such an implementation gap. However, they offer no valid clue on how to overcome it…

Just to be clear, Ciborra offers no definitive solutions either. However, he offers  “clues on how to bridge the gap” by  looking into some of the informal techniques and approaches that people “on the ground” – users, designers, developers or managers –  use to work and cope with technology. He is not concerned with techniques or methodologies per se, but rather with how people deal with the messy day-to-day business of working with technology in organisations.

The book is organised as a collection of essays based on Ciborra’s research papers spanning a couple of decades – from the mid 1980s until a few years prior to his death in 2005. I discuss each of the chapters in order below, providing links to the original papers where I could find them.

 The divergence between models and reality

Most of the tools and techniques used in systems evaluation, design and development are based on simplified models of organisational reality. However,  organisations do not function according to organograms, data flow diagrams or entity-relationship models. Models used by systems professionals abstract away much of the messiness of real-life. The methods that come out of such simplifications cannot deal with the complexities of a real organisation.  As Ciborra states, “…concern with method is one of the key aspects of our discipline and possibly the true origin of its crisis…

Indeed, as any systems professional will attest to,  unforeseen occurrences and situations  inevitably encountered in real life  are what cause the biggest headaches in the implementation and acceptance of systems. Those on the ground deal with such exceptions by creative but essentially ad-hoc  approaches. Much of the book is a case-study based discussion of such improvised approaches to systems development.

 Making (do) with what is at hand

Ciborra argues that successful systems are invariably imitated by competitors, so any competitive advantage offered by such systems is, at best, limited. A similar argument holds for standards and best practices – they  promote uniformity rather than distinction. Given this, organisations should strive towards practices that cannot be copied. They should work towards inimitability.

In art, bricolage refers to a process of creating a work from whatever is at hand. Among other things it involves tinkering, improvising and generally making do with what is available. Ciborra argues that many textbook cases of strategic systems in fact evolved through bricolage, tinkering and serendipity, rather than plan. Some of the cases he discusses include Sabre Reservation System developed by American Airlines, and the development of Email (as part of the ARPANET project). Moreover, although the Sabre System afforded American Airlines a competitive advantage for a while, it soon became a part of the travel reservation infrastructure thereby becoming an operational necessity rather than an advantage. This is much the same point that Nicholas Carr made in his article, IT Doesn’t Matter.

The question that you may be asking at this point is: “All this is well and good, but does Ciborra have any solutions to offer?” Well, that’s the problem: Ciborra tells us that bricolage and improvisation ought to be encouraged, but offers little advice on how this can be done. For example, he tells  us to “Value bricolage strategically”, “Design tinkering” and “Establish systematic serendipity”  – sounds great in theory, but what does it really mean? It  is platitudinous advice that is hard to action.

Nevertheless his main point is a good one: that managers should encourage informal, creative practices instead of clamping down on them. This advice has not generally been heeded. Indeed, corporate IS practices have gone the other wa, down the road of standardisation and best practices. Ciborra tells us in no uncertain terms  that this is not a good thing.

 The enframing effect of technology

This part is, in my opinion, the most difficult chapter in the book. It is based on a paper by Ciborra and Hanseth entitled, From tool to Gestell: Agendas for managing the information infrastructure.   In German the term Gestell means shelf or rack.  The philosopher Martin Heidegger used the term to describe the way  in which technology frames the way we view (or “organise”)  the world.   Ciborra highlights the way in which existing infrastructure affects the success of  businesses processes and practices.  Ciborra emphasises that technology-based enterprise initiatives are doomed to fail unless they pay due attention to:

  1. Existing or installed infrastructure.
  2. Local needs and concerns.

Instead of attempting to oust old technology, system designers and implementers need to co-opt or cultivate the installed base (and the user community) if they are to succeed at all.  In this sense installed infrastructure is an actor (like an individual) with its own interest and agenda. It provides a context for the way people think and also influences future development.

The notion of Gestell thus reminds us of how existing technology influences and limits the way we think. To get around this, Ciborra suggests that we should:

  1. Be aware of technology and standards, but not be captive to them.
  2. Think imaginatively, but pay attention to the installed base (existing platforms and users).
  3. Remember that top down technology initiatives rarely succeed.

The drifting of information infrastructure

Ciborra uses Donald Schoen’s metaphor of the high ground and the swamp to highlight the gap between theory and practice of information systems (see this paper by Schoen, for a discussion of the metaphor). The high ground is the executive management view,where methodologies and management theories hold sway, while the swamp is the coalface where messy, day-to-day reality of organisational work unfolds. In the swamp of day-to-day work, people tend to use available technology in any way possible to solve real (and messy) problems. So, although a particular technology may have an espoused or intended aim, it may well be used in ways that are completely unforeseen by its designers.

The central point of this essay is that the full implications of a technology are often realised only after it has been implemented and used for a while. In Ciborra’s words, technology drifts – that is, it is put to uses that cannot be foreseen. Moreover, it may be never be used in ways that were intended by the designer.   Although Ciborra lists several cases that demonstrate this point, in my opinion, his blanket claim  that technology drifts is a bit over the top. Sure, in some cases, technologies may be used in unforeseen ways, but by and large they are used in ways that are intended and planned.

 The organisation as a host

Reactions to a  new technology in an organisation are generally mixed – some people may view the technology with some trepidation (because of the changes to their work routines, for instance) while others may welcome it (because of promised efficiencies, say).  In metaphorical terms, the new technology is a “guest,” whose “desires” and “intentions” aren’t fully known. Seen in this light of this metaphor, the notion of hospitality makes sense:  as Ciborra puts it, the organisation hosts the technology.

To be sure, the idea of hospitality applying to objects such as information systems will probably cause a few raised eyebrows. However it isn’t as “out there” as it sounds. Consider, for example, the following implications of the metaphor

  • Interaction between the host and guest can change both parties.
  • If the technology is perceived as unfriendly, it will be rejected (or even ejected!).
  • System development and operations methodologies are akin to cultural rituals (it is how we “deal with” the guest).
  • Technologies, like guests, stay for a while but not forever.

Ciborra’s intent  in this and most of the other essays is to make us ponder over the way we design, develop and run systems,  and possibly view what we do in a different light.

 The organisation as a platform

In this essay Ciborra looks at the way in which successful technology organisations adapt and adjust to rapidly changing environments. It is  based on his paper entitled, The Platform Organization: Recombining Strategies, Structures and Surprises, Using a case-study, he makes the point that the only way organisations can respond to rapidly evolving technology markets is to be open to recombining available resources in flexible ways: it is impossible to start from scratch; one has work with what is at hand, using it in creative ways.

Another point he makes is that the organisation of an organisation (hierarchy and structure) at any particular time is less important than how it gets there, where it’s headed and what are the obstacles in the way. To quote from the book:

 …analysing and evaluating the platform organisation at a fixed point in time is of little use: it may look like a matrix, or a functional hierarchy, and one may wonder how well its particular form fits the market for that period and what its level of efficiency really is. What should be appreciated, instead, is the whole sequence of forms adopted over time, and the speed and friction in shifting from one to the other.

However, the identification of such a trajectory can be misleading – despite after-the-fact rationalisations, management in such situations is often based on improvised actions rather than carefully laid plans.  Although this may not always be so, I suspect it is more common than managers would care to admit.

 Improvisation and mood

By now the reader would have noted that Ciborra’s focus is squarely on the unexpected occurrences in day-to-day organisational work. So it will come as no surprise that the last essay in the book deals with improvisation.

Ciborra argues that most studies on improvisation have a cognitive focus – that is, they deal with how people respond to emerging situations by “quick thinking.” In his opinion, such studies ignore the human aspect of improvised actions, the emotions and moods evoked by situations that call for improvisation. These, he suggests, can be the difference between improvised actions and panic.

As he puts it, people are not cognitive robots – their moods will determine whether they respond to a situation with indifference or interest and engagement. This human dimension of improvisation, though elusive, is the key to understanding improvisation (and indeed, any creative / innovative action)

He also discusses the relationship between improvisation and time – something I have discussed at length in an earlier post, so I’ll say no more about it here.

 A methodological postscript

In a postscript to the book, Ciborra discusses his research philosophy – the thread that links the essays in the book.. His basic contention is that methodologies and organisational models are based on after-the-fact rationalisations of real phenomena. More often than not such methods and models are idealisations that omit the messiness of real life organisations. They are abstractions, not reality. As such they can guide us, but we should be ever open to the surprises that real life may afford us.

Summarising

The essential message that Ciborra conveys is a straightforward one – that the real world is a messy place and that the simplistic models on which systems are based cannot deal with this messiness in full. Despite our best efforts there will always be stuff that “leaks out” of our plans and models. Ciborra’s book celebrates this messiness and reminds us that people matter more than systems or processes.

Written by K

September 8, 2011 at 10:41 pm

Processes and intentions: a note on cause and effect in project management

with 9 comments

Introduction

In recent years the project work-form has become an increasingly popular mode of organizing activities in many industries. As the projectization bandwagon has gained momentum there have been few questions asked about the efficacy of project management methodologies. Most practitioners take this as a given and move on to seek advice on how best to implement project management processes. Industry analysts and consultants are, of course, glad to oblige with reams of white papers, strategy papers or whatever else they choose to call them (see this paper by Gartner Research, for example).

Although purveyors of methodologies  do not claim their methods guarantee project success, they imply that there is a positive relationship between the two.  For example, the PMBOK Guide tells us that, “…the application of appropriate knowledge, processes, skills, tools and techniques can have a significant impact on project success”  (see page 4 of the 4th Edition).  This post is a brief exploration of the cause-effect relationship between the two.

Necessary but not sufficient

In a post on cause and effect in management, I discussed how  the link between high-level management actions and their claimed outcomes is tenuous. The basic reason for this is that there are several factors that can affect organizational outcomes and it is impossible to know beforehand which of these factors are significant. The large number of  factors, coupled with the fact that  controlled experimentation is impossible in organizations, makes it impossible to establish with certainty that a particular managerial action will lead to a desired outcome.

Most project managers are implicitly aware of this – they know that factors extrinsic to their projects can often spell the difference between success and failure. A mid-project change in organizational priorities is a classic example of such a factor. The effect of such factors can be accounted for using a concept of causality proposed by the philosopher Edgar Singer, and popularised by the management philosopher Russell Ackoff. In a paper entitled Systems, Messes and Interactive Planning, Ackoff had this to say about cause and effect in systems – i.e. entities that interact with each other and their environment, much like  organizations and projects do:

It will be recalled that in the Machine Age cause-effect was the central relationship in terms of which all actions and interactions were explained. At the turn of this century the distinguished American philosopher of science E.A. Singer, Jr.noted that cause-effect was used in two different senses. First… a cause is a necessary and sufficient condition for its effect. Second, it was also used when one thing was taken to be necessary but not sufficient for the other. To use Singer’s example, an acorn is necessary but not sufficient for an oak; various soil and weather conditions are also necessary. Similarly, a parent is necessary but not sufficient for his or her child. Singer referred to this second type of cause-effect as producer-product. It has also been referred to since as probabilistic or nondeterministic cause effect.

The role of intentions

A key point is that one cannot ignore the role of human intentions in management. As Sumantra Ghoshal wrote  in a classic paper:

 The basic building block in the social sciences, the elementary unit of explanation is individual action guided by some intention. In the presence of such intentionality functional [and causal] theories are suspect, except under some special and relatively rare circumstances, because there is no general law in the social sciences comparable to [say] the law of natural selection in biology

A producer-product view has room for human intentions and choices. As Ackoff stated in the his paper on systems and messes,

Singer went on to show why studies that use the producer-product relationship were compatible with, but richer than, studies that used only deterministic cause-effect. Furthermore, he showed that a theory of explanation based on producer-product permitted objective study of functional, goal-seeking and purposeful behavior. The concepts free will and choice were no longer incompatible with mechanism; hence they need no longer be exiled from science.

A producer-product view of cause and effect in project management recognizes that there will be a host of factors that affect project outcomes, many of which are beyond a manager’s ken and control. Further, and possibly more important, it acknowledges the role played by intentions of individuals who work on projects.  Let’s take a closer look at these two points.

Processes and intentions

To begin, it is worth noting that that project management lore is rife with projects that failed despite the use of formal project management processes. Worse, in many cases it appears that failure is a consequence of over-zealous adherence to methodology (see my post entitled The myth of the lonely project for a detailed example).  In such cases the failure is often attributed to causes other than the processes being used –   common reasons include lack of organizational readiness and/or improper implementation of methodology from which the processes are derived.   However, these causes  can usually be traced back to  lack of  employee buy-in,  i.e.  of getting front-line project teams and managers to believe in the utility of the proposed processes and to make them want to use them. It stands to reason that people will use processes only if they  believe they will  help. So the first action should be to elicit buy-in from people who will be required to use the proposed processes. The most obvious way to do this is by seeking their input in formulating the processes.

Most often though, processes are “sold” to employees in a very superficial way (see this post for a case in point).  Worse still, many times  organizations do not even bother getting buy-in: the powers that be simply mandate the process with employees having little or no say in how processes are to be used or implemented. This approach is doomed to fail because – as I have discussed in this post – standards, methodologies and best practices capture only superficial aspects of processes.  The details required to make processes work can be provided only by project managers and others who work at the coal-face of projects. Consequently employee buy-in shouldn’t be an afterthought, it should be the centerpiece of any strategy to implement a methodology. Buy-in and input are essential to gaining employee commitment, and employee commitment is absolutely essential for the processes to take root.

..and so to sum up

Project management processes are necessary for project success, but they are far from sufficient. Employee intentions and buy-in are critical factors that are often overlooked when implementing these.  As a first step to addressing this, project management processes should be considered and implemented in a way that makes sense to those who work on projects.  Those who miss this point are setting themselves up for failure.

Written by K

September 1, 2011 at 3:12 am

%d bloggers like this: