Eight to Late

Organisational surprise and its relevance to project management

with 4 comments

Introduction

As humans, we tend to believe that events and processes will unfold in the way we expect them to.  Unfortunately things rarely go according to our expectations and we end up being, well… surprised! But it isn’t just individuals, entire organisations can be caught unawares by events. In this post I draw on a paper entitled, Clues, Cues and Complexity: Unpacking the Concept of Organisational Surprise, to elaborate on the different ways in which surprises can crop up in organisational settings and in particular, in projects.  My focus is on the latter because the temporary and one-off nature of projects seems to render them particularly prone to surprises.

Organizational surprise

The authors define organizational surprise as, “any event that happens unexpectedly or any expected event that takes an unexpected shape.”  One does not have to look far to see examples of organizational surprises. It is almost certain that any experienced project manager would have encountered a number of surprises in the course of his or her professional work. These could be unforeseen events (such as a project sponsor leaving for greener pastures) or unexpected twists and turns in what ought to have been a straightforward process (such as a software upgrade turning out to be more complicated than expected).

The ubiquity of organizational surprises begs the question as to why we are still “surprised by surprises.” The short answer is that this happens because we tend to overestimate our ability to control the future.  The authors suggest that we would be better served by regarding surprise as an inherent property of all open systems  (which includes entities such as organisations and projects). After living through the consequences of many over-optimistic managerial actions (both, my own and those of others), I would have to agree.

Classifying surprise – a typology of surprises

Nothing I have said so far would be surprising to readers: indeed, project management is largely about managing uncertainty…and all project managers know that. What might be new, however, is a classification of surprises proposed by the authors of the paper. I have hinted at the classification in the previous section where I gave one example each of a surprising event and a surprising process. It is now time to generalize this to a typology of surprises.

The authors classify surprises along two dimensions:

  • Issue – An issue can occur in one of two ways:
    • When something unusual happens.
    • When something that usually happens does not happen.

    It is important to note that although the term issue has negative connotations in project management parlance, it is used here in a neutral sense – i.e. issues can be either positive or negative.

  • Process – a process is a chain of related events that unfolds in an unusual manner. For example, when an ATM cash withdrawal fails because the machine does not have enough money left to process the withdrawal.

From the perspective of surprise, issues and processes can be either expected or unexpected. This gives us the four categories illustrated in Fig 1.

Figure 1: A typology of surprises

Figure 1: A typology of surprises

Let’s take a tour of the four categories

  1. Expected issue and process: This is the zone of predictability where one-off events tend to go as foreseen.  An example of an expected issue that was successfully dealt with through planning was the Y2K problem. Another example is provided by (successful) risk management activities that are triggered when a foreseen risk eventuates.
  2. Unexpected issue, expected process:  This is where a surprising issue occurs, but the consequences follow are expected. An example this would be a chance occurrence (say, a project team member on a troubled project stumbles on a novel technique that saves development time), and this leads to the project being completed within time and budget (expected process following the event).
  3. Expected issue, unexpected process:  This occurs when an expected event evolves in an unexpected way – i.e. leads to a surprising process. A common example of this in a project environment is when a  front-end project  decision unfolds in unexpected ways.   Another common example is an organizational change that has unintended consequences.
  4. Unexpected issue, unexpected process: This is a situation where both the event and the processes around it are counter to  conventional wisdom. In this case, those involved need to understand, or make sense of the situation and hence the term sensemaking crisis. An example of this is when project managers fail to anticipate factors that turn out to have a major influence on the way their projects evolve.  One could argue that many high profile project failures were the result of such crises.  The Denver Baggage Handling System and the Merck Vioxx affair are good examples. In both cases, the projects failed because those responsible failed to react to certain events that changed the trajectory of the projects irrevocably.

Let’s now take a brief look at the usefulness of this classification.

Coping with surprise

Managers expend a great deal of effort in attempting to predict surprises and hence corral them into the zone of predictability (Reminder: this is the bottom left quadrant in Figure 1).  As mentioned earlier, this is difficult because organisations are open systems, and novelty is an inherent property of such systems. The main implication of this is that surprises in quadrants other than the zone of predictability cannot be foreseen.  So, instead of worrying about predicting surprises, project and program managers would do better by focusing their efforts on creating an environment that enables team members to cope with nasty surprises and take advantage of good ones.

What might such an environment look like?

This question is best approached via a related question: what are the qualities displayed by project teams that are able to cope with surprises?

Here are some essential ones that are mentioned in the paper:

  1. Vigilance / problem sensing– a deep awareness of the project environment, with the ability to sense any changes in it.
  2. Resilience – the capacity to adjust to changes in the internal and external environment.
  3. Ability to improvise– the ability to respond to the unexpected by devising appropriate courses of action under pressure

The striking thing about these qualities is that they are impossible to create or engender by management fiat: teams will not improvise unless they feel empowered to, nor will they be resilient or vigilant unless they are intrinsically motivated to be so.These characteristics are emergent in the sense that they will be displayed spontaneously by teams that are in a frame of mind that comes out of being in  the right environment.

The primary task of a project manager, or any manager for that matter, is to create such a  holding environment that provides psychological safety to the team and encourages rational (or open) dialogue  between all project stakeholders (yes, including project sponsors). I won’t elaborate on these terms here  since they are dealt with at length in the articles that I have provided links to in the previous sentence.

Different types of surprise require different approaches

Having the right environment is the key to dealing with all four kinds of surprises. However, even within such an environment, it is important to note that different types of surprises have to be tackled in different ways. In particular:

  1. Predictable surprises are best tackled through traditional management approaches (as discussed in PMBOK, for example). In view of the prevalence of such approaches, I should perhaps emphasise again that they work only for a small subset of all possible surprises (only those that lie in the first quadrant)
  2. Surprising events and surprising processes are best dealt with by the people who are at the coalface of the problem since they are intimately familiar with the context and history of the problem.
  3. Sensemaking crises are best handled by collaborative problem solving approaches such as Dialogue Mapping.

The above yet again underscores the importance of the creating the right environment, for although predictable surprises can be tackled through traditional approaches to project management, those that lie in the other three quadrants cannot.

Conclusion

A fact of organizational life is that project managers are often caught unawares by unforeseen events and their dynamics. In this post, I have summarized a typology of organizational surprises and have elaborated on its relevance to project management.  I have also briefly discussed the ways in which different types of surprises can be tackled, emphasising that the key to tackling surprise lies in creating an environment that provides psychological safety and encourages open dialogue.

In closing, I reiterate that projects and organisations are open systems, and surprises are characteristic of such systems. The biggest surprise, therefore, is that we are continually surprised by some of the events and processes that occur within them

Written by K

November 27, 2013 at 9:38 pm

4 Responses

Subscribe to comments with RSS.

  1. This is going off on a bit of a tangent, but in regards to “vigilance and problem solving”:

    In my experience, something that happens throughout projects is that team members are often dealing with “surprises” within their area of responsibility. To the extent that they handle them well or are indeed handling those truly within their area of responsibility, it’s generally deemed to be okay that few others are aware of this.

    However there is always the question of how much of this individual heroism is useful versus escalating to the group. For example there are the proverbial cats that get out of the bag, where an individual effort to quietly deal with something simply provides time for it to fester. And at the best of times, there are the team members who are getting inexplicably burnt out, even though their assigned work load appears reasonable. On the flip side, I’ve seen micro managers who won’t accept any deviance from “the plan” and go ballistic upon finding any actions being taken by capable people to quietly deal with minor intrusions, i.e. the absence of a holding environment.

    I guess this comes down to providing team members with a manner to record what they may be dealing with on the side such that there is visibility if these things repeat or otherwise continue to impact the project. And there’s cultural aspects to this as well (organizational as well as upbringing).

    David Turnbull

    November 28, 2013 at 1:00 pm

    • Hi David,

      Thanks for reading and for making an important point.

      Indeed, individual team members do sometimes encounter surprises that are restricted to their area of responsibility – and they often deal with these by themselves. Most often, this is OK. Problems arise, however, when unanticipated dependencies show up. It is therefore generally a good idea to talk to others about such matters, if for no other reason than four eyes are better than two…or, more correctly, two brains better than one. It is easy to carried away by one’s own ideas; I know this from bitter experience because I’ve often been a victim of my own “brilliant” ideas.

      In a holding environment it would be perfectly natural – and even expected – that such things are discussed. Frankly, managers should butt out of these discussions, unless they have the technical expertise to contribute meaningfully.

      In the end the real job of a project manager is to support the team so that they can get the job done. A non-collaborative organisational culture makes this difficult, but also makes it ever more important that the PM takes his or her support role seriously.

      Thanks again for taking the time to read and comment.

      Regards,

      Kailash.

      K

      November 28, 2013 at 7:38 pm

  2. As always, excellent points. One hopes that project managers take their job seriously, and that the organization supports them in their work, including the work when it overlaps with departmental managers.

    Perhaps we need to mix in some anthropology and social psychology to project management? Or build on the networks of formal and informal communications — the grapevine — as early warning systems for surprises?

    As long as we continue to think that filled in templates represent the reality of the project, as long as we believe that thoughts are facts — and behave, make decisions from that perspective — there will be surprises. A project with too many surprises means that too many people are living and working to the narrow view of their expectations of how things (and people) ought to be, not necessarily how things are in reality.

    Project management and its components are good tools — but it is just a tool, and if I do say with tongue firmly planted in cheek — a tool with a bit of a cult status built around it. ;-) Sort of like astrology — and we know how accurate that field is. ;-)

    My learning of project management might have been out of the ordinary. As it was explained to me, project management evolved out of military strategic and tactical planning; that in order to be successful, every project manager in every business environment will need to come to terms with the reality of project management: that it’s 95 per cent people management; and finally, that project management plans are only and ever — at any given time — even as dates and deliverables are set in stone — a best guess as to what and how it might all unfold.

    Somehow, in our business world, we seem to think that once we have a plan mapped out, that the map becomes the territory — and it will unfold in precisely the way all of the filled in plan templates have described: our expectations are set out — our thoughts become facts, and how is that not fodder for surprise in any project where there are increasing layers of dependencies..? Life’s just not that way, including life inside organizations, cuz inside organizations are the messy bits we know as people.

    FS

    December 3, 2013 at 10:27 pm

    • Hi FS,

      Thanks for reading and commenting. I really like the point you make about “an early warning system for surprises”. I reckon that most surprises that organisations encounter would have been foreseen by someone within the organisation. The problem is to figure out how to surface such “weak signals” and bring them to the attention of decision-makers. Informal networks are a great way to do this.

      I suspect many such signals are brought to the attention of decision-makers, but are then ignored because they do not fit into the neat picture painted by plans and processes.

      There are some PMs/executives who maintain informal networks to keep themselves informed of things that lurk on the periphery of their projects/organisations. I fear, though, that they are a minority.

      Thanks (as always!) for your insightful remarks.

      Regards,

      K.

      K

      December 7, 2013 at 7:13 pm


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 248 other followers

%d bloggers like this: