Organisational surprise and its relevance to project management

Introduction

As humans, we tend to believe that events and processes will unfold in the way we expect them to.  Unfortunately things rarely go according to our expectations and we end up being, well… surprised! But it isn’t just individuals, entire organisations can be caught unawares by events. In this post I draw on a paper entitled, Clues, Cues and Complexity: Unpacking the Concept of Organisational Surprise, to elaborate on the different ways in which surprises can crop up in organisational settings and in particular, in projects.  My focus is on the latter because the temporary and one-off nature of projects seems to render them particularly prone to surprises.

Organizational surprise

The authors define organizational surprise as, “any event that happens unexpectedly or any expected event that takes an unexpected shape.”  One does not have to look far to see examples of organizational surprises. It is almost certain that any experienced project manager would have encountered a number of surprises in the course of his or her professional work. These could be unforeseen events (such as a project sponsor leaving for greener pastures) or unexpected twists and turns in what ought to have been a straightforward process (such as a software upgrade turning out to be more complicated than expected).

The ubiquity of organizational surprises begs the question as to why we are still “surprised by surprises.” The short answer is that this happens because we tend to overestimate our ability to control the future.  The authors suggest that we would be better served by regarding surprise as an inherent property of all open systems  (which includes entities such as organisations and projects). After living through the consequences of many over-optimistic managerial actions (both, my own and those of others), I would have to agree.

Classifying surprise – a typology of surprises

Nothing I have said so far would be surprising to readers: indeed, project management is largely about managing uncertainty…and all project managers know that. What might be new, however, is a classification of surprises proposed by the authors of the paper. I have hinted at the classification in the previous section where I gave one example each of a surprising event and a surprising process. It is now time to generalize this to a typology of surprises.

The authors classify surprises along two dimensions:

• Issue – An issue can occur in one of two ways:
  • When something unusual happens.
  • When something that usually happens does not happen.

  It is important to note that although the term issue has negative connotations in project management parlance, it is used here in a neutral sense – i.e. issues can be either positive or negative.

• Process – a process is a chain of related events that unfolds in an unusual manner. For example, when an ATM cash withdrawal fails because the machine does not have enough money left to process the withdrawal.

From the perspective of surprise, issues and processes can be either expected or unexpected. This gives us the four categories illustrated in Fig 1.

Figure 1: A typology of surprises

Let’s take a tour of the four categories

1. Expected issue and process: This is the zone of predictability where one-off events tend to go as foreseen.  An example of an expected issue that was successfully dealt with through planning was the Y2K problem. Another example is provided by (successful) risk management activities that are triggered when a foreseen risk eventuates.
2. Unexpected issue, expected process:  This is where a surprising issue occurs, but the consequences follow are expected. An example this would be a chance occurrence (say, a project team member on a troubled project stumbles on a novel technique that saves development time), and this leads to the project being completed within time and budget (expected process following the event).
3. Expected issue, unexpected process:  This occurs when an expected event evolves in an unexpected way – i.e. leads to a surprising process. A common example of this in a project environment is when a  front-end project  decision unfolds in unexpected ways.   Another common example is an organizational change that has unintended consequences.
4. Unexpected issue, unexpected process: This is a situation where both the event and the processes around it are counter to  conventional wisdom. In this case, those involved need to understand, or make sense of the situation and hence the term sensemaking crisis. An example of this is when project managers fail to anticipate factors that turn out to have a major influence on the way their projects evolve.  One could argue that many high profile project failures were the result of such crises.  The Denver Baggage Handling System and the Merck Vioxx affair are good examples. In both cases, the projects failed because those responsible failed to react to certain events that changed the trajectory of the projects irrevocably.

Let’s now take a brief look at the usefulness of this classification.

Coping with surprise

Managers expend a great deal of effort in attempting to predict surprises and hence corral them into the zone of predictability (Reminder: this is the bottom left quadrant in Figure 1).  As mentioned earlier, this is difficult because organisations are open systems, and novelty is an inherent property of such systems. The main implication of this is that surprises in quadrants other than the zone of predictability cannot be foreseen.  So, instead of worrying about predicting surprises, project and program managers would do better by focusing their efforts on creating an environment that enables team members to cope with nasty surprises and take advantage of good ones.

What might such an environment look like?

This question is best approached via a related question: what are the qualities displayed by project teams that are able to cope with surprises?

Here are some essential ones that are mentioned in the paper:

1. Vigilance / problem sensing– a deep awareness of the project environment, with the ability to sense any changes in it.
2. Resilience – the capacity to adjust to changes in the internal and external environment.
3. Ability to improvise– the ability to respond to the unexpected by devising appropriate courses of action under pressure

The striking thing about these qualities is that they are impossible to create or engender by management fiat: teams will not improvise unless they feel empowered to, nor will they be resilient or vigilant unless they are intrinsically motivated to be so.These characteristics are emergent in the sense that they will be displayed spontaneously by teams that are in a frame of mind that comes out of being in  the right environment.

The primary task of a project manager, or any manager for that matter, is to create such a  holding environment that provides psychological safety to the team and encourages rational (or open) dialogue  between all project stakeholders (yes, including project sponsors). I won’t elaborate on these terms here  since they are dealt with at length in the articles that I have provided links to in the previous sentence.

Different types of surprise require different approaches

Having the right environment is the key to dealing with all four kinds of surprises. However, even within such an environment, it is important to note that different types of surprises have to be tackled in different ways. In particular:

1. Predictable surprises are best tackled through traditional management approaches (as discussed in PMBOK, for example). In view of the prevalence of such approaches, I should perhaps emphasise again that they work only for a small subset of all possible surprises (only those that lie in the first quadrant)
2. Surprising events and surprising processes are best dealt with by the people who are at the coalface of the problem since they are intimately familiar with the context and history of the problem.
3. Sensemaking crises are best handled by collaborative problem solving approaches such as Dialogue Mapping.

The above yet again underscores the importance of the creating the right environment, for although predictable surprises can be tackled through traditional approaches to project management, those that lie in the other three quadrants cannot.

Conclusion

A fact of organizational life is that project managers are often caught unawares by unforeseen events and their dynamics. In this post, I have summarized a typology of organizational surprises and have elaborated on its relevance to project management.  I have also briefly discussed the ways in which different types of surprises can be tackled, emphasising that the key to tackling surprise lies in creating an environment that provides psychological safety and encourages open dialogue.

In closing, I reiterate that projects and organisations are open systems, and surprises are characteristic of such systems. The biggest surprise, therefore, is that we are continually surprised by some of the events and processes that occur within them

Routines and reality – on the gap between the design and use of enterprise systems

Introduction

Enterprise IT systems that are built or customised according to specifications sometimes fail because of lack of adoption by end-users. Typically this is treated as a problem of managing change, and is dealt with in the standard way: by involving key stakeholders in system design, keeping them informed of progress, generating and maintaining enthusiasm for the new system and, most important, spending enough time and effort in comprehensive training programmes.

Yet these efforts are sometimes in vain and the question, quite naturally, is: “Why?”  In this post I offer an answer, drawing on a paper by Brian Pentland and Martha Feldman entitled, Designing Routines: On the Folly of Designing Artifacts While Hoping for Patterns of Action.

Setting the context

From a functional point of view, enterprise software embodies organisational routines – i.e. sequences of actions that have well-defined objectives, and are typically performed by business users in specific contexts.   Pentland and Feldman argue that although enterprise IT systems tend to treat organisational routines as well defined processes (i.e. as objects), it is far from obvious that they are so. Indeed, the failure of business process “design” or “redesign” initiatives can often be traced  back to a fundamental misunderstanding of what organisational routines are. This is a point that many system architects, designers and even managers / executives would do well to understand.

As Pentland and Feldman state:

… the frequent disconnect between [system or design] goals and results arises, at least in part, because people design artifacts when they want patterns of action…we believe that designing things while hoping for patterns of action is a mistake. The problem begins with a failure to understand the nature of organizational routines, which are the foundation of any work process that involves coordination among multiple actors… even today, organizational routines are widely misunderstood as rigid, mundane, mindless, and explicitly stored somewhere. With these mistaken assumptions as a starting point, designing new work routines would seem to be a simple matter of creating new checklists, rules, procedures, and software. Once in place, these material artifacts will determine patterns of action: software will be used, checklists will get checked, and rules will be followed.

The fundamental misunderstanding is that design artifacts, checklists, and rules and procedures encoded in software are mistaken for the routine instead of being seen for what they are: idealised representations of the routine. Many software projects fail  because designers do not understand this.  The authors describe a case study that highlights this point and then  draw some general inferences from it.  I describe the case study in the next section and then look into what can be learnt from it.

The situation

The case study deals with a packaged software implementation at a university. The university had two outreach programs that were quite similar, but were administered independently by two different departments using separate IT systems. Changes in the university IT infrastructure meant that one of the departments would lose the mainframe that hosted their system.

An evaluation was performed, and the general consensus was that it would be best for the department losing the mainframe to start using the system that the other department was using.  However, this was not feasible because the system used by the other department was licensed only for a single user. It was therefore decided to upgrade to a groupware version of the product. Since the proposed system was intended to integrate the outreach-related work of the two departments, this also presented an opportunity to integrate some of the work processes of the two departments.

A project team was set-up, drawing on expertise from both departments. Requirements were gathered and a design was prepared based on the requirements. The system was customised and tested as per the design, and data from the two departments was imported from the old systems into the new one. Further, the project team knew the importance of support and training: additional support was organised as were training sessions for all users.

But just as everything seemed set to go, things started to unravel. In the author’s words:

As the launch date grew nearer, obstacles emerged. The implementation met resistance and stalled. After dragging their feet for weeks, [the department losing the mainframe] eventually moved their data from the mainframe to a stand-alone PC and avoided [the new system] entirely. The [other department] eventually moved over [to the new system], but used only a subset of the features. Neither group utilized any of the functionality for accounting or reporting, relying instead on their familiar routines (using standalone spreadsheets) for these parts of the work. The carefully designed vision of unified accounting and reporting did not materialize.

People in the department that was losing the mainframe worked around the new system by migrating their data to a standalone PC and using that to run their own system. People in the other department did migrate to the new system, but used only a small subset of its features.

All things considered, the project was a failure.

So what went wrong?

The authors emphasise that the software was more than capable of meeting the needs of both departments, so technical or even functional failure can be ruled out as a reason for non-adoption. On speaking with users, they found that the main objections to the system had to with the work patterns that were imposed by it. Specifically, people objected to having to give up control over their work practices and their identities (as members of specific) departments. There was a yawning gap between the technical design of the system and the work processes as they were understood and carried out by people in the two departments.

The important thing to note here is that people found ways to work around the system despite the fact that the system actually worked as per the requirements. The system failed despite being a technical success. This is a point that those who subscribe to the mainstream school of enterprise system design and architecture would do well to take note of.

Dead and live routines

The authors go further: to understand resistance to change, they invoke the notion of dead and live routines. Dead routines are those that have been represented in technical artifacts such as documentation, flowcharts, software etc. whereas live routines are those that are actually executed by people. The point is, live routines are literally living objects – they evolve in idiosyncratic ways because people inevitably tweak them, sometimes even without being conscious that they are making changes. As a consequence live routines often generate new patterns of action.

The crux of the problem is that software is capable of representing dead routines, not live ones.

Complementary aspects of routines

Routines are composed of people’s understandings of the routines (the authors call this the ostensive aspect) and the way in which they are actually performed or carried out (the authors call this the performative aspect). The two aspects, the routine as understood and the routine as performed, complement each other:  new actions will modify one’s understanding of a routine, and the new understanding in turn modifies future actions.

Now, the interesting thing is that no artifact can represent all aspects of a routine – something is always left out. Even though certain artifacts such as written procedures may be intended to change both the ostensive and performative aspects of a routine (as they usually are), there is no guarantee that they will actually influence behaviour.  Managers who encounter resistance to change will have had first-hand experience of this: it is near impossible to force change upon a group that does not want it.

This leads us to another set of complementary aspects of routines. On the one hand, technology puts in place structures that enable certain actions while constraining others – this is the standard technological or material view of processes as represented in systems. In contrast, according to the social or agency view, people are free to use technology in whatever way they like and sometimes even not use it at all. In the case study, the former view was the one that designers took whereas users focused on the latter one.

The main point to take away from the foregoing discussion is that designers/ engineers and users often have very different perspectives on processes.  Software that is based on a designer/engineer perspective alone will invariably end up causing problems for end users.

Routines and reality

Traditional systems design proceeds according to the following steps:

• Gather requirements
• Analyse and encode them in rules
• Implement rules in a software system
• Provide people with incentives and training to follow the rules
• Rollout the system

Those responsible for the implementation of the system described in the case study followed the steps faithfully, yet the system failed because one department didn’t use it at all and the other used it selectively.  The foregoing discussion tells us that the problem arises from confusing artifacts with actions, or as I like to put it, routines with reality.

As the authors write:

This failure to understand the difference between artifacts and actions is not new…the failure to distinguish the social (or human) from the technical is a “category mistake.” Mistakenly treating people like machines (rule-following automata) results in a wide range of undesirable, but not entirely surprising, outcomes. This category difference has been amply demonstrated for individuals, but it applies equally (if not more so) to organizational routines. This is because members’ embodied and cognitive understandings are often diverse, multiple, and conflicting.

The failure to understand the difference between routines and reality is not new, but it appears that the message is yet to get through: organisations continue to implement (new routines via) enterprise systems without putting in the effort to understand ground-level reality.

Some tips for designers

The problem of reconciling user and designer perspectives is not an easy one. Nevertheless, the authors describe an approach that may prove useful to some. The first concept is that of a narrative network – a collection of functional events that are related by the sequence in which they occur. A functional event is one in which two actants (or objects that participate in a routine) are linked by an action. The actants here could be human or non-human and the action represents a step in the process. Examples of functional events would be:

1. The sales rep calls on the customer.
2. The customer orders a product.
3. The product is shipped to the customer.

The important thing to note is that the functional event does not specify how the process is carried out – that can be done in a myriad different ways. For example, the sales rep may meet the customer face-to-face or she may just make a phone call. The event only specifies the basic intent of the action, not the detail of how it is to be carried out. This leaves open the possibility for improvisation if the situation calls for it. More importantly it places the responsibility for making that decision on the person responsible for the carrying out the action (in the case of a human actant).

Then it is useful to classify a functional event based on the type of actants participating in the event – human or (non-human) artifact. There are four possibilities as shown in Figure 1.

Figure 1: Human-artifact grid

The interesting thing to note is that from the perspective of system design, an artifact-artifact event is one over which the designer has the strongest control (actions involving only artifacts can be automated) whereas a designer’s influence is the weakest in human-human events (humans rarely, if ever, do exactly as they are told!). Decomposing a routine in this way can thus serve to focus designer efforts in areas where they are likely to pay off and, more importantly, help them understand where they are likely to get some push back.

Finally, there are some general points to consider when designing a system. These include:

1. Reinforce the patterns of the routine:  Much as practise makes an orchestra perform better, organisations have to invest in practicing the routines. Practice connects the abstract routine to the performed one.
2. Consider every stakeholder group’s point of view: This is a point I have elaborated at length in my posts on dialogue mapping so I’ll say no more about it here.
3. Understand the relationship between abstract patterns and actual performances:  This involves understanding whether there are different ways to achieve the same goal. Also consider whether it is important to enforce a specific set of actions leading up to the goal, or whether the actions matter less than the goal.
4. Encourage people to follow patterns of action that are important:  This involves creating incentives to encourage people to carry out certain important actions in specified ways. It is not enough to design and implement a particular pattern within a routine. One has to make it worthwhile for people to follow it.
5.  Make it possible for users to become designers: Most routines involve decisions points where actors have to choose between alternate, but fixed, courses of action. At such points human actors have little room to improvise. Instead, it may be more fruitful to replace decision points with design points, where actors improvise their actions.
6. Lock in actions that matter: Notwithstanding the previous point, there are certain actions that absolutely must be executed as designed. It is as important to ensure that these are executed as they should be as it is to allow people to improvise in other situations.
7. Keepan open mind and invite engagement:  Perhaps the most important point is to continually engage with people who work the routine and to be open to their input as to what’s working and what isn’t.

Most of the above points are not new; I’d even go so far as to say they are obvious. Nevertheless they are worth restating because they are often unheeded.

Conclusion

A paradox of enterprise systems is that they can fail even if built according to specification. When this happens it is usually because designers fail to appreciate the flexibility of the business process(es) in question. As the authors state, “…even today, organizational routines are widely misunderstood as rigid, mundane, mindless, and explicitly stored somewhere.”  This is particularly true for processes that involve humans.  As the authors put it, when automating processes that involve humans it is a mistake to design  software artefacts while hoping for (fixed) patterns of action. The tragedy is that this is exactly how enterprise systems are often designed.

Patterns of miscommunication in organisations

Introduction

The hierarchical structure of many workplaces tends to constrain or even stifle open exchange of ideas and information. This is particularly apparent in communication between employees who are at different levels in a hierarchy: people are generally reluctant to speak their minds in front of their managers, even when assured that it is perfectly OK to do so.   There is good reason for this: managers often “talk the talk” about being open to other points of view but contradict their words subsequently (see my article entitled, the paradox of the learning organization, for an example of this).

In this post I draw on this paper by Max Visser to describe some of the tactics or patterns of miscommunication which managers employ to sideline, devalue or even completely dismiss employee viewpoints.

Background

Those who toil in the lower echelons of an organisation’s hierarchy can easily sense the gap between managerial talk and intent.  One setting in which this gap becomes particularly evident is in group meetings, where a manager’s words may say, “speak freely” but his body language or responses may append an unspoken “be aware of the consequences”   clause.

As I have discussed in this post, communication is just as much about context (e.g. manager-subordinate relationship within an organisational setting) as it is about content.  This point of view is central to the interactional view of communication that originated in the work of Gregory Bateson and Paul Watzlawick.  According to the interactional view, communication operates at two levels: the spoken or written meaning (content) and the situation/relationship (context). Among other things, this view focuses on the ways in which the content of a message – such as “speak freely” – may be rendered ambiguous by signals that appear to contradict it.  In the remainder of this post we’ll look at a few ways in which managers do this via verbal communication. We’ll also take a brief look at the different ways in which employees respond to such behaviour.

Patterns of miscommunication

The best way to describe these patterns is through an example. Consider the following situation:

An employee presents a business case for a new CRM system to his manager. In the presentation, the employee describes the rationale for implementing a new system and then evaluates a few products based on agreed financial, technical and other criteria. Finally, he recommends a particular product, System X, based on the evaluation and then seeks feedback from his manager.

The manager, who does not want to commit to a course of action may choose one of the following strategies to devalue the employee’s work:

Tangentialisation

In this case the manager makes a statement that acknowledges the employee’s message but ignores its  content and intent by saying something like:

“So how long have you been working on this?”

By going off on a tangent, the manager  avoids giving a relevant response.

Disqualification

There are four types of disqualification

Evasion

This occurs when a manager avoids giving a response by changing the topic.  For example, the manager might glance at his watch and saying:

“Oh is that the time?  I have to go, I’m late for a meeting with my boss.”

The difference between tangentialisation and evasion is that in the latter, the manager does not even acknowledge the message.

Sleight of hand

Here the manager appears to acknowledge the message,  but then switches the topic. An example of this would be a response along the lines of:

“Yes, you enough data for a Phd thesis here [laughs]. I think we’re drowning in data.“

The point here is that the manager initiates a discussion about a side issue – the volume of  information presented in the business case rather  than its relevance or veracity. Moreover this is done in an apparently light-hearted, yet somewhat demeaning way. Thus although the manager avoids giving direct feedback, he still makes it clear  he does not think that the employee’s work is up to scratch.

Status disqualification

Here the manager switches the focus from the message to the messenger. Usually status disqualification is accompanied by insinuations regarding the messenger’s competence. A typical example of this would be a comment like:

“It’s clear you have not done these kinds of presentations before!”

Without saying it explicitly, the manager is implying that the employee has not done a good job and therefore no further discussion is necessary.

Redundant question

This is where the manager lobs the ball back in the employee’s court by asking a question that implicitly challenges the employee’s conclusions. An example would be:

“[smiles knowingly] I see, but does your data justify your choice of System X?”

Such a question signals the manager is not convinced, but without explicit disagreement. The onus is now on the employee to justify his conclusions.

Mystification

Here the manager changes the context of the discussion altogether by saying something like:

“Let me tell you something about CRM systems.”

Here the manager changes the frame of the discussion – it is now about educating the employee rather than evaluating the product. Of course, in doing so he also insinuates that the employee’s analysis is not worthy of a response.

Employee responses to managerial miscommunication

When faced with any of the above tactics, the employee can respond in one of the following ways:

1. Meta-communication: Here the employee understands the manager’s tactics and attempts to point out the inconsistency and double speak in the manager’s response.  This is a risky course of action because the manager may view it as a direct challenge to his or her authority. However, if done right, the manager may actually become aware of the incongruence of his/her response and change behaviour accordingly.
2. Evasion: Here the employee withdraws from the conversation by ignoring the manager’s message altogether.  One way to do this is to offer no response at all, but this might not be possible as the manager may well insist on a response.
3. Acceptance: In this case the employee accepts the content of the manager’s response, but ignores the non-verbal signals (derogatory tone, looking at watch etc.).  In doing so, the employee effectively accepts the manager’s criticisms.
4. Countering: Here the employee counters the manager’s message by  using one of the tactics of the previous section. This generally leads to a verbal escalation as the manager will view such a response as a direct challenge to his authority and thus respond in kind.

Because of the nature of the manager-employee relationship and the fear of challenging authority, I would hazard a guess that majority of employees would respond by acceptance or (more infrequently) by evasion. In an ideal organisation, of course, they would respond by meta-communicating.

Conclusion

In this post I have described some common patterns of miscommunication between managers and the managed in organisation-land.  The common element in all the patterns is that the manager acknowledges the message at one level but responds in such a way as to leave the employee confused about how the response should be interpreted. In effect, the miscommunicating manager avoids giving a response.

The interactional view of communication tells us that context and relationship are more important than the content of a message because what is not said is often more significant than what is. The patterns listed above make this amply clear:  managers who miscommunicate are asserting their positional authority rather than saying anything of substance or value.