Project complexity redux

In recent years there has been much discussion on complexity in project management (see this book or this standard, for example).  Unfortunately, the term “complexity” has been interpreted in all kinds of different ways, creating more confusion than clarity. So, despite all that’s been written and said, project management practitioners are still left wondering what is meant when the words “project” and “complex” (or its variants such as “complexity”) are strung together. This post is aimed at clarifying some of the confusion.

In the natural and social sciences, the term complexity is used to describe systems that consist of several interacting parts.  However, there’s more to complexity than just that. Complex systems  display several characteristic features including nonlinearity, continuous interactions with their environment (open systems) and complex feedback loops. Many complex systems also display emergent behaviour – i.e. behaviour that is more than a sum of the the behaviour of individual components. The collective behaviour of bird flocks is an example of emergence: individual birds follow a relatively simple set of behaviours or rules based on the movements of their neighbours, giving rise to a stable flying pattern on the scale of the entire flock. This large-scale flocking pattern is an emergent phenomenon, not easily discernable from the simple rules followed by individual birds.

Now, as I’ve mentioned in a prior post,  the term complex is used in at least two distinct senses in the project management literature. These are:

1. Projects that are difficult or complicated because they are high risk or involve a multitude of management and technical factors. I have used the term in this sense in my post entitled a short note on project complexity. This kind of complexity is relatively easy to measure, at least qualitatively if not quantitatively. 
2. Projects that are complex in the sense of complex systems as discussed above. This type of complexity is typically hard to measure, even qualitatively.

Although it is possible (and not uncommon) for projects to be complex in both senses described above, it is important to make a distinction between the two for reasons that I discuss below.

For starters,  to ensure consistency with other disciplines the adjective “complex” should (ideally) be used to refer to projects that are complex in the second sense. That brings us to the nub of the matter: how are we to know if a project is complex in this sense or not? This question remains open at this time. As yet no one has devised a metric, scale or even a definition  by which one would be able to determine whether a project is truly complex or not. Nevertheless, it is worth looking at project complexity from a couple of  perspectives in order to get a feel for what a definition of a complex project might look like. I do this in the next few paragraphs, with the caveat that my discussion is more vignette than panorama. Those looking for the latter may want to read the review paper published by Cooke-Davis and others in 2007. If wading through a research paper sounds uninviting, I can recommend my post entitled rumours of a new project management paradigm, which is basically a summary and review of the paper.

Moving on, in a recent paper, Jon Whitty and Harvey Maylor discuss the nature of complexity in projects. They point out that there are two elements to complexity – structural and dynamic. In the context of projects, the structural aspect consists of individual elements that make up the project and its environment – stakeholders, project tasks, etc. – and the interactions between these. In contrast, the dynamic aspect refers to changes in elements, the consequent changes in their interactions, and the further changes in the elements as a result of changing interactions. Whitty and Maylor suggest that a project be termed complex only when it displays both structural and dynamic complexity.   By this token, many projects currently classified as complex are incorrectly deemed so, because they display mainly structural, not dynamic, complexity. Put another way, they exhibit complexity in the first sense described earlier, but not the second. For example, many large projects consist of a large number of interacting elements (and hence display structural complexity), but the elements and their interactions are relatively stable (and hence not dynamically complex).  At the other end of the spectrum, one could have small projects that do display dynamic complexity. Size by itself is not a good measure of dynamic complexity.

Whitty and Maylor’s discussion suggests that structural complexity maps to complexity in the first sense  described above (i.e. complicated projects) and dynamic complexity to the second (i.e. truly complex projects). In what follows below, the term complex should be read as dynamically complex. 

Although the definition of a (dynamically) complex project is still an open question, Whitty and Maylor look into what might be meant by “managing a complex project”. In general, it is difficult (and often impossible) to predict long-term emergent behaviour in complex systems. In complex projects this lack of predictability is a consequence of complicated, dynamic interactions between various project elements such as resources, tasks etc. Note that this is lack of predictability, in principle – i.e.  it is not  a consequence of incomplete knowledge (such as incomplete scope definition) or any other controllable cause. This being the case, many of the tools used in conventional project management, which includes all Big Design Up Front methodologies, would simply not apply to complex projects. For example, how can one build a (long-term) schedule if one doesn’t know what’s going to happen? Further, it is also clear that any methodologies that use short development cycles, which includes all  iterative / incremental approaches, would have a much better chance of success here. In this context, the terminology used by the adaptive approach is particularly apt: faced with uncertainty, one doesn’t plan, one speculates. To sum up, although we don’t know what exactly a complex project is, we do know – broadly speaking – which project management approaches might work with them (and, more importantly, which won’t!).

For another perspective on project complexity, let’s turn to a paper entitled, Dilemmas in a General Theory of Planning, published by Rittel and Webber in 1973. The paper focuses on the characteristics of many social problems or issues – such as education, crime etc. – which can’t be solved (or even formulated!) in a conventional, scientific sense. The authors coin the term wicked problem to refer to such problems.  In contrast, problems that can be analysed and solved using known techniques are referred to as tame problems. In their paper, Rittel and Webber propose the following as defining characteristics of wicked problems:

1. There is no definitive formulation of a wicked problem.
2. Wicked problems have no stopping rule.
3. Solutions to wicked problems are not true-or-false, but good or bad.
4. There is no immediate and no ultimate test of a solution to a wicked problem.
5. Every solution to a wicked problem is a “one-shot operation”; because there is no opportunity to learn by trial-and-error, every attempt counts significantly.
6. Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.
7. Every wicked problem is essentially unique.
8. Every wicked problem can be considered to be a symptom of another problem.
9. The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem’s resolution.
10. The planner has no right to be wrong (planners are liable for the consequences of the actions they generate).

Although these characteristics are qualitative, they are relatively unambiguous – i.e.  given a problem, we can figure out whether it is wicked or not.  Looking through the list, it is tempting to draw a link between wickedness and complexity. Although I’m out on a limb here, other writers have commented on  the similarity between complex systems and wicked problems too. 

In an article  published in 2002, Mary Poppendieck dubbed a wicked project as a project that tackles a wicked problem as though it were a tame one. I would simplify the definition to: a wicked project is one that tackles a wicked problem. Then, assuming a  connection between wickedness and complexity exists, one would have the basis for a qualitative definition of a class of complex projects. I say “a class” because even if it is true that wicked problems are complex, the converse doesn’t necessarily hold – i.e. not all complex problems are wicked.

All the above – some of which is speculation –  leaves open the question of how a project is to be deemed complex or not. This, as mentioned earlier, awaits an unambiguous definition (and measure) of project complexity. We’re a long way off from that at present.  So, at least for now, one has to be careful when labelling a project “complex” because the term has a very precise meaning in the social and natural sciences.  Given this, it may be best to refer to so-called complex projects by other adjectives such as complicated, difficult, hard, intricate, convoluted or  even labyrinthine. Any synonymous term would get the point across, whilst sparing the project management community a whole lot of confusion.