Eight to Late

Sensemaking and Analytics for Organizations

Archive for August 2010

Thinking outside the BOKs

with 14 comments

Introduction

Mainstream project management practices are codified in various “bodies of knowledge” (BOKs) that serve as definitive guides or standards for the profession.  This formalization of project management knowledge serves to propagate and legitimize practices regardless of their actual utility. Moreover, it also leads to a perception that project management practices are domain-independent.   In a recent paper entitled,  21st century project management = open source body of knowledge,  Jon Whitty calls for an open source approach to the development and dissemination of project management knowledge, with a view to addressing the aforementioned limitations  of the “BOK-centric” paradigm.   This post summarises the main ideas presented in the paper.

Why step outside the BOKs?

According to Whitty, many project management tools and techniques have become standard practice for reasons other than their utility. Some of these reasons include:

  1. Techniques that are easy to replicate (copy and pass on to others) have a greater chance of acceptance, regardless of whether they are useful or not. This point is discussed at length in my post on the memetic view of project management.
  2. Techniques that give project managers (and their managers) an emotional affect have a better chance of acceptance.  See my post on emotional effect of project management artefacts for more on this.

As Whitty puts it, “One might say that we have been duped or perhaps more poetically speaking romanced by some aspects of modern project management…

The main consequence of the above is that “standard” project management knowledge as embodied in books and BOKs isn’t necessarily a reflection of how project management  is (or even, should be) practiced.   Yet this is the official, authorized version of knowledge, used to train and certify project managers.

Another point that Whitty makes is that despite of the discipline’s attempt to rid itself of domain specific connotation, project management (in practice) is domain specific – there is a difference, say, between a corporate reporting system project and a health care project. It is wishful thinking to expect that a project manager will be able run a project in an unfamiliar area without first learning a bit about the domain. Consequently, Whitty writes that:

…we must acknowledge that each domain of work (e.g. health, arts, sciences, education) is capable of evolving its own methods for managing projects if given the freedom to do so. I suggest there is a moral obligation on all scholars and practitioners of project management to enable these domain specific methods to emerge…

Another point that I would add in support of Whitty’s arguments is that there is a difference between project management theory and project management practice (see this paper, for example).  Projects are seldom, if ever, run by the book; it is a fact that most project managers do “think outside the BOKs” when practicing their profession.  A body of knowledge reflecting how projects are actually run in practice would thus be very different from the prescriptions presented in books and BOKs.

Project management as a (controlled) democracy

To address the issues mentioned in the previous section, Whitty suggests democratizing project management knowledge by:

  1. A bottom-up approach to project decision making, involving all stakeholders, rather than a top-down approach implicitly advocated by books and BOKs.
  2. The recognition that project management needs to be tailored to the needs of specific domains, rather than striving for the lowest common denominator of domain independent tools and techniques.
  3. An open source approach to project management knowledge whereby anyone can contribute techniques and case studies based on their own (possibly domain specific) experiences.
  4. A culture that enables project managers to report their specific practices (and failures) without fear of reprisals.

Whitty clarifies that open source means more than free access. Among other things, it offers a means by which

  1. New techniques can be shared and evaluated by practicing project managers and academics.
  2. Domain specific project management knowledge can be compiled, tested and disseminated. This knowledge will evolve as it is supported (or refuted) through case studies and other data contributed by practitioners.

Obviously open source does not mean that anything goes; that would be a recipe for chaos and confusion. Oversight is needed to ensure that the repository is coherent and meaningful, and that contributed material meets certain quality standards. This is nothing new – most open source efforts operate this way already – Wikipedia being the closest in spirit to the one proposed by Whitty.

The role of professional organisations in an OS-BOK world

What role would professional organisations have in such an open source BOK world?  Here’s what Whitty thinks,

The implications for an open source body of knowledge for project management will be significant for the current project management professional associations who control existing BOKs, particularly the PMI who hold the intellectual property to the current Guide. The PMI and others will need to abandon their ideals for creating a professional class of project managers. They could take a leadership role in the development and on‐going coordination of the human and technological mechanisms that develop and maintain the OS‐PMBOK and validate the source evidence of the various knowledge libraries.   Their role might also be more like that of a vocational skills awarding body. As the various domain specific bodies of knowledge for project management emerge, the project management associations could have a place in developing and administering specific project management certification in conjunction with key industry bodies that represent the various domains. The project management associations will also need to establish new alliances with the academic community and lend support, cooperation, and channel industry funding towards evidence-based practice research.

Many of the activities listed by Whitty are already carried out by most professional project management organisations. As I see it, these organisations would continue to play an important role in developing the profession. However, they would no longer be the arbiters of what is legitimate and what isn’t.

Conclusion

Existing project management knowledge tells us how projects should be managed, not how they are actually managed. Seen in this light, Whitty’s proposal for an open source approach to project management makes eminent sense. Such an approach may lead to a body of knowledge that reflects the diverse ways in which project management is practiced in various (corporate and national) cultures and domains.

Written by K

August 27, 2010 at 5:37 am

Unforeseen consequences – an unexpected sequel to my previous post

with 4 comments

Introduction – a dilemma resolved?

Last week I published a post about how a friend and I used the Issue-based Information System (IBIS) notation to map out a dilemma he was facing – whether to accept or decline a job offer.  The final map  of that discussion is reproduced in Figure 1 below.

Figure 1: Final map from previous post

The map illustrates how we analysed the pros and cons of the options before him.

As I’d mentioned in the post, a couple of days after we did the map, he called to tell me that he had accepted the offer. I was pleased that it had worked out for him and was pretty sure that the dilemma was essentially resolved: he’d accepted the job, so that was that.

..or so I thought.

An unforeseen consequence

Last Saturday I happened to meet  him again.  Naturally,  I asked him how things were going – how his current employer had reacted to his resignation etc.

“You’re not going to believe this,” he said. “After he heard that I had resigned, the CEO (who is many levels above me in the org chart) asked to see me immediately.”

“Wow!”  I couldn’t help interjecting.

“…Yeah. So, I met him and we had a chat. He told me that management had me marked for a role at a sister organization, at a much higher level than I am at now. So he asked me to hold off for a day or two, until I’d heard what was on offer. When I told him I’d signed the contract already, he said that I should hear what they had to say anyway.”

Wow, indeed –   we couldn’t have anticipated this scenario… or could we?

An expert’s observations

In a comment on that post Tim van Gelder makes two important points:

  1. The options explored in the map (accept/decline) are in fact the same point – one is simply the negation of the other.  So this should be represented as a single option (either accept or decline), with the pros arguing for the represented option and the cons arguing for the unrepresented one.
  2. Options other than the obvious one (accept or decline) should have been explored.

In my response I pointed out that although representing the two options as separate points causes redundancies, it can help participants “see” arguments that may not be obvious immediately.  One is drawn to consider each of the actions separately because they are both represented as distinct options, each with their own consequences (I’ll say more about this later in this post). The downside, as Tim mentions,  is more clutter and superfluity. This is not necessarily a problem for small maps but can be an issue in larger ones.

However, it is the second point that is more relevant here. In my response to Tim’s comment I stated:

For completeness we ought to have explored options other than the two (one?) that we did, and had we more time we may have done so. That said, my mate viewed this very much as an accept/decline dilemma (precluding other options) because he had only a day to make a decision.

Clearly, in view of what happened, my argument about not having enough time is a complete cop out:  we should have made the time to explore the options because it is precisely what had come back to bite us.

Choice and consequence

In hindsight it’s all very well to say that we could have done this or should have done that. The question is: how could we have given ourselves the best possible chance to have foreseen the eventuality that occurred (and others ones that didn’t)?

One way to do this is to explore other ideas in response to the root question: what should I do? (see Figure1).  However, it can be difficult for the person(s) facing the dilemma to come up with new ideas when one option looms so much larger than all others. It is the facilitator’s job to help the group come up with options when this happens, and I had clearly failed on this count.

Sure, it can be difficult to come up with options out of the blue – especially when one is not familiar with the context and background of the problem. This highlights the importance of getting a feel for things before the discussion starts. In this case, I should have probed my friends current work situation, how he was regarded at his workplace and his motivations for moving before starting with the map. However, even had I  done so, it is moot whether we would have foreseen the particular consequence that occurred.

So,  is there any way to get participants thinking about consequences of their choices?  Remember, in IBIS one “evaluates” an  idea in terms of its pros and cons. Such an analysis  may not include consequences .

In my opinion, the best way to get folks thinking about consequences is to ask the question explicitly: “What are the consequences of this idea/option?”

Figure 2 illustrates how this might have worked in the case of my friend’s dilemma. Had we brainstormed the consequences of accepting, he may well have come up with the possibility that actually eventuated.

Figure 2: Exploring consequences (branch highlighted in yellow

The branch highlighted in yellow shows how we might have explored the consequences of accepting the job. For each consequence one could then consider how one might respond to it. The exploration of responses could be done on the same map or hived off into its own map as I’ve shown in the figure. Note that clicking on a map node in Compendium (the free software tool used to create IBIS maps)  simply opens up a new map. Such sub-maps offer a convenient way to organise complex discussions into relatively self-contained subtopics.

I emphasise that the above is largely a  reframing of pros and cons: all the listed consequences  can be viewed as  pros or cons (depending on whether the consequence is perceived as a negative or a positive one). However asking for consequences explicitly prompts participants to think in terms of what could happen, not just known pros and cons.

Of course, there is no guarantee that this process would have enabled us to foresee the situation that actually occurred. This deceptively simple dilemma is indeed wicked.

Epilogue

On Sunday I happened to re-read Rittel and Webber’s classic paper on wicked problems. In view of what had occurred, it isn’t surprising that the following lines in the paper had a particular resonance:

With wicked problems…. any solution, after being implemented, will generate waves of consequences over an extended–virtually an unbounded– period of time. Moreover, the next day’s consequences of the solution may yield utterly undesirable repercussions which outweigh the intended advantages or the advantages accomplished hitherto. In such cases, one would have been better off if the plan had never been carried out.

The full consequences cannot be appraised until the waves of repercussions have completely run out, and we have no way of tracing all the waves through all the affected lives ahead of time or within a limited time span.

I can’t hope to put it any better than that.

Written by K

August 20, 2010 at 5:50 am

To accept or to decline: mapping life’s little dilemmas using IBIS

with 8 comments

Introduction

Last weekend I met up with a friend I hadn’t seen for a while. He looked a bit preoccupied so I asked him what the matter was. It turned out he had been offered a job and was mulling over whether he should accept it or not. This dilemma is fairly representative of some of the tricky personal decisions that we are required to make at various junctures in our lives: there’s never enough information to be absolutely certain about the right course of action, and turning to others for advice isn’t particularly helpful because their priorities and motivations are likely differ from ours.

The question of whether or not to accept a job offer is a wicked problem because it has the following characteristics:

  1. It is a one-shot operation.
  2. It is unique.
  3. One is liable to face the consequences of whatever choice one makes (one has no right to be wrong)
  4. Since one can make only one choice, one has no opportunity to test the correctness of ones choice.
  5. The choice one makes (the solution) depends on which aspects of the two choices one focuses on (i.e. the solution depends on ones explanation of the issue).

Admittedly, there are some characteristics of wickedness that the job dilemma does not satisfy. For example, the problem has a definitive formulation (specified by the choices) and there are an enumerable number of choices (accept or decline). Nevertheless, since it satisfies five criteria, one can deem the problem as being wicked. No doubt, those who have grappled with this question in real life will probably agree.

Horst Rittel, the man who coined the term wicked problem also invented the Issue-based Information System (IBIS) notation to document and manage such problems (note that I don’t use the word “solve” because such problems can’t be solved in the conventional sense). The notation is very easy to learn because it has just three elements and a simple grammar – see this post for a quick introduction.  Having used IBIS to resolve work related issues before (see this post, for example), I thought it might help my friend if we could used it to visualize his problem and options. So, as we discussed the problem,   I did a pencil and paper IBIS map of the issue (I didn’t have my computer handy at the time).   In the remainder of this post, I reconstruct the map based on my recollection of the conversation.

Developing the map

All IBIS maps begin with a root question that summarises the issue to be resolved.  The dilemma faced by my friend can be summarized by the question:

What should I do (regarding the job offer)?

Questions are represented by blue-green question marks in IBIS (see Figure 1).

There are two possible responses to the question– accept or decline. Responses are referred to as ideas in IBIS, and are denoted by yellow lightbulbs.  The root question and the two ideas responding to it are shown in Figure 1.

Figure 1

An IBIS map starts with the root questions and develops towards the right as the discussion progresses.

The next step is to formulate arguments for and against each idea.  These are referred to as pros and cons in IBIS, and are denoted by a green + and red – sign respectively (See Figures 2 and 3 for examples of these).

We focused on the “accept” option first. The pros were quite easy to see as the offer was generous and the new role a step up from where he was now. I asked him to describe list all the pros he could think of.  He mentioned the following:

  1. Better pay (30% more than current)
  2. Greater responsibility compared to current job.
  3. Good learning opportunity (new domain)
  4. Better career prospects

These points have been depicted in Figure 2.

I felt the second point needed clarification, so I asked him to elaborate on what he meant by “Greater responsibility.” Now,  any node in IBIS can be elaborated by asking appropriate questions and responding to them. He mentioned that he would be managing a bigger team which was responsible for a wider range of functions. I broke these up into two points elaborating on the “greater responsibility” point. The elaborations are on the extreme right in Figure 2.

Figure 2

“Do you see any downside to taking the job?” I asked.

“A big one,” he said, “there are all the unknowns that go with a new working environment: the need to build relationships from scratch, greater pressure and possibly longer working hours until I establish my credibility.”

I mapped these points with “New working environment” as the main point and the others as elaborations of it.

I then asked him if there were any other cons he could think of.

“Just one,” he said, “From what I hear,  the company may not be as stable as my current employer.”

I added this in and   asked him to check if the map was an accurate representation of what he meant. “Yes,” he said, “that’s a good summary of my concerns.”

The map at this stage is shown in Figure 3.

Figure 3

He seemed to be done with the “accept” option, so I suggested moving on to outlining arguments for and against staying with his present employer.

Tellingly, he outlined the pros first:

  1. He was in his comfort zone – he knew what was expected of him and was able to deliver it with ease.
  2. The working environment in his company was excellent.
  3. The company was doing very well and the outlook for the foreseeable future was good.

I mapped these as he spoke. After he was done, I asked him to outline the cons of staying. Almost right away he mentioned the following:

  1. No prospects for career advancement; the job was a dead end.
  2. After five odd years, he was getting a bit bored.

The map at this stage is shown in Figure 4.

Figure 4

Through the conversation, he was watching me sketch out the map on paper, but he hadn’t been looking too closely (perhaps because my handwriting is pure chicken scrawl). Nevertheless, at this point he turned the paper around to have a closer look, and said, “Hey, this isn’t bad – my options and reasons for and against them in one glance.”

I asked him if the analysis thus far suggested a choice.

“No,” he said shaking his head.

To make progress, I suggested looking at the cons for both options, to see if these could be mitigated in some way.

We looked at the “stay with the current employer option” – there was really nothing that could be done about the issue of career stagnation or the fact that the work was routine. The company couldn’t create a new position just for him, and the work was all routine because  he had done it for so long.

Finally, we turned our attention to the cons of the “move to the new employer” option.

After a while, he said “The uncertainty regarding a new work environment isn’t specific to this employer. It applies to any job – including the ones I may consider in future.”

“That’s an excellent point,” I said, “So we should disregard it, right?”

“Yes. So we’re left with only the stability issue.”

“Perhaps you can search the Web for some information on the company and may be even ask around.”

“I think things are a lot clearer now,” he said as I  added the points we’d just discussed to the map.

Figure 5

Figure 5 shows the final result of our deliberations.

Afterword

A few days later he called to tell me that he had accepted the job, but only after satisfying himself  that the company was sound.  He had to rely on information that he found on the web because no one in his network of accquaintances knew anything about the company.  But that’s OK, he said –   absolute certainty is impossible when dealing with life’s little dilemmas.

Written by K

August 12, 2010 at 10:34 pm

Cause and effect in management

with 2 comments

Introduction

Management schools and gurus tell us that specific managerial actions will lead to desirable consequences – witness the prescriptions for success in books such as Good to Great or In Search of Excellence. But can one really attribute success (or failure) to specific actions?  A cause-effect relationship is often assumed, but in reality the causal connection between strategic management actions and organisational outcomes is tenuous.  This post, based on a paper by Glenn Shafer entitled Causality and Responsibility, is an exploration of the causal connection between managerial actions and their (assumed) consequences.

Note that the discussion below applies to strategic – or “big picture” – management decisions, not operational ones. In the latter, cause and effect is generally quite clear cut. For example, the decision to initiate a project sets in motion several processes that have fairly predictable outcomes.  However, taking a big picture view,  initiating a project (or even the successful completion of one) does not imply that the strategic aims of the project will be met. It is the latter point that is of interest here – the causal connection between a strategic decision and its assumed outcome.

Shafer’s paper deals with causality and responsibility in legal deliberations: specifically, the process by which judges and juries reach their verdict as to whether the accused (person or entity) is actually responsible (in a causal sense) for the outcome they are charged with. In short, did the actions of the accused cause the outcome?  The arguments Shafer makes are quite general, and have applicability to any discipline. In the following paragraphs I’ll look at a couple of the key points he makes and outline their implications for cause and effect in management actions.

Deterministic cause-effect relationships

The first point that Shafer makes is that we should infer that a particular action causes a particular outcome only if it is improbable that the outcome could have happened without the action preceding it. In Shafer’s words:

…we are on safe ground in attributing responsibility if we do so based on our knowledge of impossibilities. It is not surprising, therefore, that the classical legal concept of cause – necessary and sufficient cause – is defined in terms of impossibility. According to this concept, an action causes an event if the event must happen (it is impossible for it not to happen) when the action is taken and cannot happen (it is impossible for it to happen) if the action is not taken.

This is, in fact, what legal arguments attempt to do: they attempt to prove, beyond reasonable doubt, that the crime occurred because of the defendants actions.

The reason that impossibilities are a better way of “proving” causal relationships is that such relationships cannot be invalidated as our knowledge of the situation increases providing the knowledge that we already have is valid.  In other words,  once something is deemed impossible (using valid knowledge) then it remains so even if we get to know more about the situation. In contrast, if something is deemed possible in the light of existing knowledge, it can be rendered false by a single contradictory fact.

The implication of the above for cause and effect in management is clear: a manager can (should!) claim responsibility for a particular outcome only if:

  1. The outcome must (almost always) happen if the managerial action occurs.
  2. It is highly unlikely that the outcome could have occurred without the action occurring prior to it.

Seen in this light, many of the prescriptions laid out in management bestsellers are little better than herpetological oleum.

Probabilistic cause-effect relationships

Of course, deterministic cause-effect relationships aren’t the norm in management –  only the supremely confident (foolhardy?) would claim that a specific managerial action will always lead to a specific organisational outcome.  This begs the question: what about probabilistic relationships? That is, what can we say about claims that a particular action results in a particular effect, but only in a fraction of the instances in which the action occurs?

To address this question, Shafer makes the important point that probabilities not close to zero or one have no meaning in isolation. They have meaning only in a system, and their meaning derives from the impossibility of a successful gambling strategy—the probability close to one that no one can make a substantial amount of money betting at the odds given by the probabilities.  The last part of the previous statement is a consequence of how probabilities are validated empirically. In Shafer’s words:

We validate a system of probabilities empirically by performing statistical tests. Each such test checks whether observations have some overall property that the system says they are practically certain to have. It checks, in other words, on whether observations diverge from the probabilistic model in a way that the model says is practically (approximately) impossible. In Probability and Finance: It’s Only a Game, Vovk and I argue that both the applications of probability and the classical limit theorems (the law of large numbers, the central limit theorem, etc.) can be most clearly understood and most elegantly explained if we treat these asserted practical impossibilities as the basic meaning of a probabilistic or statistical model, from which all other mathematical and practical conclusions are to be derived.  I cannot go further into the argument of the book here, but I do want to emphasize one of its consequences: because the empirical validity of a system of probabilities involves only the approximate impossibilities it implies, it is only these approximate impossibilities that we should expect to see preserved in a deeper causal structure. Other probabilities, those not close to zero or one, may not be preserved and hence cannot claim the causal status.

An implication of the above is that probabilities not close to zero or one are not fundamental properties of the system/situation; they are subject to change as our knowledge of the situation/system improves. A simple example may serve to explain this point.  Consider the following hypothetical claim from a software vendor:

“80% of our customers experience an increase in sales after implementing our software system.”

Presumably, the marketing department responsible for this statement has the data to back it up. Despite that, the increase in sales for a particular customer cannot (should not!) be attributed to the software. Why?  Well, for the following reasons:

  1. The particular customer may differ in important ways from those used in estimating the probability.This is a manifestation of the reference class problem.
  2. Most statistical studies of the kind used in marketing or management studies are enumerative, not analytical – i.e they can be used to classify data, but not to establish cause-effect relationships. See my post entitled Enumeration or Analysis for  more onthe differences between enumerative and analytical studies.

There is an underlying reason for the above which I’ll discuss next.

The root of the problem  – too many variables

The points made above – that outcomes cannot be attributed to actions unless the probabilities involved are close to zero or one – is a consequence of the fact that most organisational outcomes are results of several factors.   Therefore it is incorrect to attribute the outcome to a single factor (such as farsighted managerial action). Nancy Cartwright makes this point in her paper entitled Causal Laws and Effective Strategies, where she states that a cause ought to increase the frequency of its purported outcome,  but this increase can be masked by other causal factors that have not been taken into account. She uses the somewhat dated and therefore incorrect example of the relationship between smoking and heart disease. However, it serves to illustrate the point, so I’ll quote it below:

…a cause ought to increase the frequency of its effect. But this fact may not show up in the probabilities if other causes are at work. Background correlations between the purported cause and other causal factors may conceal the increase in probability which would otherwise appear. A simple example will illustrate. It is generally supposed that smoking causes heart disease. Thus, we may expect that the probability of heart disease on smoking is greater than otherwise (K’s note: i.e. the conditional probability of heart disease given that the person is a smoker, P(H/S),  is greater than the probability of heart disease in the general population, P(H)). This expectation is mistaken, however. Even if it is true that smoking causes heart disease, the expected increase in probability will not appear if smoking is correlated with a sufficiently strong preventative, say exercising. To see why this is so, imagine that exercising is more effective at preventing heart disease than smoking at causing it. Then in any population where smoking and exercising are highly enough correlated,  it can be true that P(H/S) = P(H), or even P(H/S) < P(H).  For the population of smokers also contains a good many exercisers, and when the two are in combination, the exercising tends to dominate….

In the case of strategic outcomes, it is impossible to know all the factors involved. Moreover, the factors are often interdependent and subject to positive feedback (see my previous post for more on this). So the problem is even worse than implied by Cartwright’s example.

Conclusions

The implications of the above can be summarised as follows: the efficacy of most strategic managerial actions is questionable because the probabilities involved in such claims are rarely close to zero or one. This shouldn’t be a surprise:  most organisational outcomes are consequences of several factors acting in concert, many of which combine in unpredictable ways.   Given this  is unreasonable to expect that   managerial actions will result in predictable organisational outcomes.  That said, it is only natural to claim responsibility for desirable outcomes and shift the blame for undesirable ones, as it is to seek simplistic solutions to difficult organisational problems. Hence the insatiable market for management snake oil.

To make matters worse, the factors are often interdependent (see my previous post for more on this).

Written by K

August 5, 2010 at 10:09 pm

%d bloggers like this: