Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘Decision Making’ Category

From ambiguity to action – a paper preview

with 12 comments

The powerful documentary The Social Dilemma highlights the polarizing effect of social media, and how it hinders our collective ability to address problems that impact communities, societies and even nations. Towards the end of the documentary, the technology ethicist, Tristan Harris, makes the following statement:

“If we don’t agree on what is true or that there’s such a thing as truth, we’re toast. This is the problem beneath all other problems because if we can’t agree on what is true, then we can’t navigate out of any of our problems.”

The central point the documentary makes is that the strategies social media platforms use to enhance engagement also tend to encourage the polarization of perspectives. A consequence is that people on two sides of a contentious issue become less likely to find common ground and build a shared understanding of a complex problem.

A similar dynamic plays out in organisations, albeit on a smaller and less consequential scale. For example, two departments – say, sales and marketing – may have completely different perspectives on why sales are falling.  Since their perspectives are different, the mitigating actions they advocate may be completely different, even contradictory. In a classic paper, published half a century ago, Horst Rittel and Melvin Webber coined the term wicked problem to describe such ambiguous dilemmas.

In contrast, problems such as choosing the cheapest product from a range of options are unambiguous because the decision criteria are clear. Such problems are sometimes referred to as tame problems.  As an aside, it should be noted that organisations often tend to treat wicked problems as tame, with less-than-optimal consequences down the line. For example, choosing the cheapest product might lead to larger long-term costs due to increased maintenance, repair and replacement costs.

The problem with wicked problems is that they cannot be solved using rational approaches to decision making. The reason is that rational approaches assume that a) the decision options can be unambiguously determined upfront, and b) that they can be objectively rated.  This implicitly assumes that all those who are impacted by the decision will agree on the options and the rating criteria. Anyone who has been involved in making a contentious decision will know that these are poor assumptions. Consider, for example, management and employee perspectives on an organizational restructuring.

In a book published in 2016, Paul Culmsee and I argued that the difference between tame and wicked problems lies in the nature of uncertainty associated with the two. In brief, tame problems are characterized by uncertainties that can be easily quantified (e.g., cost or time in projects) whereas wicked problems are characterized by uncertainties that are hard to quantify (e.g., the uncertainties associated with a business strategy).  One can think of these as lying at the opposite ends of an ambiguity spectrum, as shown below:

Figure 1: The Ambiguity Spectrum

It is important to note that most real-world problems have both quantifiable and unquantifiable uncertainties and the first thing that one needs to do when one is confronted with a decision making situation is to figure out, qualitatively, where the problem lies on the ambiguity spectrum:

Figure 2: Where does your problem lie on the ambiguity spectrum?

The key insight is that problems that have quantifiable uncertainties can be tackled using rational decision making techniques whereas those with unquantifiable uncertainties cannot. Problems of the latter kind are wicked, and require a different approach – one that focuses on framing the problem collectively (i.e., involving all impacted stakeholders) prior to using rational decision making approaches to address it. This is the domain of sensemaking, which I like to think of as the art of extracting or framing a problem from a messy situation.

Sensemaking is something we all do instinctively when we encounter the unfamiliar – we try to make sense of the situation by framing it in familiar terms. However, in an unfamiliar situation, it is unlikely that a single perspective on a problem will be an appropriate one. What is needed in such situations is for people with different perspectives to debate their views openly and build a shared understanding of the problem that synthesizes the diverse viewpoints. This is sometimes called collective sensemaking.

Collective sensemaking is challenging because it involves exactly the kind of cooperation that Tristan Harris calls for in the quote at the start of this piece.

But when people hold conflicting views on a contentious topic, how can they ever hope to build common ground? It turns out there are ways to build common ground, and although they aren’t perfect (and require diplomacy and doggedness) they do work, at least in many situations if not always. A technique I use is dialogue mapping which I have described in several articles and a book co-written with Paul Culmsee.

Figure 3: An example dialogue (or issue) map

Regardless of the technique used, the point I’m making is that when dealing with ambiguous problems one needs to use collective sensemaking to frame the problem before using rational decision making methods to solve it. When dealing with an ambiguous problem, the viability of a decision hinges on the ability of the decision maker to: a) help stakeholders distinguish facts from opinions, b) take necessary sensemaking actions to find common ground between holders of conflicting opinions, and c) build a base of shared understanding from which a commonly agreed set of “facts” emerge. These “facts” will not be absolute truths but contingent ones. This is often true even of so-called facts used in rational decision making: a cost quotation does not point to a true cost, rather it is an estimate that depends critically on the assumptions made in its calculation. Such decisions, therefore, cannot be framed based on facts alone but ought to be co-constructed with those affected by the decision.  This approach is the basis of a course on decision making under uncertainty that I designed and have been teaching across two faculties at the University of Technology Sydney for the last five years.

In a paper, soon to be published in Management Decision, a leading journal on decision making in organisations, Natalia Nikolova and I describe the principles and pedagogy behind the course in detail. We also highlight the complementary nature of collective sensemaking and rational decision making, showing how the former helps in extracting (or framing) a problem from a situation while the latter solves the framed problem. We also make the point that decision makers in organisations tend to jump into “solutioning” without spending adequate time framing the problem appropriately.  

Finally, it is worth pointing out that the hard sciences have long recognized complementarity to be an important feature of physical theories such as quantum mechanics. Indeed, the physicist Niels Bohr was so taken by this notion that he inscribed the following on his coat of arms: contraria sunt complementa (opposites are complementary). The integration of apparently incompatible elements into a single theory or model can lead to a more complete view of the world and hence, how to act in it. Summarizing the utility of our approach in a phrase: it can help decision makers learn how to move from ambiguity to action.

For copyright reasons, I cannot post the paper publicly. However, I’d be happy to share it with anyone interested in reading / commenting on it – just let me know via a comment below.

Note added on 13 May 2022:

The permalink to the published online version is: https://www.emerald.com/insight/content/doi/10.1108/MD-06-2021-0804/full/html

Written by K

May 3, 2022 at 7:59 am

Monte Carlo Simulation of Projects – an (even simpler) explainer

with 3 comments

In this article I’ll explain how Monte Carlo simulation works using an example of a project that consists of two tasks that must be carried out sequentially as shown in the figure:

Task 1 takes 3 to 7 days

Task 2 takes 2 to 5 days

The two tasks do not have any dependencies other than that they need to be completed in sequence.

(Note: in case you’re wondering about “even simpler” bit in the title – the current piece is, I think, even easier to follow than this one I wrote up some years ago).

Assume the project has been carried a number of times in the past – say 20 times – and we have the data shown below for the two tasks. For each task, we have the frequency of completion by day. So, Task 1 was completed twice on day 3 , four times on day 4 and so on. Similarly, Task 2 was completed twice on the 2nd day after the task started and 10 times the 3rd day after the task started and so on.

Consider Task 1. Since it was completed 2 times on day 3 and 4 times on day 4, it is reasonable to assume that it twice as likely that it will finish on day 4 than on day 3. In other words, the number of times a task is completed on a particular day is proportional to the probability of finishing on that day.  

One can therefore approximate the probability of finishing on a particular day by dividing the number of completions on that day by the total number of times the task was performed. So, for example, the probability of finishing task 1 on day 3 is 2/20 or 0.1 and the probability of finishing it on day 4 is 0.2.

It is straightforward to calculate the probability for each of the completion days. The tables displayed below show the calculated probabilities. The tables also show the cumulative probability – this is sum of all probabilities of completion prior to (and including) current completion day. This gives the probability of finishing by the particular day – that is, on that day or any day before it.  This, rather than the probability, is typically what you want to know.

The cumulative probability has two useful properties

  1. It is an increasing function (that is, it increases as the completion day increases)
  2. It lies between 0 and 1

What this means is that if we pick any number between 0 and 1, we will be able to find the “completion day” corresponding to that number. Let’s try this for task one:

Say we pick 0.35. Since 0.35 lies between 0.3 and 0.75, it corresponds to a completion between day 4 and day 5. That is, the task will be completed by day 5. Indeed, any number picked between 0.3 and 0.75 will correspond to a completion by day 5.

Say we pick 0.79. Since 0.79 lies between 0.75 and 0.95, it corresponds to a completion between day 5 and day 6. That is, the task will be completed by day 6.

….and so on.  It is easy to see that any random number between 0 and 1 corresponds to a specific completion day depending on which cumulative probability interval it lies in.

Let’s pick a thousand random numbers between 0 and 1 and find the corresponding completion days for each.  It should be clear from what I have said so far that these correspond to 1000 simulations of task 1, consistent with the historical data that we have on the task.

We will do the simulations in Excel. You may want to download the workbook that accompanies this post and follow along.

Enter the completion days and the cumulative probabilities corresponding to them in rows 1 through 8 of columns A and B  as shown below.

Then enter the Excel RAND() function in cell A10 as shown in the figure below. This generates a random number between 0 and 1 (note that the random number you generate will be different from mine).

Next, fill down to cell A1009 to generate 1000 random numbers between 0 and 1 – see figure below ( again your random numbers will be different from mine)

Now in cell B10, input the formula shown below:

This nested IF() function checks which cumulative probability interval the random number lies in and returns the corresponding completion day. This is the completed by day corresponding to the inputted probability.

Fill this down to cell B1009. Your first few rows will look something like shown in the figure below:

You have now simulated Task 1 thousand times.

Next, enter the data for task 2 in columns D and E (from rows 1 through 7) and follow a similar procedure to simulate Task 2 thousand times. When you’re done, you will have something like what’s shown below (again, your random numbers and hence your completed by days will differ from mine):

Each line from row 10 to 1009 corresponds to a simulation of the project. So, this is equivalent to running the project 1000 times.

We can get completion times for each simulation by summing columns B and E, which will give us 1000 project completion times. Let’s do this in column G.

Using the MIN() and MAX() functions over the range G10:G1009, we see  that the earliest and latest days for project completion are day 5 and day 12 respectively.

Using the simulation results, we can now get approximate cumulative probabilities for each of the possible completion days (i.e days 5 through 12).

Pause for a minute and have a think about how you would do this.

–x–

OK, so here’s how you would do it for day 5

Count the number of 5s in the range G10:G1009 using the COUNTIF() function.   To estimate the probability of completion on day 5, divide this number by the total number of simulations.  

To get the cumulative probability you would need to add in the probabilities for all prior completion days.  However, since day 5 is the earliest possible completion day, there is no prior day.

Let’s do day 6

Count the number of 6s in the range G10:G1009 using the COUNTIF() function.   To estimate the probability of completion on day 6, divide this number by the total number of simulations.  

To get the cumulative probability you would need to add the estimated  probability of completion for day 5 to the estimated  probability of completion for day 6.

…and so on.

The resulting table, show below, is excerpted from  columns J and K of the Excel workbook linked to above. Your numbers will differ (but hopefully by not too much) from the ones shown in the table.

Now that we have done all this work, we can make statements like:

  1. It is highly unlikely that we will finish before day 7.
  2. There’s an 80% chance that we will finish by day 9.
  3. There’s a 95% chance we’ll finish by day 10.

…and so on.

And that’s how Monte Carlo simulations work in the context of project estimation

Before we close, a word or two about data. The method we have used here assumes that you have detailed historical completion data for the tasks. However, you probably know from experience that it is rarely the case that you have this.

What do you do then?

Well, one can develop probability distributions based on subjective probabilities. Here’s how: ask the task performer for a best guess earliest, most likely and latest completion time. Based on these, one can construct triangular probability distributions that can be used in simulations. It would take me far too long to explain the procedure here so I’ll point you to an article instead.

And that’s it for this explainer. I hope it has given you a sense for how Monte Carlo simulations work.

Written by K

January 4, 2022 at 5:29 pm

Two ways of knowing

with 2 comments

I usually don’t pick up calls from unknown numbers but that day, for some reason, I did.

“It’s Raj,” he said, “from your decision-making class.”

As we exchanged pleasantries, I wondered what he was calling about.

“I’m sorry to call out of the blue, but I wanted to tell you about something that happened at work. It relates to what you talked about in last week’s class – using sensemaking techniques to help surface and resolve diverse perspectives on wicked problems.”

[Note: wicked problems are problems that are difficult to solve because stakeholders disagree on what the problem is. A good example of contemporary importance is climate change]

Naturally, I asked Raj to tell me more.

It turned out that he worked for a large IT consulting firm where his main role was to design customised solutions for businesses.  The incident he related had occurred at a pre-sales meeting with a potential customer. During the meeting it had become clear that the different stakeholders from the customer side had differing views on what they wanted from the consulting firm. 

On the spur of the moment, Raj decided to jump in and help them find common ground.

To do so, he adapted a technique I had discussed at length in class – and I’ll say more about the technique later. It worked a treat – he helped stakeholders resolve their differences and thereby reframe their problem in a more productive way. Ironically, this led to the customer realising that they did not need the solution Raj’s company was selling. However, a senior manager on the customer’s side was so impressed with Raj’s problem framing and facilitation skills that he was keen to continue the dialogue with Raj’s company.

“We think we know our customers through the data we have about them,” said Raj, “but understanding them is something else altogether.”

–x–

In his book, Ways of Attending, Iain McGilchrist draws attention to the asymmetry in the human brain and its consequences for the ways in which we understand the world. His main claim is that the two hemispheres of the brain perceive reality very differently: the left hemisphere sees objects and events through the lens of theories, models and abstractions whereas the right sees them as embedded in and thus related to a larger world.  As he puts it:

The left hemisphere tends to see things more in the abstract, the right hemisphere sees them more embedded in the real-world context in which they occur. As a corollary, the right hemisphere seems better able to appreciate actually existing things in all their uniqueness, while the left hemisphere schematises and generalises things into categories.”

McGilchrist emphasises that both hemispheres are involved in reasoning and emotion, but in very different ways. 

In the left hemisphere, we “experience” our experience in a special way: a “re-presented” version of it, containing now static, separable, bounded, but essentially fragmented entities, grouped into classes on which predictions can be based. This kind of attention isolates, fixes and makes each thing explicit by bringing it under the spotlight of attention. In doing so it renders things inert, mechanical, lifeless. In the other, that of the right hemisphere, we experience the live, complex, embodied world of individual, always unique, beings, forever in flux, a net of interdependencies, forming and reforming wholes, a world with which we are deeply connected.”

It struck me that Raj’s epiphany was deeply connected with McGilchrist’s distinction between the ways in which the two hemispheres of our brains attend to the world.

–x–

The data we collect on our customers (or anything else for that matter) focuses on facts, attributes that are easy to classify or measure.  However, such data has its limitations. As McGilchrist notes in his magnum opus, The Master and His Emissary:

…there is [a] kind of knowledge that comes from putting things together from bits. It is a knowledge of what we call facts. This is not usually well-applied to knowing people. We could have a go – for example, ‘born on 16 September 1964’, ‘lives in New York’, ‘5ft 4inches tall’, ‘red hair’ …, and so on. Immediately you get a sense of somebody who you don’t actually know….What’s more, it sounds as though you’re describing an inanimate object…

Facts by themselves do not lead to understanding.

–x–

Understanding something requires one to connect the dots between facts, to build a mental model of what is going on. This is a creative act that requires a blend of imaginative and logical thinking.

Since mental models we build are necessarily influenced our beliefs and past experiences, it is unreasonable to expect any two individuals will understand a given situation in exactly the same way. Understanding is a mental process, not a thing, and knowledge (as in knowing something) is an ever-evolving byproduct of the process. Education is not about conveying knowledge, rather it is about helping students expand and refine their individual processes of understanding.

As Heinz von Foerster put it:

No wonder that an educational system that confuses the process of creating new processes with the dispensing of goods called ‘knowledge’ may cause some disappointment in the hypothetical receivers, for the goods are just not forthcoming: there are no goods.

Historically, I believe, the confusion by which knowledge is taken as substance comes from a witty broadsheet printed in Nuremberg in the Sixteenth Century. It shows a seated student with a hole on top of his head into which a funnel is inserted. Next to him stands the teacher who pours into this funnel a bucket full of “knowledge,” that is, letters of the alphabet, numbers and simple equations.”

The entire business of education is predicated on the assumption that knowledge is transferable and is assimilated by different individuals in exactly the same way. But this cannot be so. As von Foerster so eloquently noted, “the processes [of assimilating knowledge] cannot be passed on… for your nervous activity is just your nervous activity and, alas, not mine

Since assimilating knowledge (understanding, by another name) is a highly individual activity, it should not be surprising that individuals perceive and understand situations in very different ways. This is precisely the problem that Raj ran into: two stakeholders on the client’s side had different understandings of the problem.

Raj adapted a sensemaking technique called dialogue mapping to help the two stakeholders arrive at a shared understanding of the problem. The technique itself is not as important as the rationale behind it. And that is best conveyed through another story.

—x–

Many years ago, when I worked as a data architect at a large multinational, I was invited to participate in a regional project aimed at building a data warehouse for subsidiaries across Asia. The initiative was driven by the corporate IT office located in Europe. Corporate’s interest in sponsoring this was to harmonize a data landscape that was – to put it mildly – messy. On the other hand, the subsidiaries thought their local systems were just fine. They were suspicious of corporate motives which they saw as a power play that would result in loss of autonomy over data and reporting.

The two parties had diametrically opposite understandings of the problem.

Around that time, I stumbled on the notion of a wicked problem and was researching ways to manage such problems in work contexts. It was clear that the solution lay in getting the two parties on the same page. The question was how.

The trick is to find a way to surface and reconcile diverse viewpoints in a way that takes the heat out of the discussion.  One therefore needs a means to make multiple perspectives explicit in a manner that separates opinions from individuals and thus enables a group to develop a shared understanding of contentious issues. There is a visual notation called Issue Based Information System or IBIS that can help facilitators do this.  Among other things, IBIS enables one to capture the informal logic of a conversation using a visual notation that has just four node types (see Figure 1).

Figure 1: IBIS node types

In brief: questions (also called issues) capture the problem being discussed; ideas are responses offered to questions; pros and cons are arguments for and against ideas. The claim is that the informal logic of conversations can be captured using just these four node types.

After playing around a bit with the notation, I was convinced it would be of great value in the discussion about the corporate data warehouse.  A day prior to the meeting, I met the project lead to canvass the possibility of using IBIS to map the discussion. After seeing a brief demo, he was quite taken by the idea and was happy to have me use it, providing the other participants had no objections.

The discussion took place over the course of a day, with participants drawn from three groups of stakeholders: corporate IT, local IT and business reps from the subsidiaries.

Mapping the conversation using IBIS enabled the group to:

  1. Agree on the root (key) question – what approach should we take?
  2. Surface different ideas (options) in response to the question.
  3. Capture arguments for and against ideas.

As the conversation progressed, I noticed that IBIS took the heat out of the discussion by objectifying discussion points. It did this by separating opinions from their holders.   This made it possible for participants to understand (if not quite accept) the rationale behind opposing perspectives. This led them to a more nuanced appreciation of the problem and the proposed solutions.

As the morning wore on, the group gradually converged to a shared understanding of the problem.

By the end of the discussion, it was clear to everyone in the meeting that a subsidiary-focused design that enabled a consistent roll up of data for corporate would be the best option. 

Those interested in the details of what I did and how I did it may want to have a look at this paper. However, I should emphasise that the technique is not the point. What happened is that the parties involved changed their minds even though the facts of the matter remained unchanged, and the point is to create the conditions for that to happen.

–x–

The event occurred over a decade ago but has gained significance for me over the years. As we drown in an ever increasing deluge of facts, we are fast losing the capacity to understand. The most pressing problems of today will not be solved by knowing facts, they will be solved by knowing of the other kind.

–x–x–

%d bloggers like this: