Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘Decision Making’ Category

Monte Carlo Simulation of Projects – an (even simpler) explainer

leave a comment »

In this article I’ll explain how Monte Carlo simulation works using an example of a project that consists of two tasks that must be carried out sequentially as shown in the figure:

Task 1 takes 3 to 7 days

Task 2 takes 2 to 5 days

The two tasks do not have any dependencies other than that they need to be completed in sequence.

(Note: in case you’re wondering about “even simpler” bit in the title – the current piece is, I think, even easier to follow than this one I wrote up some years ago).

Assume the project has been carried a number of times in the past – say 20 times – and we have the data shown below for the two tasks. For each task, we have the frequency of completion by day. So, Task 1 was completed twice on day 3 , four times on day 4 and so on. Similarly, Task 2 was completed twice on the 2nd day after the task started and 10 times the 3rd day after the task started and so on.

Consider Task 1. Since it was completed 2 times on day 3 and 4 times on day 4, it is reasonable to assume that it twice as likely that it will finish on day 4 than on day 3. In other words, the number of times a task is completed on a particular day is proportional to the probability of finishing on that day.  

One can therefore approximate the probability of finishing on a particular day by dividing the number of completions on that day by the total number of times the task was performed. So, for example, the probability of finishing task 1 on day 3 is 2/20 or 0.1 and the probability of finishing it on day 4 is 0.2.

It is straightforward to calculate the probability for each of the completion days. The tables displayed below show the calculated probabilities. The tables also show the cumulative probability – this is sum of all probabilities of completion prior to (and including) current completion day. This gives the probability of finishing by the particular day – that is, on that day or any day before it.  This, rather than the probability, is typically what you want to know.

The cumulative probability has two useful properties

  1. It is an increasing function (that is, it increases as the completion day increases)
  2. It lies between 0 and 1

What this means is that if we pick any number between 0 and 1, we will be able to find the “completion day” corresponding to that number. Let’s try this for task one:

Say we pick 0.35. Since 0.35 lies between 0.3 and 0.75, it corresponds to a completion between day 4 and day 5. That is, the task will be completed by day 5. Indeed, any number picked between 0.3 and 0.75 will correspond to a completion by day 5.

Say we pick 0.79. Since 0.79 lies between 0.75 and 0.95, it corresponds to a completion between day 5 and day 6. That is, the task will be completed by day 6.

….and so on.  It is easy to see that any random number between 0 and 1 corresponds to a specific completion day depending on which cumulative probability interval it lies in.

Let’s pick a thousand random numbers between 0 and 1 and find the corresponding completion days for each.  It should be clear from what I have said so far that these correspond to 1000 simulations of task 1, consistent with the historical data that we have on the task.

We will do the simulations in Excel. You may want to download the workbook that accompanies this post and follow along.

Enter the completion days and the cumulative probabilities corresponding to them in rows 1 through 8 of columns A and B  as shown below.

Then enter the Excel RAND() function in cell A10 as shown in the figure below. This generates a random number between 0 and 1 (note that the random number you generate will be different from mine).

Next, fill down to cell A1009 to generate 1000 random numbers between 0 and 1 – see figure below ( again your random numbers will be different from mine)

Now in cell B10, input the formula shown below:

This nested IF() function checks which cumulative probability interval the random number lies in and returns the corresponding completion day. This is the completed by day corresponding to the inputted probability.

Fill this down to cell B1009. Your first few rows will look something like shown in the figure below:

You have now simulated Task 1 thousand times.

Next, enter the data for task 2 in columns D and E (from rows 1 through 7) and follow a similar procedure to simulate Task 2 thousand times. When you’re done, you will have something like what’s shown below (again, your random numbers and hence your completed by days will differ from mine):

Each line from row 10 to 1009 corresponds to a simulation of the project. So, this is equivalent to running the project 1000 times.

We can get completion times for each simulation by summing columns B and E, which will give us 1000 project completion times. Let’s do this in column G.

Using the MIN() and MAX() functions over the range G10:G1009, we see  that the earliest and latest days for project completion are day 5 and day 12 respectively.

Using the simulation results, we can now get approximate cumulative probabilities for each of the possible completion days (i.e days 5 through 12).

Pause for a minute and have a think about how you would do this.

–x–

OK, so here’s how you would do it for day 5

Count the number of 5s in the range G10:G1009 using the COUNTIF() function.   To estimate the probability of completion on day 5, divide this number by the total number of simulations.  

To get the cumulative probability you would need to add in the probabilities for all prior completion days.  However, since day 5 is the earliest possible completion day, there is no prior day.

Let’s do day 6

Count the number of 6s in the range G10:G1009 using the COUNTIF() function.   To estimate the probability of completion on day 6, divide this number by the total number of simulations.  

To get the cumulative probability you would need to add the estimated  probability of completion for day 5 to the estimated  probability of completion for day 6.

…and so on.

The resulting table, show below, is excerpted from  columns J and K of the Excel workbook linked to above. Your numbers will differ (but hopefully by not too much) from the ones shown in the table.

Now that we have done all this work, we can make statements like:

  1. It is highly unlikely that we will finish before day 7.
  2. There’s an 80% chance that we will finish by day 9.
  3. There’s a 95% chance we’ll finish by day 10.

…and so on.

And that’s how Monte Carlo simulations work in the context of project estimation

Before we close, a word or two about data. The method we have used here assumes that you have detailed historical completion data for the tasks. However, you probably know from experience that it is rarely the case that you have this.

What do you do then?

Well, one can develop probability distributions based on subjective probabilities. Here’s how: ask the task performer for a best guess earliest, most likely and latest completion time. Based on these, one can construct triangular probability distributions that can be used in simulations. It would take me far too long to explain the procedure here so I’ll point you to an article instead.

And that’s it for this explainer. I hope it has given you a sense for how Monte Carlo simulations work.

Written by K

January 4, 2022 at 5:29 pm

Two ways of knowing

with one comment

I usually don’t pick up calls from unknown numbers but that day, for some reason, I did.

“It’s Raj,” he said, “from your decision-making class.”

As we exchanged pleasantries, I wondered what he was calling about.

“I’m sorry to call out of the blue, but I wanted to tell you about something that happened at work. It relates to what you talked about in last week’s class – using sensemaking techniques to help surface and resolve diverse perspectives on wicked problems.”

[Note: wicked problems are problems that are difficult to solve because stakeholders disagree on what the problem is. A good example of contemporary importance is climate change]

Naturally, I asked Raj to tell me more.

It turned out that he worked for a large IT consulting firm where his main role was to design customised solutions for businesses.  The incident he related had occurred at a pre-sales meeting with a potential customer. During the meeting it had become clear that the different stakeholders from the customer side had differing views on what they wanted from the consulting firm. 

On the spur of the moment, Raj decided to jump in and help them find common ground.

To do so, he adapted a technique I had discussed at length in class – and I’ll say more about the technique later. It worked a treat – he helped stakeholders resolve their differences and thereby reframe their problem in a more productive way. Ironically, this led to the customer realising that they did not need the solution Raj’s company was selling. However, a senior manager on the customer’s side was so impressed with Raj’s problem framing and facilitation skills that he was keen to continue the dialogue with Raj’s company.

“We think we know our customers through the data we have about them,” said Raj, “but understanding them is something else altogether.”

–x–

In his book, Ways of Attending, Iain McGilchrist draws attention to the asymmetry in the human brain and its consequences for the ways in which we understand the world. His main claim is that the two hemispheres of the brain perceive reality very differently: the left hemisphere sees objects and events through the lens of theories, models and abstractions whereas the right sees them as embedded in and thus related to a larger world.  As he puts it:

The left hemisphere tends to see things more in the abstract, the right hemisphere sees them more embedded in the real-world context in which they occur. As a corollary, the right hemisphere seems better able to appreciate actually existing things in all their uniqueness, while the left hemisphere schematises and generalises things into categories.”

McGilchrist emphasises that both hemispheres are involved in reasoning and emotion, but in very different ways. 

In the left hemisphere, we “experience” our experience in a special way: a “re-presented” version of it, containing now static, separable, bounded, but essentially fragmented entities, grouped into classes on which predictions can be based. This kind of attention isolates, fixes and makes each thing explicit by bringing it under the spotlight of attention. In doing so it renders things inert, mechanical, lifeless. In the other, that of the right hemisphere, we experience the live, complex, embodied world of individual, always unique, beings, forever in flux, a net of interdependencies, forming and reforming wholes, a world with which we are deeply connected.”

It struck me that Raj’s epiphany was deeply connected with McGilchrist’s distinction between the ways in which the two hemispheres of our brains attend to the world.

–x–

The data we collect on our customers (or anything else for that matter) focuses on facts, attributes that are easy to classify or measure.  However, such data has its limitations. As McGilchrist notes in his magnum opus, The Master and His Emissary:

…there is [a] kind of knowledge that comes from putting things together from bits. It is a knowledge of what we call facts. This is not usually well-applied to knowing people. We could have a go – for example, ‘born on 16 September 1964’, ‘lives in New York’, ‘5ft 4inches tall’, ‘red hair’ …, and so on. Immediately you get a sense of somebody who you don’t actually know….What’s more, it sounds as though you’re describing an inanimate object…

Facts by themselves do not lead to understanding.

–x–

Understanding something requires one to connect the dots between facts, to build a mental model of what is going on. This is a creative act that requires a blend of imaginative and logical thinking.

Since mental models we build are necessarily influenced our beliefs and past experiences, it is unreasonable to expect any two individuals will understand a given situation in exactly the same way. Understanding is a mental process, not a thing, and knowledge (as in knowing something) is an ever-evolving byproduct of the process. Education is not about conveying knowledge, rather it is about helping students expand and refine their individual processes of understanding.

As Heinz von Foerster put it:

No wonder that an educational system that confuses the process of creating new processes with the dispensing of goods called ‘knowledge’ may cause some disappointment in the hypothetical receivers, for the goods are just not forthcoming: there are no goods.

Historically, I believe, the confusion by which knowledge is taken as substance comes from a witty broadsheet printed in Nuremberg in the Sixteenth Century. It shows a seated student with a hole on top of his head into which a funnel is inserted. Next to him stands the teacher who pours into this funnel a bucket full of “knowledge,” that is, letters of the alphabet, numbers and simple equations.”

The entire business of education is predicated on the assumption that knowledge is transferable and is assimilated by different individuals in exactly the same way. But this cannot be so. As von Foerster so eloquently noted, “the processes [of assimilating knowledge] cannot be passed on… for your nervous activity is just your nervous activity and, alas, not mine

Since assimilating knowledge (understanding, by another name) is a highly individual activity, it should not be surprising that individuals perceive and understand situations in very different ways. This is precisely the problem that Raj ran into: two stakeholders on the client’s side had different understandings of the problem.

Raj adapted a sensemaking technique called dialogue mapping to help the two stakeholders arrive at a shared understanding of the problem. The technique itself is not as important as the rationale behind it. And that is best conveyed through another story.

—x–

Many years ago, when I worked as a data architect at a large multinational, I was invited to participate in a regional project aimed at building a data warehouse for subsidiaries across Asia. The initiative was driven by the corporate IT office located in Europe. Corporate’s interest in sponsoring this was to harmonize a data landscape that was – to put it mildly – messy. On the other hand, the subsidiaries thought their local systems were just fine. They were suspicious of corporate motives which they saw as a power play that would result in loss of autonomy over data and reporting.

The two parties had diametrically opposite understandings of the problem.

Around that time, I stumbled on the notion of a wicked problem and was researching ways to manage such problems in work contexts. It was clear that the solution lay in getting the two parties on the same page. The question was how.

The trick is to find a way to surface and reconcile diverse viewpoints in a way that takes the heat out of the discussion.  One therefore needs a means to make multiple perspectives explicit in a manner that separates opinions from individuals and thus enables a group to develop a shared understanding of contentious issues. There is a visual notation called Issue Based Information System or IBIS that can help facilitators do this.  Among other things, IBIS enables one to capture the informal logic of a conversation using a visual notation that has just four node types (see Figure 1).

Figure 1: IBIS node types

In brief: questions (also called issues) capture the problem being discussed; ideas are responses offered to questions; pros and cons are arguments for and against ideas. The claim is that the informal logic of conversations can be captured using just these four node types.

After playing around a bit with the notation, I was convinced it would be of great value in the discussion about the corporate data warehouse.  A day prior to the meeting, I met the project lead to canvass the possibility of using IBIS to map the discussion. After seeing a brief demo, he was quite taken by the idea and was happy to have me use it, providing the other participants had no objections.

The discussion took place over the course of a day, with participants drawn from three groups of stakeholders: corporate IT, local IT and business reps from the subsidiaries.

Mapping the conversation using IBIS enabled the group to:

  1. Agree on the root (key) question – what approach should we take?
  2. Surface different ideas (options) in response to the question.
  3. Capture arguments for and against ideas.

As the conversation progressed, I noticed that IBIS took the heat out of the discussion by objectifying discussion points. It did this by separating opinions from their holders.   This made it possible for participants to understand (if not quite accept) the rationale behind opposing perspectives. This led them to a more nuanced appreciation of the problem and the proposed solutions.

As the morning wore on, the group gradually converged to a shared understanding of the problem.

By the end of the discussion, it was clear to everyone in the meeting that a subsidiary-focused design that enabled a consistent roll up of data for corporate would be the best option. 

Those interested in the details of what I did and how I did it may want to have a look at this paper. However, I should emphasise that the technique is not the point. What happened is that the parties involved changed their minds even though the facts of the matter remained unchanged, and the point is to create the conditions for that to happen.

–x–

The event occurred over a decade ago but has gained significance for me over the years. As we drown in an ever increasing deluge of facts, we are fast losing the capacity to understand. The most pressing problems of today will not be solved by knowing facts, they will be solved by knowing of the other kind.

–x–x–

Boundaries and horizons

with 2 comments

James Carse once said, “It is the freedom we all know we have that terrifies us.” 

So deep is this terror that we do not want to acknowledge our freedom.  As a result, we play within boundaries defined by fear.

–x–

Some time ago, I bumped into a student who had taken a couple of classes that I taught some years ago. Over a coffee, we got talking about his workplace, a large somewhat bureaucratic organisation.

At one point he asked, “We need to change the way we think about and work with data, but I’m not a manager and have no authority to do what needs to be done.”

“Why don’t you demonstrate what you are capable of without waiting for permission?” I replied. “Since you are familiar with your data, it should be easy enough to frame and solve a small problem that makes a difference.”

“My manager will not like that,” he said.

“It is easier to beg forgiveness than seek permission,” I countered.

“He might feel threatened and make life difficult for me.”

“On the other hand, he might appreciate your efforts.”

“You don’t know him,” he replied.

“If you’re not appreciated, you are always free to leave.  Moreover, the skills you have learnt in the last two years should give you confidence to exercise that freedom.”

“I’m comfortable where I am,” he said sheepishly, “with my mortgage and all this uncertainty in the economy, I can ill-afford any risk.”

I didn’t say so at the time, but thought it unfortunate that he had set boundaries for himself.

–x–

Boundaries are characteristic of what Carse calls finite games: games that are played with the purpose of winning. These are the games of convention, those that we are familiar with. He contrasts these with infinite games: those whose purpose is the continuation of play.

As a corollary, an infinite game has no winner (or loser) because the game never ends.

Finite games are bounded, both temporally (they last a finite time) and spatially (they are played within a bounded region). As Carse notes in his book, “finite players play within boundaries; infinite players play with boundaries.”

And then a bit later, he tells us how to play with boundaries. “What will undo any boundary is the awareness that it is our vision, and not what we are viewing, that is limited.”

–x–

“I’m resigning,” he said, before launching into an explanation. As he talked, the thing that came to mind was the contrast between his attitude and the student’s. 

His explanation was completely unnecessary. I understood.

There comes a tide in the affairs of humans etc…and often that tide is evident only to those who are able and willing to look up and see the possibilities on the distant horizon.

–x–

In contrast to boundaries, horizons are not fixed. As you move towards a horizon it moves away from you. As Carse tells us:

One never reaches a horizon. It is not a line; it has no place; it encloses no field; its location is always relative to the view. To move toward a horizon is simply to have a new horizon.

Much of the talk about lifelong learning (which now has its own Wikipedia entry!) is really about taking a horizonal view of life. It has less to do with “staying current” or “learning employable skills” than with gaining new perspectives.

But that does not mean one has to take in the entire vista in one glance. Some new things…no, most new things, take time.

–x–

“I can’t handle failure,” she said. “I’ve always been at the top of my class.”

She was being unduly hard on herself. With little programming experience or background in math, machine learning was always going to be hard going.  “Put that aside for now,” I replied. “Just focus on understanding and working your way through it, one step at a time. In four weeks, you’ll see the difference.”

“OK,” she said, “I’ll try.”

She did not sound convinced but to her credit, that’s exactly what she did. Two months later she completed the course with a distinction.

“You did it!” I said when I met her a few weeks after the grades were announced.

“I did,” she grinned. “Do you want to know what the made the difference?”

Yes, I nodded.

“I stopped treating it like a game I had to win,” she said, “and that took the pressure right off.  I then started to enjoy learning.”

–x–

For many, success in work…or even in life… is largely a matter of appearances: if one’s career is not marked by a series of increasingly impressive titles then one is likely to be labelled an also-ran, if not an outright failure.

But what is a title? Carse tells us the following:

What one wins in a finite game is a title. A title is the acknowledgment of others that one has been the winner of a particular game. Titles are public. They are for others to notice. I expect others to address me according to my titles, but I do not address myself with them – unless, of course, I address myself as another. The effectiveness of a title depends on its visibility, its noticeability, to others.”

In these lines, Carse makes some important points. Firstly, a title has to be given to us by others.  Secondly, the effectiveness of a title depends on others paying attention to it.  That is, its significance lies in the significance that others give it. This is the reason why titles matter to those who compete for them.

–x–

A couple of weeks ago, I invited an ex-student to give a talk to my machine learning class. As I had expected, he did a brilliant job, introducing the class to some tools that they are likely to find useful the future. But the gold lay in something he said in the Q and A session that followed.

“How do you stay up to date in this field?” a student asked.

“Yes, this is a question I struggled with when I started out, ” said Jose. “Data science is a rapidly expanding field and it is impossible to keep pace with it…but let me show you something.” He navigated to his LinkedIn profile and started scrolling through his list of certifications.

It was a long list.

“When I started out,” he continued, “I constantly felt this fear that I was missing out. So, what did I do? I tried to learn everything I could, collecting a bunch of certifications that I kept adding to my profile. One day, I woke up feeling burnt out and asked myself why I was doing this. The only honest answer was because others seemed to think it necessary and even important. That shook me. I started thinking deeply about what I thought was important for myself, what my purpose is. I realised I did not have one; I was running like crazy down a path set by others, not my own.  That realisation changed everything for me.”

–x–

Life’s too short to play games and chase titles that others deem important. We are free to play our own game and keep playing it as long as we wish to.

Yes, that can be terrifying.

It can also be liberating.

—-xx—-

Written by K

May 3, 2021 at 7:58 pm

%d bloggers like this: