Sherlock Holmes and the case of the terminated PMO

“Tch, tch,” clucked Holmes, shaking his head. “What a tragedy, Watson,” he continued, “yet another project management office cut down in its prime.”

Watson said nothing; he knew his friend did not like interruptions when he was surveying a crime scene.

Holmes walked around as he always did,  in apparently random fashion, his sharp eyes darting from here to there taking in the details –  the process flowcharts on a wall,  project schedules displayed over on the other side,  the printed portfolio reports  that lay on the table and the many other artefacts that are part and parcel of a PMO.

After watching  his friend  for what seemed like an eternity, Watson could hold his curiosity no longer: “What’s your guess, Holmes?” he asked.

“I never guess. It is a shocking habit—destructive to the logical faculty.” He looked up sharply, “You should know better than to ask Watson….”

“I know, Holmes, but my curiosity gets the better of me. What do you think happened?”

“Ah yes, what I think. What I think is not important, Watson,” he said, wagging his index finger in his friend’s direction. “We must focus on what we know – the facts.”

“So, what are the facts?” asked Watson wearily. His friend could be an insufferable pedant.

“You know my methods, Watson. Look around you. What do you see?”

Oh, they were going to play that game again. Shaking his head in exasperation, Watson said, “Why don’t you save time and tell me, Holmes. You are the genius, not I.”

“Ah Watson, sarcasm does not become you. Anyway, I take no offence and will offer you some hints so that you may begin to discern the real reason for the failure of this PMO.”

He walked over to the flowcharts on the wall and asked,” Tell me Watson, What are these and what do they  tell you?”

Watson  walked over to the charts, looked at  them intently and said, “I think we can safely say these describe project management processes.” Then, jabbing his finger at a chart, he continued, “This one  describes the process of authorisation. It seems sensible enough –  a need is identified, a business case drawn up and submitted to the project governance board, it is evaluated against certain criteria and then a decision is made on whether the project should be authorised or not. And look at this one, ‘tis a work of art….”

“Do you know, Watson,” interrupted Holmes, “that it is one of the curses of a mind with a turn like mine that I must look at everything with reference to my own special subject. You look at these attractive flowcharts, and you are impressed by their beauty. I look at them, and the only thought which comes to me is a feeling of their isolation and of the impunity with which they may be subverted.”

“Huh?” blurted Watson, not knowing quite what to make of this.

“I see you are perplexed, Watson. Let me put it another way,  a  PMO may require that project managers comply with certain process, but it cannot enforce compliance.”

“So you think the PMO failed because it could not get project managers to follow processes?”

“Yes, Watson. But experience tells me that although that may be a visible symptom, it is not the cause. You’re a doctor so  I don’t need to tell you that identifying symptoms is necessary but, to cure the disease, one must find the cause. It is all too easy to label the symptom as the cause – many  consultants have done so, and have thus made recommendations that are worse than useless.”

“Worse than useless? I don’t understand, Holmes.”

“Yes, worse than useless. If  organisations focus on curing symptoms rather than causes, they will end up exacerbating the underlying dysfunctions. For example, if a consultant mistakenly labels the fact that project managers did not follow processes as the cause, the organisation may put in place procedures that forces managers to comply with processes. That, as you will no doubt appreciate,  is doing exactly the wrong thing – it will only make things worse.”

“Why is it the wrong thing? Surely if they are forced to comply, they will and the processes will then be followed as they should be.”

“Ah, Watson,” said Holmes, shaking his head in exasperation, “that’s the army man in you talking.” He continuted sharply,  “This is not the military, Sir! This is the messy world of organisation-land where people are autonomous agents even though management orthodoxy would have us believe otherwise.”

“’Tis a matter of discipline, Holmes. Surely you do not advocate letting project managers behave as they would want – as, how do you say it…autonomous agents.”

“ You know Watson, may be you are right,”  said Holmes.  “Perhaps when a man has special knowledge and special powers like my own, it rather encourages him to seek a complex explanation when a simpler one is at hand.”

“Indeed, I think you are over-complicating matters my dear Holmes. This is an open and shut case – a failure of enforcement and compliance.” said Watson.

“Possibly, Watson. However, the truth is not to be found here in the PMO. It lies elsewhere, in the hallowed heights of the executive floor… Anyway there is a more immediate matter that needs our attention: it is late and the sun sinks rapidly. We must make our way to that fine establishment I noticed at the end of the street – I could do with a pint or three.”

“Well said, Holmes!”

The two made their way towards the exit.

———

“Come,  friend Watson, the curtain rings up for the last act,” murmured Holmes, as the two of them entered the elevator. They had come to the head office to meet the executive director.

The two found their way to the meeting room on the executive floor and entered.

“Hello Holmes, it is good to see you again,” boomed the executive director, “and I see you have brought Dr. Watson with you. Good to see you too, sir. Do come in and meet my management team.”.

After the mandatory round of introductions and business card exchanges, the director continued,”I take it you have something for us, Holmes.”

“Yes sir, I have a number of questions.“

“Questions? I don’t understand, Holmes. We hired you to find us some answers about the failure of our PMO, and you tell me have  a few questions. I take it you have some answers too. The CIO expects answers not questions,” he said with a nervous chuckle.

“No,I have no answers…but a hypothesis that I hope to validate soon.”

“I do not understand the need for this drama,” said the director.

“Watson here will tell you that I can never resist a touch of the dramatic.”

“OK, Holmes, you had better get to it then,” said the director shortly.

“I’ll get right to it sir,” he said, and turned to face the seated managers. “Ladies and gentlemen, pray what was the objective of your PMO?”

There was a stunned silence. Finally, one of the managers spoke up, “Surely that is obvious Mr, Holmes.”

“Thank you.  I do realise my question may seem a little simple minded to you, but I beg that you answer it in a way that you would to someone who knows nothing about PMOs.” He turned to the executive director for confirmation.

“Yes, yes, answer his question,” said the executive director impatiently.

“OK, if you insist. The basic objective  of the PMO can be summarised in a line. It was to ensure that all our strategic projects are delivered on time, within the agreed budget and to the required standards of quality.  Needless to say,  the PMO failed to deliver: as I recall, out of the 12 strategic projects we have, 8 or 9 are in serious trouble – over budget and/or time by more than 50%,”  said the manager. “That is all the relevant detail… I trust it is not too much  for you, Mr Holmes,” he added.

“”I am glad of all details, whether they seem to you to be relevant or not,”  retorted Holmes. Then,  in a gentler tone, he asked, “How exactly was the PMO expected to achieve these objectives?”

The managers looked at each other, nonplussed at the question.

Finally,  one of them asked, “Mr. Holmes, what do you mean by “how”? I do not understand your question…and I think I speak for my colleagues too. We followed the advice of Lord Gartner and Baron McKinsey in setting up our PMO. Among many other things, we are fully aware of the importance of giving a PMO complete authority to oversee and control IT projects across the organisation. I am sure  you are aware that our PMO had implemented a set of proven best practice project and portfolio management standards to ensure control and oversight.”

“Yes, we have seen the process charts…they are impressive indeed,” piped up Watson. Holmes gave him The Look.

“That is so, and the fact that some projects have succeeded shows that the processes do work,” said  another manager.

“My dear sir, results without causes are impressive but assuming a causal link between them, sans proof, is not,” said Holmes. “Let me ask you a simple question, sir. Would you say your organisation is unique – one of a kind?”

“Of course it is,” said the manager. “We have just been voted a ‘best employer’ and we won several industry awards in previous years. Indeed we are unique.”

“…and yet you implement standardised processes?”

“What is your point, Mr. Holmes?”

“Let me spell it out: your organisation is unique, as are your people. Right?”

“Yes,” said the manager. Others around the room were nodding their assent.

“In view of your uniqueness, don’t you think you ought to develop – rather evolve – your own unique  processes in collaboration with your project managers rather than impose one-size-fits-all “best practice” standards on them?”

“But…why should we do that…and how ?”  Asked the executive director.

“Sir, I’ve already answered the “why.” I will leave the “how” for you and your team to figure out. Whatever else you do,  I  cannot overemphasise the importance of including your frontline managers and employees in the discussions about how your PMO should function,  and also in selecting and designing appropriate processes.”

“I see…,” said the director thoughtfully.

“Sir, your PMO failed because it attempted to transplant practices that allegedly worked elsewhere into your unique –dare I say, special – organisation. As was inevitable, the transplant was roundly rejected: your people found the processes strange, even arbitrary, and resented them. Consequently, they found ways to work around them instead of with them. Failure of your PMO was preordained because of your focus on processes rather than intentions.

The executive director nodded thoughtfully, as the penny dropped. “Thank you Holmes,” he said, “I see your point….finally.”

“Thank you sir…and thank you all,” said Holmes nodding at each of the seated managers in turn. “There is much work for you all to do now, so Dr. Watson and I will show ourselves out.”

The two gathered their papers and left, shutting the door behind them gently.

“Never underestimate the power of a question to illuminate the truth,” said Holmes sententiously as he and Watson entered the elevator.

Watson rolled his eyes; his friend was brilliant, but he could also be a pompous ass.

——–

Acknowledgements:

Thanks to Arati Apte and Paul Culmsee for encouragement and feedback on earlier drafts of this story.

Notes:

1. Spot the quote (for Sherlock Holmes trainspotters):  there are eight quotes from various Sherlock Holmes adventures in this post; most are verbatim, but a couple of the longer ones have been adapted to fit the narrative.
2. If you enjoyed this piece, you might want to have a look  at the other business fables on this blog.

On the disconnect between business intelligence and strategic decision-making

One of the stated aims of business intelligence (BI) systems is to support better business decision making in organisations (see the Wikipedia article on BI, for example). However, as I have discussed in an earlier post, the usefulness of BI systems in making decisions regarding complex or ambiguous matters is moot. Quoting from that post:

…many decisions [in organisations] have to be made based on incomplete and/or ambiguous information that can be interpreted in a variety of ways. Examples include issues such as what an organization should do in response to increased competition or formulating a sales action plan in a rapidly changing business environment. These issues are wicked; among other things, there is a diversity of viewpoints on how they should be resolved. A business manager and a sales representative are likely to have different views on how sales action plans should be adjusted in response to a changing business environment. The shortcomings of BI systems become particularly obvious when dealing with such problems.

This brings up the question as to how is BI actually used in organisations.

Quoting again from my earlier article:

BI systems are perfectly adequate – even indispensable – for certain situations. Examples of these include, financial reporting (when done right!) and other operational reporting (inventory, logistics etc).  These generally tend to be routine situations with clear cut decision criteria and well-defined processes. Simply put, they are the kinds of decisions that can be programmed.

Typically programmed decisions are made when checking on or monitoring business activities.  I would hazard a guess that BI applications are generally used to carry out such routine monitoring of business processes (and take rule-based corrective action, if necessary) rather than in making complex decisions.  To use a phrase coined by James March, BI applications are used in surveillance mode rather than decision mode.

Unfortunately most BI vendors are yet to address this gap. Most new features that vendors come up with operate in surveillance mode rather than decision mode – that is, they help organisations track (and correct) performance rather than decide on complex/uncertain matters. Thus, despite vendor claims to the contrary, BI is still used as a means to measure and manage  operational matters rather than to make strategic decisions.

Incidentally, this is also true of over-hyped new areas such as Big Data and Predictive Analytics.

Big data refers to a set of technologies and techniques that are useful when analysing large volumes of fast-changing, unstructured data to make operational decisions. For example, commercially available big data products such as splunk can monitor  vast numbers  of unstructured server logs in real time and tell you what corrective actions need to be taken (an operational decision) but they cannot tell you what IT investments you should make over the next five years (a strategic decision).

Predictive analytics refers to a wide range of techniques that are used to identify patterns in past data in order to make predictions about the future events. However, the predictions made using such techniques can only be as good as the underlying mathematical models. Consequently, success in predictive analytics depends crucially on knowing the key variables that govern the phenomena of interest. Identifying these variables can be difficult, if not impossible, in the case of business decisions because of human factors (intentions, motivations etc.). As  Gregory Piatetski-Shapiro puts it in this article, “Predictive analytics can figure out how to land on Mars, but not who will buy a Mars bar.”

So, the question arises:  what do BI vendors need to do in order to facilitate decision-making on complex matters?

To answer this we need to take a brief look at the process of decision-making. The traditional  view is that the decisions are made by working through the following steps:

1. Identifying available options
2. Understanding the consequences of each option.
3. Rating options based on preferences for those consequences
4. Selecting an option (based on rules and ratings)

However, as I have discussed in a post on the nature of decision making in organisations, in the case of complex decisions not only is it hard to identify all options and their consequences, even preferences and/or selection rules may change as one’s knowledge of the options improves.  As a consequence, such decisions necessarily involve informal reasoning  – a deliberative process that takes into account partipants’ values and beliefs,  in addition to logic and  “hard facts”.  The important point, as Tim van Gelder notes in a brilliant post entitled, The missing “I” in BI, is that none of the BI suites in the market support informal reasoning.  The lack of support is especially strange because there are well-known techniques such as Issue Based Information System (IBIS) and Argument Mapping  that can be used to facilitate and capture such reasoning.

This gap does not matter in the case of operational decisions as the choice is made on the basis of straightforward (or programmable) rules, as in steps 1 through 4 above. However, the situation is different in case of complex or non-programmable decisions such as those that are made in the face of uncertainty. In these cases the lack of support for facilitating, capturing and storing decision rationale becomes a huge handicap.

In summary:  Currently available BI tools are good for operational rather than strategic decision-making because they do not offer any support for the deliberative process that is needed to make complex decisions in the face of ambiguity or uncertainty. The adage, “data doesn’t make decisions, people do”  is particularly true for strategic decisions, but it appears  BI vendors are yet to recognise this.

Acknowledgement:

This post was inspired by a recent comment on one of my earlier posts on business intelligence.