Archive for the ‘Best Practice’ Category
There are two facets to the operation of IT systems and processes in organisations: governance, the standards and regulations associated with a system or process; and execution, which relates to steering the actual work of the system or process in specific situations.
An example might help clarify the difference:
The purpose of project management is to keep projects on track. There are two aspects to this: one pertaining to the project management office (PMO) which is responsible for standards and regulations associated with managing projects in general, and the other relating to the day-to-day work of steering a particular project. The two sometimes work at cross-purposes. For example, successful project managers know that much of their work is about navigate their projects through the potentially treacherous terrain of their organisations, an activity that sometimes necessitates working around, or even breaking, rules set by the PMO.
Governance and steering share a common etymological root: the word kybernetes, which means steersman in Greek. It also happens to be the root word of Cybernetics which is the science of regulation or control. In this post, I apply a key principle of cybernetics to a couple of areas of enterprise IT.
An oft quoted example of a cybernetic system is a thermostat, a device that regulates temperature based on inputs from the environment. Most cybernetic systems are way more complicated than a thermostat. Indeed, some argue that the Earth is a huge cybernetic system. A smaller scale example is a system consisting of a car + driver wherein a driver responds to changes in the environment thereby controlling the motion of the car.
Cybernetic systems vary widely not just in size, but also in complexity. A thermostat is concerned only the ambient temperature whereas the driver in a car has to worry about a lot more (e.g. the weather, traffic, the condition of the road, kids squabbling in the back-seat etc.). In general, the more complex the system and its processes, the larger the number of variables that are associated with it. Put another way, complex systems must be able to deal with a greater variety of disturbances than simple systems.
The law of requisite variety
It turns out there is a fundamental principle – the law of requisite variety– that governs the capacity of a system to respond to changes in its environment. The law is a quantitative statement about the different types of responses that a system needs to have in order to deal with the range of disturbances it might experience.
According to this paper, the law of requisite variety asserts that:
The larger the variety of actions available to a control system, the larger the variety of perturbations it is able to compensate.
V(E) > V(D) – V(R) – K
Where V represents variety, E represents the essential variable(s) to be controlled, D represents the disturbance, R the regulation and K the passive capacity of the system to absorb shocks. The terms are explained in brief below:
V(E) represents the set of desired outcomes for the controlled environmental variable: desired temperature range in the case of the thermostat, successful outcomes (i.e. projects delivered on time and within budget) in the case of a project management office.
V(D) represents the variety of disturbances the system can be subjected to (the ways in which the temperature can change, the external and internal forces on a project)
V(R) represents the various ways in which a disturbance can be regulated (the regulator in a thermostat, the project tracking and corrective mechanisms prescribed by the PMO)
K represents the buffering capacity of the system – i.e. stored capacity to deal with unexpected disturbances.
I won’t say any more about the law of requisite variety as it would take me to far afield; the interested and technically minded reader is referred to the link above or this paper for more (full pdf here).
Implications for enterprise IT
In plain English, the law of requisite variety states that only “variety can absorb variety.” As stated by Anthony Hodgson in an essay in this book, the law of requisite variety:
…leads to the somewhat counterintuitive observation that the regulator must have a sufficiently large variety of actions in order to ensure a sufficiently small variety of outcomes in the essential variables E. This principle has important implications for practical situations: since the variety of perturbations a system can potentially be confronted with is unlimited, we should always try maximize its internal variety (or diversity), so as to be optimally prepared for any foreseeable or unforeseeable contingency.
This is entirely consistent with our intuitive expectation that the best way to deal with the unexpected is to have a range of tools and approaches at ones disposal.
In the remainder of this piece, I’ll focus on the implications of the law for an issue that is high on the list of many corporate IT departments: the standardization of IT systems and/or processes.
The main rationale behind standardizing an IT process is to handle all possible demands (or use cases) via a small number of predefined responses. When put this way, the connection to the law of requisite variety is clear: a request made upon a function such as a service desk or project management office (PMO) is a disturbance and the way they regulate or respond to it determines the outcome.
Requisite variety and the service desk
A service desk is a good example of a system that can be standardized. Although users may initially complain about having to log a ticket instead of calling Nathan directly, in time they get used to it, and may even start to see the benefits…particularly when Nathan goes on vacation.
The law of requisite variety tells us successful standardization requires that all possible demands made on the system be known and regulated by the V(R) term in the equation above. In case of a service desk this is dealt with by a hierarchy of support levels. 1st level support deals with routine calls (incidents and service requests in ITIL terminology) such as system access and simple troubleshooting. Calls that cannot be handled by this tier are escalated to the 2nd and 3rd levels as needed. The assumption here is that, between them, the three support tiers should be able to handle majority of calls.
Slack (the K term) relates to unexploited capacity. Although needed in order to deal with unexpected surges in demand, slack is expensive to carry when one doesn’t need it. Given this, it makes sense to incorporate such scenarios into the repertoire of the standard system responses (i.e the V(R) term) whenever possible. One way to do this is to anticipate surges in demand and hire temporary staff to handle them. Another way is to deal with infrequent scenarios outside the system- i.e. deem them out of scope for the service desk.
Service desk standardization is thus relatively straightforward to achieve provided:
- The kinds of calls that come in are largely predictable.
- The work can be routinized.
- All non-routine work – such as an application enhancement request or a demand for a new system- is dealt with outside the system via (say) a change management process.
All this will be quite unsurprising and obvious to folks working in corporate IT. Now let’s see what happens when we apply the law to a more complex system.
Requisite variety and the PMO
Many corporate IT leaders see the establishment of a PMO as a way to control costs and increase efficiency of project planning and execution. PMOs attempt to do this by putting in place governance mechanisms. The underlying cause-effect assumption is that if appropriate rules and regulations are put in place, project execution will necessarily improve. Although this sounds reasonable, it often does not work in practice: according to this article, a significant fraction of PMOs fail to deliver on the promise of improved project performance. Consider the following points quoted directly from the article:
- “50% of project management offices close within 3 years (Association for Project Mgmt)”
- “Since 2008, the correlated PMO implementation failure rate is over 50% (Gartner Project Manager 2014)”
- “Only a third of all projects were successfully completed on time and on budget over the past year (Standish Group’s CHAOS report)”
- “68% of stakeholders perceive their PMOs to be bureaucratic (2013 Gartner PPM Summit)”
- “Only 40% of projects met schedule, budget and quality goals (IBM Change Management Survey of 1500 execs)”
The article goes on to point out that the main reason for the statistics above is that there is a gap between what a PMO does and what the business expects it to do. For example, according to the Gartner review quoted in the article over 60% of the stakeholders surveyed believe their PMOs are overly bureaucratic. I can’t vouch for the veracity of the numbers here as I cannot find the original paper. Nevertheless, anecdotal evidence (via various articles and informal conversations) suggests that a significant number of PMOs fail.
There is a curious contradiction between the case of the service desk and that of the PMO. In the former, process and methodology seem to work whereas in the latter they don’t.
The answer, as you might suspect, has to do with variety. Projects and service requests are very different beasts. Among other things, they differ in:
- Duration: A project typically goes over many months whereas a service request has a lifetime of days,
- Technical complexity: A project involves many (initially ill-defined) technical tasks that have to be coordinated and whose outputs have to be integrated. A service request typically consists one (or a small number) of well-defined tasks.
- Social complexity: A project involves many stakeholder groups, with diverse interests and opinions. A service request typically involves considerably fewer stakeholders, with limited conflicts of opinions/interests.
It is not hard to see that these differences increase variety in projects compared to service requests. The reason that standardization (usually) works for service desks but (often) fails for PMOs is that the PMOs are subjected a greater variety of disturbances than service desks.
The key point is that the increased variety in the case of the PMO precludes standardisation. As the law of requisite variety tells us, there are two ways to deal with variety: regulate it or adapt to it. Most PMOs take the regulation route, leading to over-regulation and outcomes that are less than satisfactory. This is exactly what is reflected in the complaint about PMOs being overly bureaucratic. The solution simple and obvious solution is for PMOs to be more flexible– specifically, they must be able to adapt to the ever changing demands made upon them by their organisations’ projects. In terms of the law of requisite variety, PMOs need to have the capacity to change the system response, V(R), on the fly. In practice this means recognising the uniqueness of requests by avoiding reflex, cookie cutter responses that characterise bureaucratic PMOs.
The law of requisite variety is a general principle that applies to any regulated system. In this post I applied the law to two areas of enterprise IT – service management and project governance – and discussed why standardization works well for the former but less satisfactorily for the latter. Indeed, in view of the considerable differences in the duration and complexity of service requests and projects, it is unreasonable to expect that standardization will work well for both. The key takeaway from this piece is therefore a simple one: those who design IT functions should pay attention to the variety that the functions will have to cope with, and bear in mind that standardization works well only if variety is known and limited.
As narrated by Dr. John Watson, M.D.…
As my readers are undoubtedly aware, my friend Sherlock Holmes is widely feted for his powers of logic and deduction. With all due modesty, I can claim to have played a small part in publicizing his considerable talents, for I have a sense for what will catch the reading public’s fancy and, perhaps more important, what will not. Indeed, it could be argued that his fame is in no small part due to the dramatic nature of the exploits which I have chosen to publicise.
Management consulting, though far more lucrative than criminal investigation, is not nearly as exciting. Consequently my work has become that much harder since Holmes reinvented himself as a management expert. Nevertheless, I am firmly of the opinion that the long-standing myths exposed by his recent work more than make up for any lack of suspense or drama.
A little known fact is that many of Holmes’ insights into flawed management practices have come after the fact, by discerning common themes that emerged from different cases. Of course this makes perfect sense: only after seeing the same (or similar) mistake occur in a variety of situations can one begin to perceive an underlying pattern.
The conversation I had with him last night is an excellent illustration of this point.
We were having dinner at Holmes’ Baker Street abode when, apropos of nothing, he remarked, “It’s a strange thing, Watson, that our lives are governed by routine. For instance, it is seven in the evening, and here we are having dinner, much like we would on any other day.”
“Yes, it is,” I said, intrigued by his remark. Dabbing my mouth with a napkin, I put down my fork and waited for him to say more.
He smiled. “…and do you think that is a good thing?”
I thought about it for a minute before responding. “Well, we follow routine because we like…or need… regularity and predictability,” I said. “Indeed, as a medical man, I know well that our bodies have built in clocks that drive us to do things – such as eat and sleep – at regular intervals. That apart, routines give us a sense of comfort and security in an unpredictable world. Even those who are adventurous have routines of their own. I don’t think we have a choice in the matter, it’s the way humans are wired.” I wondered where the conversation was going.
Holmes cocked an eyebrow. “Excellent, Watson!” he said. “Our propensity for routine is quite possibly a consequence of our need for security and comfort ….but what about the usefulness of routines – apart from the sense of security we get from them?”
“Hmmm…that’s an interesting question. I suppose a routine must have a benefit, or at least a perceived benefit…else it would not have been made into a routine.”
“Possibly,” said Holmes, “ but let me ask you another question. You remember the case of the failed projects do you not?”
“Yes, I do,” I replied. Holmes’ abrupt conversational U-turns no longer disconcert me, I’ve become used to them over the years. I remembered the details of the case like it had happened yesterday…indeed I should, as it was I who wrote the narrative!
“Did anything about the case strike you as strange?” he inquired.
I mulled over the case, which (in hindsight) was straightforward enough. Here are the essential facts:
The organization suffered from a high rate of project failure (about 70% as I recall). The standard prescription – project post-mortems followed by changes in processes aimed at addressing the top issues revealed – had failed to resolve the issue. Holmes’ insightful diagnosis was that the postmortems identified symptoms, not causes. Therefore the measures taken to fix the problems didn’t work because they did not address the underlying cause. Indeed, the measures were akin to using brain surgery to fix a headache. In the end, Holmes concluded that the failures were a consequence of flawed organizational structures and norms.
Of course flawed structures and norms are beyond the purview of a mere project or program manager. So Holmes’ diagnosis, though entirely correct, did not help Bryant (the manager who had consulted us).
Nothing struck me as unduly strange as went over the facts mentally. No,” I replied, “but what on earth does that have to do with routine?”
He smiled. “I will explain presently, but I have yet another question for you before I do so. Do you remember one of our earliest management consulting cases – the affair of the terminated PMO?”
I replied in the affirmative.
“Well then, you see the common thread running through the two cases, don’t you?” Seeing my puzzled look, he added, “think about it for a minute, Watson, while I go and fetch dessert.”
He went into the kitchen, leaving me to ponder his question.
The only commonality I could see was the obvious one – both cases were related to the failure of PMOs. (Editor’s note: PMO = Project Management Office)
He returned with dessert a few minutes later. “So, Watson,” he said as he sat down, “have you come up with anything?
I told him what I thought.
“Capital, Watson! Then you will, no doubt, have asked yourself the obvious next question. ”
I saw what he was getting at. “Yes! The question is: can this observation be generalised? Do majority of PMOs fail? ”
“Brilliant, Watson. You are getting better at this by the day.” I know Holmes does not intend to sound condescending, but the sad fact is that he often does. “Let me tell you,” he continued, “Research suggests that 50% of PMOs fail within three years of being set up. My hypothesis is that failure rate would be considerably higher if the timeframe is increased to five or seven years. What’s even more interesting is that there is a single overriding complaint about PMOs: the majority of stakeholders surveyed felt that their PMOs are overly bureaucratic, and generally hinder project work.”
“But isn’t that contrary to the aim of a PMO – which, as I understand, is to facilitate project work?” I queried.
“Excellent, my dear Watson. You are getting close to the heart of the matter.
“I am?” To be honest, I was a little lost.
“Ah Watson, don’t tell me you do not see it,” said Holmes exasperatedly.
“I’m afraid you’ll have to explain,” I replied curtly. Really, he could insufferable at times.
“I shall do my best. You see, there is a fundamental contradiction between the stated mission and actual operation of a typical PMO. In theory, they are supposed to facilitate projects, but as far as executive management is concerned this is synonymous with overseeing and controlling projects. What this means is that in practice, PMOs inevitably end up policing project work rather than facilitating it.”
I wasn’t entirely convinced. “May be the reason that PMOs fail is that organisations do not implement them correctly,” I said.
“Ah, the famous escape clause used by purveyors of best practices – if our best practice doesn’t work, it means you aren’t implementing it correctly. Pardon me while I choke on my ale, because that is utter nonsense.”
“Well, one would expect after so many years, these so-called implementation errors would have been sorted out. Yet we see the same poor outcomes over and over again,” said Holmes.
“OK, but then why are PMOs are still so popular with management?”
“Now we come to the crux of matter, Watson,” he said, a tad portentously, “They are popular for reasons we spoke of at the start of this conversation – comfort and security.”
“Comfort and security? I have no idea what you’re talking about.”
“Let me try explaining this in another way,” he said. “When you were a small child, you must have had some object that you carried around everywhere…a toy, perhaps…did you not?”
“I’m not sure I should tell you this Holmes but, yes, I had a blanket”
“A security blanket, I would never have guessed, Watson,” smiled Holmes. “…but as it happens that’s a perfect example because PMOs and the methodologies they enforce are security blankets. They give executives and frontline managers a sense that they are doing something concrete and constructive to manage uncertainty…even though they actually aren’t. PMOs are popular , not because they work (and indeed, we’ve seen they don’t) but because they help managers contain their anxiety about whether things will turn out right. I would not be exaggerating if I said that PMOs and the methodologies they evangelise are akin to lucky charms or fetishes.”
“That’s a strong a statement to make on rather slim grounds,” I said dubiously.
“Is it? Think about it, Watson,” he shot back, with a flash of irritation. “Many (though I should admit, not all) PMOs and methodologies prescribe excruciatingly detailed procedures to follow and templates to fill when managing projects. For many (though again, not all) project managers, managing a project is synonymous with following these rituals. Such managers attempt to force-fit reality into standardised procedures and documents. But tell me, Watson – how can such project management by ritual work when no two projects are the same?”
“That is not all, Watson,” he continued, before I could respond, “PMOs and methodologies enable people to live in a fantasy world where everything seems to be under control. Methodology fetishists will not see the gap between their fantasy world and reality, and will therefore miss opportunities to learn. They follow rituals that give them security and an illusion of efficiency, but at the price of a genuine engagement with people and projects.”
“ I’ll have to think about it,” I said.
“You do that,” he replied , as he pushed back his chair and started to clear the table. Unlike him, I had a lot more than dinner to digest. Nevertheless, I rose to help him as I do every day.
Evening conversations at 221B Baker Street are seldom boring. Last night was no exception.
This tale was inspired David Wastell’s brilliant paper, The fetish of technique: methodology as social defence (abstract only).
“The ‘Holy Grail’ of effective collaboration is creating shared understanding, which is a precursor to shared commitment.” – Jeff Conklin.
“Without context, words and actions have no meaning at all.” – Gregory Bateson.
I spent much of last week attending a class on the TOGAF Enterprise Architecture (EA) framework. Prior experience with IT frameworks such as PMBOK and ITIL had taught me that much depends on the instructor – a good one can make the material come alive whereas a not-so-good one can make it an experience akin to watching grass grow. I needn’t have worried: the instructor was superb, and my classmates, all of whom are experienced IT professionals / architects, livened up the proceedings through comments and discussions both in class and outside it. All in all, it was a thoroughly enjoyable and educative experience, something I cannot say for many of the professional courses I have attended.
One of the things about that struck me about TOGAF is the way in which the components of the framework hang together to make a coherent whole (see the introductory chapter of the framework for an overview). To be sure, there is a lot of detail within those components, but there is a certain abstract elegance – dare I say, beauty – to the framework.
That said TOGAF is (almost) entirely silent on the following question which I addressed in a post late last year:
Why is Enterprise Architecture so hard to get right?
Many answers have been offered. Here are some, extracted from articles published by IT vendors and consultancies:
- Lack of sponsorship
- Not engaging the business
- Inadequate communication
- Insensitivity to culture / policing mentality
- Clinging to a particular tool or framework
- Building an ivory tower
- Wrong choice of architect
It is interesting that the first four issues listed are related to the fact that different stakeholders in an organization have vastly different perspectives on what an enterprise architecture initiative should achieve. This lack of shared understanding is what makes enterprise architecture a socially complex problem rather than a technically difficult one. As Jeff Conklin points out in this article, problems that are technically complex will usually have a solution that will be acceptable to all stakeholders, whereas socially complex problems will not. Sending a spacecraft to Mars is an example of the former whereas an organization-wide ERP (or EA!) project or (on a global scale) climate change are instances of the latter.
Interestingly, even the fifth and sixth points in the list above – framework dogma and retreating to an ivory tower – are usually consequences of the inability to manage social complexity. Indeed, that is precisely the point made in the final item in the list: enterprise architects are usually selected for their technical skills rather than their ability to deal with ambiguities that are characteristic of social complexity.
TOGAF offers enterprise architects a wealth of tools to manage technical complexity. These need to be complemented by a suite of techniques to reconcile worldviews of different stakeholder groups. Some examples of such techniques are Soft Systems Methodology, Polarity Management, and Dialogue Mapping. I won’t go into details of these here, but if you’re interested, please have a look at my posts entitled, The Approach – a dialogue mapping story and The dilemmas of enterprise IT for brief introductions to the latter two techniques via IT-based examples.
<Advertisement > Better yet, you could check out Chapter 9 of my book for a crash course on Soft Systems Methodology and Polarity Management and Dialogue Mapping, and the chapters thereafter for a deep dive into Dialogue Mapping </Advertisement>.
Apart from social complexity, there is the problem of context – the circumstances that shape the unique culture and features of an organization. As I mentioned in my introductory remarks, the framework is abstract – it applies to an ideal organization in which things can be done by the book. But such an organization does not exist! Aside from unique people-related and political issues, all organisations have their own quirks and unique features that distinguish them from other organisations, even within the same domain. Despite superficial resemblances, no two pharmaceutical companies are alike. Indeed, the differences are the whole point because they are what make a particular organization what it is. To paraphrase the words of the anthropologist, Gregory Bateson, the differences are what make a difference.
Some may argue that the framework acknowledges this and encourages, even exhorts, people to tailor the framework to their needs. Sure, the word “tailor” and its variants appear almost 700 times in the version 9.1 of the standard but, once again, there is no advice offered on how this tailoring should be done. And one can well understand why: it is impossible to offer any sensible advice if one doesn’t know the specifics of the organization, which includes its context.
On a related note, the TOGAF framework acknowledges that there is a hierarchy of architectures ranging from the general (foundation) to the specific (organization). However despite the acknowledgement of diversity, in practice TOGAF tends to focus on similarities between organisations. Most of the prescribed building blocks and processes are based on assumed commonalities between the structures and processes in different organisations. My point is that, although similarities are important, architects need to focus on differences. These could be differences between the organization they are working in and the TOGAF ideal, or even between their current organization and others that they have worked with in the past (and this is where experience comes in really handy). Cataloguing and understanding these unique features – the differences that make a difference – draws attention to precisely those issues that can cause heartburn and sleepless nights later.
I have often heard arguments along the lines of “80% of what we do follows a standard process, so it should be easy for us to standardize on a framework.” These are famous last words, because some of the 20% that is different is what makes your organization unique, and is therefore worthy of attention. You might as well accept this upfront so that you get a realistic picture of the challenges early in the game.
To sum up, frameworks like TOGAF are abstractions based on an ideal organization; they gloss over social complexity and the unique context of individual organisations. So, questions such as the one posed in the title of this post are akin to the pseudo-choice between Coke and Pepsi, for the real issue is something else altogether. As Tom Graves tells us in his wonderful blog and book, the enterprise is a story rather than a structure, and its architecture an ongoing sociotechnical drama.
In a recent article in Forbes, Jason Bloomberg asks if Enterprise Architecture (EA) is “completely broken”. He reckons it is, and that EA frameworks, such as TOGAF and the Zachman Framework, are at least partly to blame. Here’s what he has to say about frameworks:
EA generally centers on the use of a framework like The Open Group Architecture Framework (TOGAF), the Zachman Framework™, or one of a handful of others. Yet while the use of such frameworks can successfully lead to business value, [they] tend to become self-referential… that [Enterprise Architects end up] spending all their effort working with the framework instead of solving real problems.
From experience, I’d have to agree: many architects are so absorbed by frameworks that they overlook their prime imperative, which is to deliver tangible value rather than pretty diagrams.
In this post I present six (possibly heretical!) practices that underpin an evolutionary or emergent approach to enterprise architecture. I believe that these will go some way towards addressing the issues that Bloomberg and others bemoan.
Here are they are, in no particular order:
1. Use an incremental approach
As Bloomberg notes in the passage above, many enterprise architects spend a great deal of their time creating blueprints and plans based on frameworks. The problem is, this activity rarely leads to anything of genuine business value because blueprints or plans are necessarily:
- Incomplete – they are high-level abstractions that omit messy, ground-level details. Put another way, they are maps that should not be confused with the territory.
- Static – they are based on snapshots of an organization and its IT infrastructure at a point in time. Even roadmaps that are intended to describe how organisations’ processes and technologies will evolve in time are, in reality, extrapolations from information available at a point in time.
The straightforward way to address this is to shun big architectures upfront and embrace an iterative and incremental approach instead. The Agile movement has shown the way forward for software development. There are many convincing real-life demonstrations of the use of agile approaches to enterprise-scale initiatives (Ralph Hughes’ approach to Agile data warehousing, for example). Such examples suggest that the principles behind agile software development can be fruitfully adapted to the practice of enterprise architecture.
The key principle is to start as small as possible and scale-up from there. Such an approach would enable architects to learn along the way, giving them a better chance of getting their heads around those messy but important details that get overlooked when one considers the entire enterprise upfront. Another benefit is that it would give them adequate time to develop an awareness of the factors that are prone to change frequently and those that will remain relatively static. Most important, though, is that it would offer several opportunities to correct errors and fine tune things at little extra cost.
2. Understand the unique features of your organisation
Enterprise architects tend to be people with a wide experience in technology. As a result they are often treated (and see themselves) as experts. The danger of this attitude is that it can lead them to believe that they have privileged insights into the ills of their organizations. A familiarity with frameworks tends to reinforce this view because frameworks, with their focus on standardization, tend to emphasise the common features of organisations. However, I believe that instead of focusing on similarities an enterprise architect would be better served by focusing on differences – the things that make his or her organisation unique. Only when one develops an understanding of differences can one start to build an architecture that actually supports the organization.
To be sure, there are structures and processes that are common to many organisations (for example, functions such as 1st level support or expense reimbursement processes) and these could potentially benefit from the implementation of standardized designs / practices. However, the identification of such commonalities should be an outcome of research rather than an upfront assumption. And this requires ongoing, open dialogue with key stakeholders in the business (more on that later)
3. Form your own opinions about what works and what doesn’t
The surfeit of standardized “best practice” approaches to EA tends to make enterprise architects intellectually lazy. This is actually a symptom of a wider malaise in IT: I have noticed that professionals in the upper echelons of IT are happy to “outsource their thinking” by following someone else’s advice or procedures without much thought. The problem with this attitude, penalty clauses notwithstanding, is that one cannot “outsource the blame” when things go wrong.
I therefore believe one of the key attributes of a good architect is to develop a critical attitude towards frameworks, “best practices” and even consulting advice (especially if it comes from a Big 4 consultancy or guru). If you’re really as experienced as you claim to be, you need to develop your own opinions of what works and what doesn’t and be willing to subject those opinions to scrutiny by others.
4. Understand your organisation’s constraints
Humans tend to be over-optimistic when it comes to planning: witness the developer who blithely tells his boss that it will take a week to do a certain task…and then takes a month because of problems he had not anticipated. Similarly, many architects create designs that look great on paper but are not realistic because they have overlooked potential organizational constraints. Examples of such constraints include:
- Top-down reporting structures in which all decisions have to be made or approved by top management.
- Rigid organizational hierarchies that discourage cross-functional communication and collaboration.
Although constraints can also be technical (such as the limitations of a particular technology), the above examples illustrate that organizational constraints are considerably harder to address, primarily because they involve changing behaviour of influential people within the organization.
Architects lack the positional authority to do anything about such constraints directly. However, they need to develop an understanding of the main constraints so that they can bring them to the attention of those who can do something about them.
5. Focus on problem finding rather than problem solving
In his book, Management in Small Doses, Russell Ackoff wrote that in the real world problems are seldom given; they must be taken. This is good advice for enterprise architects to live by. Indeed, one of the shortcomings of frameworks is that they tend to present the architect with ready-made problems – specifically, any process or technology that does not comply with the framework is seen as a problem. Consequently, many architects spend considerable effort in fixing such “deviations”. However, non-conforming processes or technologies are seldom the most pressing problems that the organization faces. Instead, an enterprise architect will be better served by of finding problems that actually need fixing.
6. Understand the social implications of enterprise architecture
Enterprise architecture is seen as a primarily technical undertaking. As a result architects often overlook the social implications of their designs. Here are a couple of common ways in which this happens:
- Enterprise architecture invariably involves trade-offs. When evaluating these, architects typically focus on economic variables such as cost, productivity, efficiency, throughput etc., ignoring human factors such as ease of use and user preferences.
- Any design choice will (usually) benefit certain stakeholders while disadvantaging others.
Architects need to remember that their plans and designs are going to change the way people work. Nobody likes change without consultation, so it critical to seek as wide a spectrum of opinions as possible before making important design decisions. One can never satisfy everyone, but one has a better chance of making sensible compromises if one knows where all the stakeholders stand.
To summarise: enterprise architects need to take an emergent approach to design. Such an approach proceeds in an incremental fashion, pays due attention to the unique features and constraints of an organization, focuses on real rather than imagined problems…and, above all, acknowledges that the success or failure of enterprise architecture rests neither on frameworks nor technology, but on people. My (admittedly limited) experience suggests that such an approach would be more fruitful than the framework-driven approaches that have come to dominate the profession.
I’d greatly appreciate your feedback. Please be gentle though, this is a first attempt 🙂
PS: I’ve just realised this is the shortest post I’ve ever written.
What is business intelligence?
I recently asked a few acquaintances to answer this question without referring to that great single point of truth in the cloud. They duly came up with a variety of responses ranging from data warehousing and the names of specific business intelligence tools to particular functions such as reporting or decision support.
After receiving their responses, I did what I asked my respondents not to: I googled the term. Here are a few samples of what I found:
According to CIO magazine, Business intelligence is an umbrella term that refers to a variety of software applications used to analyze an organization’s raw data.
Wikipedia, on the other hand, tells us that BI is a set of theories, methodologies, architectures, and technologies that transform raw data into meaningful and useful information for business purposes.
Finally, Webopedia, tell us that BI [refers to] the tools and systems that play a key role in the strategic planning process of the corporation.
What’s interesting about the above responses and definitions is that they focus largely on processes and methodologies or tools and techniques. Now, without downplaying the importance of either, I think that many of the problems of business intelligence practice come from taking a perspective that is overly focused on methodology and technique. In this post, I attempt to broaden this perspective by making some potentially controversial statements –or heresies – that challenge this view. My aim is not so much to criticize current practice as to encourage – or provoke – business intelligence professionals to take a closer look at some of the assumptions underlie their practices.
Without further ado, here are my six heresies for business intelligence practice (in no particular order).
A single point of truth is a mirage
Many organisations embark on ambitious programs to build enterprise data warehouses – unified data repositories that serve as a single source of truth for all business-relevant data. Leaving aside the technical and business issues associated with establishing definitive data sources and harmonizing data, there is the more fundamental question of what is meant by truth.
The most commonly accepted notion of truth is that information (or data in a particular context) is true if it describes something as it actually is. A major issue with this viewpoint is that data (or information) can never fully describe a real-world object or event. For example, when a sales rep records a customer call, he or she notes down only what is required by the customer management system. Other data that may well be more important is not captured or is relegated to a “Notes” or “Comments” field that is rarely if ever searched or accessed. Indeed, data represents only a fraction of the truth, however one chooses to define it – more on this below.
Some might say that it is naïve to expect our databases to capture all aspects of reality, and that what is needed is a broad consensus between all relevant stakeholders as to what constitutes the truth. The problem with this is that such a consensus is often achieved by means that are not democratic. For example, a KPI definition chosen by a manager may be hotly contested by an employee. Nevertheless, the employee has to accept it because that is the way (many) organisations work. Another significant issue is that the notion of relevant stakeholders is itself problematic because it is often difficult to come up with clear criterion by which to define relevance.
There are other ways to approach the notion of truth: for example, one might say that a piece of data is true as long as it is practically useful to deem it so. Such a viewpoint, though common, is flawed because utility is in the eye of the beholder: a sales manager may think it useful to believe a particular KPI whereas a sales rep might disagree (particularly if the KPI portrays the rep in a bad light!).
These varied interpretations of what constitute a truth have implications for the notion of a single point of truth. For one, the various interpretations are incommensurate – they cannot be judged by the same standard. Further, different people may interpret the same piece of data differently. This is something that BI professionals have likely come across – say when attempting to come up with a harmonized definition for a customer record.
In short: the notion of a single point of truth is problematic because there is a great deal of ambiguity about what constitutes a truth.
There is no such thing as raw data
In his book, Memory Practices in the Sciences, Geoffrey Bowker wrote, “Raw data is both an oxymoron and a bad idea; to the contrary, data should be cooked with care.” I love this quote because it tells a great truth (!) about so-called “raw” data.
To elaborate: raw data is never unprocessed. Firstly, the data collector always makes a choice as to what data will be collected and what will not. So in this sense, data already has meaning imposed on it. Second, and perhaps more important, the method of collection affects the data. For example, responses to a survey depend on how the questions are framed and how the survey itself is carried out (anonymous, face-to-face etc.). This is also true for more “objective” data such as costs and expenses. In both cases, the actual numbers depend on specific accounting practices used in the organization. So, raw data is an oxymoron because data is never raw, and as Bowker tells us, we need to ensure that the filters we apply and the methods of collection we use are such that the resulting data is “cooked with care.”
In short: data is never raw, it is always “cooked.”
There are no best practices for business intelligence, only appropriate ones
Many software shops and consultancies devise frameworks and methodologies for business intelligence which they claim are based on best or proven practices. However, those who swallow that line and attempt to implement the practices often find that the results obtained are far from best.
I have discussed the shortcomings of best practices in a general context in an earlier article, and (at greater length) in my book. A problem with best practice approaches is that they assume a universal yardstick of what is best. As a corollary, this also suggest that practices can be transplanted from one organization to another in a wholesale manner, without extensive customisation. This overlooks the fact that organisations are unique, and what works in one may not work in another.
A deeper issue is that much of the knowledge pertaining to best practices is tacit – that is, it cannot be codified in written form. Indeed, what differentiates good business intelligence developers or architects from great ones is not what they learnt from a textbook (or in a training course), but how they actually practice their craft. These consist of things that they do instinctively and would find hard to put into words.
So, instead of looking to import best practices from your favourite vendor, it is better to focus on understanding what goes on in your environment. A critical examination of your environment and processes will reveal opportunities for improvement. These incremental improvements will cumulatively add up to your very own, customized “best practices.”
In short: develop your own business intelligence best practices rather than copying those peddled by “experts.”
Business intelligence does not support strategic decision-making
One of the stated aims of business intelligence systems is to support better business decision making in organisations (see the Wikipedia article, for example). It is true that business intelligence systems are perfectly adequate – even indispensable – for certain decision-making situations. Examples of these include, financial reporting (when done right!) and other operational reporting (inventory, logistics etc). These generally tend to be routine situations with clear cut decision criteria and well-defined processes – i.e. decisions that can be programmed.
In contrast, decisions pertaining to strategic matters cannot be programmed. Examples of such decisions include: dealing with an uncertain business environment, responding to a new competitor etc. The reason such decisions cannot be programmed is that they depend on a host of factors other than data and are generally made in situations that are ambiguous. Typically people use deliberative methods – i.e. methods based on argumentation – to arrive at decisions on such matters. The sad fact is that all the major business tools in the market lack support for deliberative decision-making. Check out this post for more on what can be done about this.
In short: business intelligence does not support strategic decision-making .
Big data is not the panacea it is trumpeted to be
One of the more recent trends in business intelligence is the move towards analyzing increasingly large, diverse, rapidly changing datasets – what goes under the umbrella term big data. Analysing these datasets entails the use of new technologies (e.g. Hadoop and NoSQL) as well as statistical techniques that are not familiar to many mainstream business intelligence professionals.
Much has been claimed for big data; in fact, one might say too much. In this article Tim Harford (aka the Undercover Economist) summarises the four main claims of “big data cheerleaders” as follows (the four phrases below are quoted directly from the article):
- Data analysis produces uncannily accurate results.
- Every single data point can be captured, making old statistical sampling techniques obsolete.
- It is passé to fret about what causes what, because statistical correlation tells us what we need to know.
- Scientific or statistical models aren’t needed.
The problem, as Harford points out, is that all of these claims are incorrect.
Firstly, the accuracy of the results that come out of a big data analysis depend critically on how the analysis is formulated. However, even analyses based on well-founded assumptions can get it wrong, as is illustrated in this article about Google Flu Trends.
Secondly, it is pretty obvious that it is impossible to capture every single data point (also relevant here is the discussion on raw data above – i.e. how data is selected for inclusion).
The third claim is simply absurd. The fact is detecting a correlation is not the same as understanding what is going on – a point made rather nicely by Dilbert. Enough said, I think.
Fourthly, the claim that scientific or statistical models aren’t needed is simply ill-informed. As any big data practitioner will tell you, big data analysis relies on statistics. Moreover, as mentioned earlier, a correlation-based understanding is no understanding at all – it cannot be reliably extrapolated to related situations without the help of hypotheses and (possibly tentative) models of how the phenomenon under study works.
Finally, as Danah Boyd and Kate Crawford point out in this paper , big data changes the meaning of what it means to know something….and it is highly debatable as to whether these changes are for the better. See the paper for more on this point. (Acknowledgement: the title of this post is inspired by the title of the Boyd-Crawford paper).
In short: business intelligence practitioners should not uncritically accept the pronouncements of big data evangelists and vendors.
Business intelligence has ethical implications
This heresy applies to much more than business intelligence: any human activity that affects other people has an ethical dimension. Many IT professionals tend to overlook this facet of their work because they are unaware of it – and sometimes prefer to remain so. Fact is, the decisions business intelligence professionals make with respect to usability, display, testing etc. have a potential impact on the people who use their applications. The impact may be as trivial as having to click a button or filter too many before they get their report, to something more significant, like a data error that leads to a poor business decision.
In short: business intelligence professionals ought to consider how their artefacts and applications affect their users.
This brings me to the end of my heresies for business intelligence. I suspect there will be a few practitioners who agree with me and (possibly many) others who don’t…and some of the latter may even find specific statements provocative. If so, I consider my job done, for my intent was to get business intelligence practitioners to question a few unquestioned tenets of their profession.
Sean yawned as he powered down his computer and stretched out in his chair. It was nearly 3 am and he had just finished proofreading his presentation for later that day. He didn’t remember ever being this tired; a great deal of effort had been expended over the last three months but it had been worth it. Now, finally, he was done.
He gazed down at the beautifully bound document on his desk with a fondness that lesser mortals might bestow on their progeny.
“That’s a fine looking document you have there,” said an oddly familiar voice from right behind his chair.
“Wha..,” squeaked Sean, shooting out of his chair, upending his coffee mug in the process.
He grabbed a couple of tissues and dabbed ineffectually at the coffee stain that was spreading rapidly across the front of his brand new chinos. “Damn,” he cursed as he looked up to find himself face-to-face with someone who looked just like him – right down to the Ralph Lauren shirt and Chinos (minus the unseemly stain).
“Pardon me,” sputtered the apparition, giving in to a fit of laughter. “That’s funniest thing I’ve seen in a long time, a scene worthy of million YouTube hits. You should’ve seen yourself jump out the chair and…”
“Pardon my rudeness, but who the f**k are you?” interrupted Sean, a tad testily. Who did this guy think he was anyway? (Lest you get the wrong idea, Sean didn’t normally use expletives, but he reckoned the situation merited it.)
“Don’t swear at me,” said the double, “because I am you…well, your conscience actually. But, in the end I’m still you.”
“Bah,” replied Sean. He figured this had to be a prank hatched by one of his workmates. “Tell me which one of my smartarse colleagues put you up to this?” he demanded, “Let me guess; it is either Mal or Liz.”
“You don’t believe me, do you? No one put me up to this. Well actually, if anyone did, it was you!”
“That’s nonsense,” spat Sean, his blood pressure rising a notch, “I have no idea who you are.”
“Ah, now we get to the nub of the matter,” said the apparition, “You have no idea who I am, and that is precisely why I’m here: to remind you that I exist and that you should listen to me from time to time. I usually start to bother you when you’re are about to do something stupid or unethical.”
“Me? Stupid? Unethical? I have no idea what you’re on about,” contested Sean.
“It appears I need to spell out for you. Well here’s the executive summary: I think you need to revise that document you’ve been working on. I’m your conscience, and I think I can help.”
“I… don’t… need… your… help,” said Sean enunciating each word exaggeratedly for emphasis, “you probably do not know this, but I have completed the biggest and most ambitious design I’ve ever done: a comprehensive systems architecture for Big Enterprise. I’m confident of what I have proposed because it is backed by solid research and industry best practice.”
“I know what you have done,” said the doppelganger, “I’m your conscience, after all.” He paused to clear his throat. “And I’m sure you believe what you have written, “he continued, “but that doesn’t make it right.”
“It is impeccably researched! You know, I’ve cited over 800 references, yeah eight hundred,” said Sean with pride. “That’s over two references per page, and most of these are to works written by acknowledged experts in the field.”
“I do not doubt your knowledge or diligence, my friend,” said the doppelganger with a smile, “what I worry about is your judgement.”
Sean was ready to blow a fuse, but was also confused (intrigued?) by the double’s choice of words. “Judgement?” he blurted, “WTF do you mean by ‘judgement?” He picked up the tome and waved it in front of the doppelganger imperiously…but then spoilt the effect by almost spraining his wrist in the process. He put the book down hurriedly saying, “this is an objective evaluation; the facts speak for themselves.”
“Do they?” queried the apparition. Sure, you’ve collected all this information and have fashioned into a coherent report. However, your recommendations, which appear to be based on facts, are in truth based on unverifiable assumptions, even opinions.”
“That’s nonsense,” dismissed Sean. “You haven’t even read the report, so you’re in no position to judge.”
“I have. I’m your conscience, remember?”
“OK, so tell me what you did and how you did it,” said the apparition evenly.
Sean held forth for a few minutes, describing how he researched various frameworks, read case studies about them and then performed an evaluation based on criteria recommended by experts.
“I concede you that you truly believe you are right, but the sad fact is that you probably aren’t,” said the double, “and worse, you refuse to entertain that possibility.”
“That’s hogwash! If you’re so sure then prove it,” countered Sean.
“Hmmm, you are thicker than I thought, let me see if I can get my point across in a different way,” said the double. “You’re doing something that will influence the future of technology in your organisation for a long time to come. That is an immense responsibility…”
“I’m aware of that, thank you,” interrupted Sean, raising his voice. He’d had enough of this presumptuous, insulting clown.
“If you say so,” said the doppelganger, “but, to be honest, I sense no doubts and see no caveats in your report.”
“That’s because I have none! I believe in and stand by what I have done,” shouted Sean.
“I have no doubt that you believe in what you have done. The question is, do others, will others?”
“I’m not stupid,” said Sean, “I’ve kept my managers and other key stakeholders in the loop throughout. They know what my recommendations are, and they are good with them.”
“How many stakeholders, and where are they located?”
“Over ten important stakeholders, senior managers, all of them, and all seated right here in the corporate head office,” said Sean. He made to pick up the tome again, but a twinge in his wrist reminded him that it might not be wise to do so. “Let me tell you that the feedback I have from them is that this is a fantastic piece of work,” he continued, emphasizing his point by rapping on the wrist-spraining tome with his knuckles. “So please go away and leave me to finish up my work.”
“Yeah, I’ll go, it seems you have no need of me,” said the double, “but allow me a couple of questions before I go. I am your conscience after all!”
“Ok, what is it?” said Sean impatiently. He couldn’t wait to see the back of the guy.
“You’re working in a multinational right? But you’ve spoken to a few stakeholders all of whom are sitting right here, in this very building. Have you travelled around and spoken with staff in other countries – say in Asia and Europe – and gotten to know their problems before proposing your all-embracing architecture?”
“Look,” said Sean, “it is impossible to talk to everyone, so, I have done the best I can: I have proposed a design that adheres to best practices, and that means my design is fundamentally sound,” asserted Sean. “Moreover, the steering committee has reviewed it, and has indicated that it will be approved.”
“I have no doubt that it will be approved,” said the apparition patiently, “the question is: what then? Think about it, you have proposed an architecture for your enterprise without consulting the most important stakeholders – the people who will actually live it and work with it. Would you have an architect build your house that way? And how would you react to one who insisted on doing things his or her way because it is “best practice” to do so?”
“That’s a completely inappropriate comparison,” said Sean.
“No it isn’t, and you know it too” said the doppelganger. “But look, I’ve nothing more to add. I’ve said what I wanted to say. Besides, I’m sure you’re keen to see the back of me…most people are.”
…and pfft…just like that, the apparition vanished, leaving a bemused architect and a rapidly drying coffee stain in its wake.