On the disconnect between business intelligence and strategic decision-making
One of the stated aims of business intelligence (BI) systems is to support better business decision making in organisations (see the Wikipedia article on BI, for example). However, as I have discussed in an earlier post, the usefulness of BI systems in making decisions regarding complex or ambiguous matters is moot. Quoting from that post:
…many decisions [in organisations] have to be made based on incomplete and/or ambiguous information that can be interpreted in a variety of ways. Examples include issues such as what an organization should do in response to increased competition or formulating a sales action plan in a rapidly changing business environment. These issues are wicked; among other things, there is a diversity of viewpoints on how they should be resolved. A business manager and a sales representative are likely to have different views on how sales action plans should be adjusted in response to a changing business environment. The shortcomings of BI systems become particularly obvious when dealing with such problems.
This brings up the question as to how is BI actually used in organisations.
Quoting again from my earlier article:
BI systems are perfectly adequate – even indispensable – for certain situations. Examples of these include, financial reporting (when done right!) and other operational reporting (inventory, logistics etc). These generally tend to be routine situations with clear cut decision criteria and well-defined processes. Simply put, they are the kinds of decisions that can be programmed.
Typically programmed decisions are made when checking on or monitoring business activities. I would hazard a guess that BI applications are generally used to carry out such routine monitoring of business processes (and take rule-based corrective action, if necessary) rather than in making complex decisions. To use a phrase coined by James March, BI applications are used in surveillance mode rather than decision mode.
Unfortunately most BI vendors are yet to address this gap. Most new features that vendors come up with operate in surveillance mode rather than decision mode – that is, they help organisations track (and correct) performance rather than decide on complex/uncertain matters. Thus, despite vendor claims to the contrary, BI is still used as a means to measure and manage operational matters rather than to make strategic decisions.
Big data refers to a set of technologies and techniques that are useful when analysing large volumes of fast-changing, unstructured data to make operational decisions. For example, commercially available big data products such as splunk can monitor vast numbers of unstructured server logs in real time and tell you what corrective actions need to be taken (an operational decision) but they cannot tell you what IT investments you should make over the next five years (a strategic decision).
Predictive analytics refers to a wide range of techniques that are used to identify patterns in past data in order to make predictions about the future events. However, the predictions made using such techniques can only be as good as the underlying mathematical models. Consequently, success in predictive analytics depends crucially on knowing the key variables that govern the phenomena of interest. Identifying these variables can be difficult, if not impossible, in the case of business decisions because of human factors (intentions, motivations etc.). As Gregory Piatetski-Shapiro puts it in this article, “Predictive analytics can figure out how to land on Mars, but not who will buy a Mars bar.”
So, the question arises: what do BI vendors need to do in order to facilitate decision-making on complex matters?
To answer this we need to take a brief look at the process of decision-making. The traditional view is that the decisions are made by working through the following steps:
- Identifying available options
- Understanding the consequences of each option.
- Rating options based on preferences for those consequences
- Selecting an option (based on rules and ratings)
However, as I have discussed in a post on the nature of decision making in organisations, in the case of complex decisions not only is it hard to identify all options and their consequences, even preferences and/or selection rules may change as one’s knowledge of the options improves. As a consequence, such decisions necessarily involve informal reasoning - a deliberative process that takes into account partipants’ values and beliefs, in addition to logic and “hard facts”. The important point, as Tim van Gelder notes in a brilliant post entitled, The missing “I” in BI, is that none of the BI suites in the market support informal reasoning. The lack of support is especially strange because there are well-known techniques such as Issue Based Information System (IBIS) and Argument Mapping that can be used to facilitate and capture such reasoning.
This gap does not matter in the case of operational decisions as the choice is made on the basis of straightforward (or programmable) rules, as in steps 1 through 4 above. However, the situation is different in case of complex or non-programmable decisions such as those that are made in the face of uncertainty. In these cases the lack of support for facilitating, capturing and storing decision rationale becomes a huge handicap.
In summary: Currently available BI tools are good for operational rather than strategic decision-making because they do not offer any support for the deliberative process that is needed to make complex decisions in the face of ambiguity or uncertainty. The adage, “data doesn’t make decisions, people do” is particularly true for strategic decisions, but it appears BI vendors are yet to recognise this.
This post was inspired by a recent comment on one of my earlier posts on business intelligence.