Eight to Late

Sensemaking and Analytics for Organizations

Posts Tagged ‘Information Management

Out damn’d SPOT: an essay on data, information and truth in organisations

with 4 comments

Introduction

Jack: My report tells me that we are on track to make budget this year.

Jill: That’s strange, my report tells me otherwise

Jack: That can’t be. Have you used the right filters?

Jill: Yes – the one’s you sent me yesterday.

Jack: There must be something else…my figures must be right, they come from the ERP system.

Jill: Oh, that must be it then…mine are from the reporting system.

Conversations such as the one above occur quite often in organisation-land.  It is one of the reasons why organisations chase the holy grail of a single point of truth (SPOT): an organisation-wide repository that holds the officially endorsed true version of data, regardless of where it originates from. Such a repository is often known as an Enterprise Data Warehouse (EDW).

Like all holy grails, however, the EDW, is a mythical object that exists in only in the pages of textbooks (and vendor brochures…). It is at best an ideal to strive towards. But, like chasing the end of a rainbow it is an exercise that may prove exhausting and ultimately, futile.

Regardless of whether or not organisations can get to that mythical end of the rainbow – and there are those who claim to have got there – there is a deeper issue with the standard view of data and information that hold sway in organisation-land.   In this post I examine these standard conceptions of data and information and truth, drawing largely on this paper by Bernd Carsten Stahl and a number of secondary sources.

Some truths about data and information

As Stahl observes in his introduction:

Many assume that information is central to managerial decision making and that more and higher quality information will lead to better outcomes. This assumption persists even though Russell Ackoff argued over 40 years ago that it is misleading

The reason for the remarkable persistence of this incorrect assumption is that there is a lack of clarity as to what data and information actually are.

To begin with let’s take a look at what these terms mean in the sense in which they are commonly used in organisations. Data typically refers to raw, unprocessed facts or the results of measurements. Information is data that is imbued with meaning and relevance because it is referred to in a context of interest. For example, a piece of numerical data by itself has no meaning – it is just a number. However, its meaning becomes clear once we are provided a context – for example, that the number is the price of a particular product.

The above seems straightforward enough and embodies the standard view of data and information in organisations. However, a closer look reveals some serious problems. For example, what we call raw data is not unprocessed – the data collector always makes a choice as to what data will be collected and what will not. So in this sense, data already has meaning imposed on it. Further, there is no guarantee that what has been excluded is irrelevant. As another example, decision makers will often use data (relevant or not) just because it is available. This is a particularly common practice when defining business KPIs – people often use data that can be obtained easily rather than attempting to measure metrics that are relevant.

Four perspectives on truth

One of the tacit assumptions that managers make about the information available to them is that it is true.  But what exactly does this mean?  Let’s answer this question by taking a whirlwind tour of some theories of truth.

The most commonly accepted notion of truth is that of correspondence, that a statement is true if it describes something as it actually is.  This is pretty much how truth is perceived in business intelligence: data/information is true or valid if it describes something – a customer, an order or whatever – as it actually is.

More generally, the term correspondence theory of truth refers to a family of theories that trace their origins back to antiquity. According to Wikipedia:

Correspondence theories claim that true beliefs and true statements correspond to the actual state of affairs. This type of theory attempts to posit a relationship between thoughts or statements on one hand, and things or facts on the other. It is a traditional model which goes back at least to some of the classical Greek philosophers such as Socrates, Plato, and Aristotle. This class of theories holds that the truth or the falsity of a representation is determined solely by how it relates to a reality; that is, by whether it accurately describes that reality.

One of the problems with correspondence theories is that they require the existence of an objective reality that can be perceived in the same way by everyone. This assumption is clearly problematic, especially for issues that have a social dimension. Such issues are perceived differently by different stakeholders, and each of these will legitimately seek data that supports their point of view. The problem is that there is often no way to determine which data is “objectively right.” More to the point, in such situations the very notion of “objective rightness” can be legitimately questioned.

Another issue with correspondence theories is that a piece of data can at best be an abstraction of a real-world object or event.  This is a serious issue with correspondence theories in the context of data in organisations. For example, when a sales rep records a customer call, he or she notes down only what is required by the customer management system. Other data that may well be more important is not captured or is relegated to a “Notes” or “Comments” field that is rarely if ever searched or accessed.

Another perspective is offered by the so called consensus theories of truth which assert that true statements are those that are agreed to by the relevant group of people. This is often the way truth is established in organisations. For example, managers may choose to calculate Key Performance Indicators (KPIs )using certain pieces of data that are deemed to be true.  The problem with this is that consensus can be achieved by means that are not necessarily democratic. For example, a KPI definition chosen by a manager may be hotly contested by an employee.  Nevertheless, the employee has to accept it because organisations are typically not democratic. A more significant issue is that  the notion of “relevant group” is problematic because there is no clear criterion by which to define relevance.

Pragmatic theories of truth assert that truth is a function of utility – i.e. a statement is true if it is useful to believe it is so. In other words, the truth of a statement is to be judged by the payoff obtained by believing it to be true.  One of the problems with these theories is that it may be useful for some people to believe in a particular statement while is useful for others to disbelieve it. A good example of such a statement is: there is an objective reality. Scientists may find it useful to believe this whereas social constructionists may not. Closer home, it may be useful for a manager to believe that a particular customer is a good prospect (based on market intelligence, say), but a sales rep who knows the customer is unlikely to switch brands may think it useful to believe otherwise.

Finally, coherence theories of truth tell us that statements that are true must be consistent with a wider set of beliefs. In organisational terms, a piece of information or data that is true only if it does not contradict things that others in the organisation believe to be true. Coherence theories emphasise that the truth of statements cannot be established in isolation but must be evaluated as part of a larger system of statements (or beliefs). For example, managers may believe certain KPIs to be true because they fit in with other things they know about their business.

…And so to conclude

The truth is a slippery beast: what is true and what is not depends on what exactly one means by the truth and, as we have seen, there are several different conceptions of truth.

One may well ask if this matters from a practical point of view.  To put it plainly: should executives, middle managers and frontline employees (not to mention business intelligence analysts and data warehouse designers) worry about philosophical theories of truth?  My contention is that they should, if only to understand that the criteria they use for determining the validity of their data and information are little more than conventions that are easily overturned by taking other, equally legitimate, points of view.

Written by K

October 17, 2012 at 9:11 pm

The unspoken life of information in organisations

with 5 comments

Introduction

Many activities in organisations are driven by information. Chief among these is decision-making : when faced with a decision, those involved will seek information on the available choices and their (expected) consequences. Or so the theory goes.

In reality, information plays a role that does not quite square up with this view. For instance, decision makers may expend considerable time and effort in gathering information, only to ignore it when making their choices.  In this case information plays a symbolic role, signifying competence of the decision-maker (the volume of information being a measure of competence) rather than being a means of facilitating a decision. In this post I discuss such common but unspoken uses of information in organisations, drawing on a paper by James March and Martha Feldman entitled Information in Organizations as Symbol and Signal.

Information perversity

As I have discussed in an earlier post, the standard view of decision-making is that choices are based on an analysis of their consequences and (the decision-maker’s) preferences for those consequences.  These consequences and preferences generally refer to events in the future and are therefore uncertain. The main role of information is to reduce this uncertainty.  In such a rational paradigm, one would expect that  information gathering and utilization are consistent with the process of decision making.  Among other things this implies that:

  1. The required information is gathered prior to the decision being made.
  2. All relevant information is used in the decision-making process.
  3. All available information is evaluated prior to requesting further information.
  4. Information that is not relevant to a decision is not collected.

In reality, the above expectations are often violated. For example:

  1. Information is gathered selectively after a decision has been made (only information that supports the decision is chosen).
  2. Relevant information is ignored.
  3. Requests for further information are made before all the information at hand is used.
  4. Information that has no bearing on the decision is sought.

On the face of it, such behaviour is perverse – why on earth would someone take the trouble to gather information if they are not going to use it?  As we’ll see next, there are good reasons for such “information perversity”, some of which are obvious but others that are less so.

Reasons for information perversity

There are a couple of straightforward reasons why a significant portion of the information gathered by organisations is never used. These are:

  1. Humans have bounded cognitive capacities, so there is a limit to the amount of information they can process. Anything beyond this leads to information overload.
  2. Information gathered is often unusable in that it is irrelevant to the decision that is to be made.

Although these reasons are valid in many situations, March and Feldman assert that there are other less obvious but possibly more important reasons why information gathered is not used. I describe these in some detail below.

Misaligned incentives

One of the reasons for the mountains of unused information in organisations is that certain groups of people (who may not even be users of information) have incentives to gather information regardless of its utility. March and Feldman describe a couple of scenarios in which this can happen:

  1. Mismatched interests: In most organisations the people who use information are not the same as those who gather and distribute it. Typically, information users tend to be from  business functions (finance, sales, marketing etc.) whereas gatherers/distributors are from IT. Users are after relevant information whereas IT is generally interested in volume rather than relevance. This can result in the collection of data that nobody is going to use.
  2.   “After the fact” assessment of decisions:  Decision makers know that many (most?) of their decisions will later turn out to be suboptimal. In other words,   after-the-fact assessments of their decision may lead to the realisation that those decisions ought to have been made differently. In view of this, decision makers have good reason to try to anticipate as many different outcomes as they can, which leads to them gathering more information than can be used.

Information as measurement

Often organisations collect information to measure performance or monitor their environments. For example, sales information is collected to check progress against targets and employees are required to log their working times to ensure that they are putting in the hours they are supposed to. Information collected in such a surveillance mode is not relevant to any decision except when corrective action is required. Most of the information collected for this purpose is never used even though it could well contain interesting insights

Information as a means to support hidden agendas

People often use information to build arguments that support their favoured positions. In such cases it is inevitable that information will be misrepresented.  Such strategic misrepresentation (aka lying!) can cause more information to be gathered than necessary. As March and Feldman state in the paper:

Strategic misrepresentation also stimulates the oversupply of information. Competition among contending liars turns persuasion into a contest in (mostly unreliable) information. If most received information is confounded by unknown misrepresentations reflecting a complicated game played under conditions of conflicting interests, a decision maker would be curiously unwise to consider information as though it were innocent. The modest analyses of simplified versions of this problem suggest the difficulty of devising incentive schemes that yield unambiguously usable information…

As a consequence, decision makers end up not believing information, especially if it is used or generated by parties that (in the decision-makers’ view) may have hidden agendas.

The above points are true enough. However, March and Feldman suggest that there is a more subtle reason for information perversity in organisations.

The symbolic significance of information

In my earlier post on decision making in organisations I stated that:

…the official line about decision making being a rational process that is concerned with optimizing choices on the basis of consequences and preferences is not the whole story. Our decisions are influenced by a host of other factors, ranging from the rules that govern our work lives to our desires and fears, or even what happened at home yesterday. In short: the choices we make often depend on things we are only dimly aware of.

One of the central myths of modern organisations is that decision making is essentially a rational process.  In reality, decision making is often a ritualised activity consisting of going through the motions of identifying choices, their consequences and our preferences for them.  In such cases, information has a symbolic significance; it adds to the credibility of the decision. Moreover, the greater the volume of information, the greater the credibility (providing, of course, that the information is presented in an attractive format!). Such a process reaffirms the competence of those involved and reassures those in positions of authority that the right decision has been made, regardless of the validity or relevance of the information used.

Information is thus a symbol of rational decision making; it signals (or denotes) competence in decision making and that the decision made is valid.

Conclusion

In this article I have discussed the  unspoken life of information in organisations –  how it is used in ways that do not square up to a rational process of decision making. As March and Feldman put it:

Individuals and organizations invest in information and information systems, but their investments do not seem to make decision-theory sense. Organizational participants seem to find value in information that has no great decision relevance. They gather information and do not use it. They ask for reports and do not read them. They act first and receive requested information later.

Some of the reasons for such “information perversity” are straightforward: they include, limited human cognitive ability, irrelevant information, misaligned incentives and even lying!  But above all, organisations gather information because it symbolises proper decision making behaviour and provides assurance of the validity of decisions, regardless of whether or not decisions are actually made on a rational basis.  To conclude: the official line about information spins a tale about its role in rational decision-making but  the unspoken life of information in organisations tells another story.

Written by K

June 14, 2012 at 5:55 am

%d bloggers like this: