Eight to Late

Sensemaking and Analytics for Organizations

Archive for October 2012

Free Will – a book review

with 12 comments

Did I write this review because I wanted to, or is it because my background and circumstances compelled me to?

Some time ago, the answer to this question would have been obvious to me but after reading Free Will by Sam Harris, I’m not so sure.

In brief: the book makes the case that the widely accepted notion of free will is little more than an illusion because our (apparently conscious) decisions originate in causes that lie outside of our conscious control.

Harris begins by noting that the notion of free will is based on the following assumptions:

  1. We could have behaved differently than we actually did in the past.
  2. We are the originators of our present thoughts and actions.

Then, in the space of eighty odd pages (perhaps no more than 15,000 words), he argues that the assumptions are incorrect and looks into some of the implications of his arguments.

The two assumptions are actually interrelated:  if it is indeed true that we are not the originators of our present thoughts and actions then it is unlikely that we could have behaved differently than we did in the past.

A key part of Harris’ argument is the scientifically established fact that we are consciously aware of only a small fraction of the activity that takes place in our brains. This has been demonstrated (conclusively?) by some elegant experiments in neurophysiology.   For example:

  • Activity in the brain’s motor cortex can be detected 300 milliseconds before a person “decides” to move, indicating that the thought about moving arises before the subject is aware of it.
  • Magnetic resonance scanning of certain brain regions can reveal the choice that will be made by a person 7 to 10 seconds before the person consciously makes the decision.

These and other similar experiments pose a direct challenge to the notion of free will: if  my  brain has  already decided on a course before I am aware of it , how can I  claim to be the author of my decisions and, more broadly, my destiny? As Harris puts it:

…I cannot decide what I will think next or intend until a thought or intention arises. What will my next mental state be? I do not know – it just happens. Where is the freedom in that?

The whole notion of free will, he argues, is based on the belief that we control our thoughts and actions.   Harris notes that although we may feel that are in control of the decisions we make, this is but an illusion: we feel that we are free, but this freedom is illusory because our actions are already “decided” before they appear in our consciousness.  To be sure, there are causes underlying our thoughts and actions, but the majority of these lie outside our awareness.

If we accept the above then the role that luck plays in determining our genes, circumstances, environment and attitudes cannot be overstated. Although we may choose to believe that we make our destinies, in reality we don’t.  Some people may invoke demonstrations of willpower – conscious mental effort to do certain things – as proof against Harris’ arguments. However, as Harris notes,

You can change your life and yourself through effort and discipline – but you have whatever capacity for effort and discipline you have in this moment, and not a scintilla more (or less). You are either lucky in this department or you aren’t – and you can’t make your own luck.

Although I may choose to believe that I made the key decisions in my life, a little reflection reveals the tenuous nature of this belief.  Sure,   some decisions I have made resulted in experiences that I would not have had otherwise.   Some of those experiences undoubtedly changed my outlook on life, causing me to do things I would not have done had I not undergone those experiences.  So to that extent, those original choices changed my life.

The question is: could I have decided differently when making those original choices?

Or, considering an even more immediate example:   could I have chosen not to write this review? Or, having written it, could I have chosen not to publish it?

Harris tells us that this question is misguided because you will do what you do. As he states,

…you can do what you decide to do – but you cannot decide what you will decide to do.

We feel that we are free to decide, but the decision we make is the one we make. If we choose to believe that we are free to decide, we are free to do so. However, this is an illusion because our decisions arise from causes that we are unaware of. This is the central point of Harris’ argument.

There are important moral and ethical implications of the loss of free will. For example what happens to the notion of moral responsibility for actions that might harm others? Harris argues that we do not need to invoke the notion of free will in order to see that this is not right – as he tells us, what we condemn in others is the conscious intent to do harm.

Harris is careful to note that his argument against free will does not amount to a laissez-faire approach wherein people are free to do whatever comes to their minds, regardless of consequences for society. As he writes:

….we must encourage people to work to the best of their abilities and discourage free riders wherever we can. And it is wise to hold people responsible for their actions when doing so influences their behavior and brings benefits to society….[however this does not need the] illusion of free will. We need only acknowledge that efforts matter and that people can change. [However] we do not change ourselves precisely – because we have only ourselves with which to do the changing -but we continually influence, and are influenced by, the world around us and the world within us. [italics mine]

Before closing I should mention some shortcomings of the book:

Firstly, Harris does not offer a detailed support for his argument.  Much of what he claims depends on the results of experiments research in neurophysiology that demonstrate the lag between the genesis of a thought in our brains and our conscious awareness of it, yet he describes only a handful experiments detail. That said there are references to many others in the notes.

Secondly, those with training in philosophy may find the book superficial as Harris does not discuss of alternate perspectives on free will.  Such a discussion would have provided much needed balance that some critics have taken him to task for (see this analysis  or this review  for example).

Although the book has the shortcomings I’ve noted, I have to say I enjoyed it because it made me think.  More specifically, it made me think about the way I think.  Maybe it will do the same for you, maybe not  – what happens in your case  may depend on thoughts that are beyond your control.

Written by K

October 28, 2012 at 9:45 pm

Out damn’d SPOT: an essay on data, information and truth in organisations

with 4 comments


Jack: My report tells me that we are on track to make budget this year.

Jill: That’s strange, my report tells me otherwise

Jack: That can’t be. Have you used the right filters?

Jill: Yes – the one’s you sent me yesterday.

Jack: There must be something else…my figures must be right, they come from the ERP system.

Jill: Oh, that must be it then…mine are from the reporting system.

Conversations such as the one above occur quite often in organisation-land.  It is one of the reasons why organisations chase the holy grail of a single point of truth (SPOT): an organisation-wide repository that holds the officially endorsed true version of data, regardless of where it originates from. Such a repository is often known as an Enterprise Data Warehouse (EDW).

Like all holy grails, however, the EDW, is a mythical object that exists in only in the pages of textbooks (and vendor brochures…). It is at best an ideal to strive towards. But, like chasing the end of a rainbow it is an exercise that may prove exhausting and ultimately, futile.

Regardless of whether or not organisations can get to that mythical end of the rainbow – and there are those who claim to have got there – there is a deeper issue with the standard view of data and information that hold sway in organisation-land.   In this post I examine these standard conceptions of data and information and truth, drawing largely on this paper by Bernd Carsten Stahl and a number of secondary sources.

Some truths about data and information

As Stahl observes in his introduction:

Many assume that information is central to managerial decision making and that more and higher quality information will lead to better outcomes. This assumption persists even though Russell Ackoff argued over 40 years ago that it is misleading

The reason for the remarkable persistence of this incorrect assumption is that there is a lack of clarity as to what data and information actually are.

To begin with let’s take a look at what these terms mean in the sense in which they are commonly used in organisations. Data typically refers to raw, unprocessed facts or the results of measurements. Information is data that is imbued with meaning and relevance because it is referred to in a context of interest. For example, a piece of numerical data by itself has no meaning – it is just a number. However, its meaning becomes clear once we are provided a context – for example, that the number is the price of a particular product.

The above seems straightforward enough and embodies the standard view of data and information in organisations. However, a closer look reveals some serious problems. For example, what we call raw data is not unprocessed – the data collector always makes a choice as to what data will be collected and what will not. So in this sense, data already has meaning imposed on it. Further, there is no guarantee that what has been excluded is irrelevant. As another example, decision makers will often use data (relevant or not) just because it is available. This is a particularly common practice when defining business KPIs – people often use data that can be obtained easily rather than attempting to measure metrics that are relevant.

Four perspectives on truth

One of the tacit assumptions that managers make about the information available to them is that it is true.  But what exactly does this mean?  Let’s answer this question by taking a whirlwind tour of some theories of truth.

The most commonly accepted notion of truth is that of correspondence, that a statement is true if it describes something as it actually is.  This is pretty much how truth is perceived in business intelligence: data/information is true or valid if it describes something – a customer, an order or whatever – as it actually is.

More generally, the term correspondence theory of truth refers to a family of theories that trace their origins back to antiquity. According to Wikipedia:

Correspondence theories claim that true beliefs and true statements correspond to the actual state of affairs. This type of theory attempts to posit a relationship between thoughts or statements on one hand, and things or facts on the other. It is a traditional model which goes back at least to some of the classical Greek philosophers such as Socrates, Plato, and Aristotle. This class of theories holds that the truth or the falsity of a representation is determined solely by how it relates to a reality; that is, by whether it accurately describes that reality.

One of the problems with correspondence theories is that they require the existence of an objective reality that can be perceived in the same way by everyone. This assumption is clearly problematic, especially for issues that have a social dimension. Such issues are perceived differently by different stakeholders, and each of these will legitimately seek data that supports their point of view. The problem is that there is often no way to determine which data is “objectively right.” More to the point, in such situations the very notion of “objective rightness” can be legitimately questioned.

Another issue with correspondence theories is that a piece of data can at best be an abstraction of a real-world object or event.  This is a serious issue with correspondence theories in the context of data in organisations. For example, when a sales rep records a customer call, he or she notes down only what is required by the customer management system. Other data that may well be more important is not captured or is relegated to a “Notes” or “Comments” field that is rarely if ever searched or accessed.

Another perspective is offered by the so called consensus theories of truth which assert that true statements are those that are agreed to by the relevant group of people. This is often the way truth is established in organisations. For example, managers may choose to calculate Key Performance Indicators (KPIs )using certain pieces of data that are deemed to be true.  The problem with this is that consensus can be achieved by means that are not necessarily democratic. For example, a KPI definition chosen by a manager may be hotly contested by an employee.  Nevertheless, the employee has to accept it because organisations are typically not democratic. A more significant issue is that  the notion of “relevant group” is problematic because there is no clear criterion by which to define relevance.

Pragmatic theories of truth assert that truth is a function of utility – i.e. a statement is true if it is useful to believe it is so. In other words, the truth of a statement is to be judged by the payoff obtained by believing it to be true.  One of the problems with these theories is that it may be useful for some people to believe in a particular statement while is useful for others to disbelieve it. A good example of such a statement is: there is an objective reality. Scientists may find it useful to believe this whereas social constructionists may not. Closer home, it may be useful for a manager to believe that a particular customer is a good prospect (based on market intelligence, say), but a sales rep who knows the customer is unlikely to switch brands may think it useful to believe otherwise.

Finally, coherence theories of truth tell us that statements that are true must be consistent with a wider set of beliefs. In organisational terms, a piece of information or data that is true only if it does not contradict things that others in the organisation believe to be true. Coherence theories emphasise that the truth of statements cannot be established in isolation but must be evaluated as part of a larger system of statements (or beliefs). For example, managers may believe certain KPIs to be true because they fit in with other things they know about their business.

…And so to conclude

The truth is a slippery beast: what is true and what is not depends on what exactly one means by the truth and, as we have seen, there are several different conceptions of truth.

One may well ask if this matters from a practical point of view.  To put it plainly: should executives, middle managers and frontline employees (not to mention business intelligence analysts and data warehouse designers) worry about philosophical theories of truth?  My contention is that they should, if only to understand that the criteria they use for determining the validity of their data and information are little more than conventions that are easily overturned by taking other, equally legitimate, points of view.

Written by K

October 17, 2012 at 9:11 pm

One evening at the station – a project management fable

with 4 comments

It had been an eventful day but Jack was still not done. He needed to update the schedule to reflect the bad news from the vendor and his development team. Then there was the status report. He was not looking forward to the meeting tomorrow. The sponsor would go ballistic when he heard about the delay.  “I’d better do the status report before I go home,” he thought to himself.  “Damned project is more trouble than it’s worth.  I should never have volunteered for it …”

The phone rang flashing his home number. He ignored it recalling the morning’s conversation with his wife – she was expecting him to be home an hour ago. “S**t, talk about conflicting priorities,” he muttered to himself as he shut down his notebook and stuffed it in to his satchel, “I’ll have to do the status report at home.”

He dashed off to catch the 8:05 train; with luck he might be home in an hour.

But luck was not on his side that evening. He heard the rumble of the departing 8:05 as he bounded down the escalator to the platform. “Oh no, that’s another half hour gone, I should’ve stayed at the office and finished the status report,” he said to himself, shaking his head in annoyance.

He must have said that out loud because a gentle voice from behind him asked, “Tough day at the office, huh?”

He turned around, annoyed at the intrusion, “Excuse me…” he stopped himself mid-sentence when he saw the voice belonged to a wizened old man with twinkling eyes.  Yoda was the word that came to mind. Jack continued in a friendlier tone:  “Yeah, you could say it’s been a hard day.”

“What do you do for a living?” asked the old man.

Jack didn’t want the conversation but didn’t want to offend the old man either, “I manage projects,” he replied.

“Forgive my ignorance,” said the old man, “but what is a project and what does it mean to manage one?”

Jack rolled eyes (well, he didn’t really, but he felt like). “A project is an initiative to create something…like a new product or service within a specified time and budget. And managing it involves doing what’s necessary to make sure the product or service is created on time and within the specified budget.”

“So how do you do that?”

Jack was starting to get a little annoyed at the intrusion and he certainly did not want to get into all that PM 101 stuff, so he simply said, “There are a bunch of standard processes and techniques to run projects efficiently. They work well when done right.”

“How are they working for you right now?” asked the old man.

“Not so well. Why? Do you have some advice to offer?” he enquired pointedly.

“Why aren’t they working well?” persisted the old man, oblivious to Jack’s irritation.

“Oh, all kinds of reasons: politics people etc. – it’s complicated.”

“And you didn’t see these complications before they occurred,” stated the old man, more as a fact than question.

“Obviously I didn’t,” retorted Jack.

“Sorry, I didn’t mean to annoy with these questions but I can’t help but wonder if …”

Jack had had enough. “Please mind your own business,” he said shortly.  “And if you’ll excuse me, I have a half hour before my train arrives and a stack of work to catch up with.”  So saying, he strode to a nearby bench and sat down. Reaching into his bag, he pulled out his notebook and turned his attention to his unfinished status report.

The old man shrugged and wandered off towards the other end of the platform.

Jack, totally engrossed in his reports and plans, paid little attention to the announcements and the hubbub of the platform. He was used to working at the station, and knew that the rumble of his train would alert him that he needed to board.

And so he continued working.

About half an hour later, he noticed that the noise around him had subsided somewhat. That didn’t feel right. He surfaced from his reports and looked around. There were very few people about. “What’s going on?” he muttered to himself.

“Ah, I see you missed the announcements.” He hadn’t noticed the old man approach.

“What announcements?” asked Jack, a tad bewildered.

“Trains have been cancelled because of an accident further down the line,” replied the old man, ‘you’ll need to find some other way to get home…”

“S**t, that’s all I need, doesn’t anything work in this damned place.”

“There were at least a couple of announcements but it appears you missed them,” said the old man.

It was going to be an expensive cab ride home or another half hour wait for a bus followed by a 45 minute commute. Jack, sighing resignedly, stuffed his computer back into his bag and got up to go.

“It is good to pay attention to what is actually happening rather than what you think will happen,” said Yoda.

“Yeah, whatever,” said Jack as he turned and walked away.

“Perhaps there’s a lesson here for your work too:  timetables and plans are not reality” said the old man.

Jack was taken aback. He stopped in his tracks, turned towards the old man and asked, “What do you mean?”

“I think I’ve said enough,” came the reply. “Make of it what you will.”

The old man shuffled away,  leaving Jack to contemplate the long ride home.

Written by K

October 2, 2012 at 6:56 am

%d bloggers like this: