On the shortcomings of cause-effect based models in management

Introduction

Business schools perpetuate the myth that the outcomes of changes in organizations can be managed using  models that  are rooted in the scientific-rational mode of enquiry. In essence, such models assume that all important variables that affect an outcome  (i.e. causes) are known and that the relationship between these variables and the outcomes (i.e.  effects) can be represented accurately by simple models.   This is the nature of explanation in the hard sciences such as physics and is pretty much the official line adopted by mainstream management research and teaching – a point I have explored at length in an earlier post.

Now it is far from obvious that a mode of explanation that works for physics will also work for management. In fact, there is enough empirical evidence that most cause-effect based management models do not work in the real world.  Many front-line employees and middle managers need no proof because they have likely lived through failures of such models in their organisations- for example,  when the unintended consequences of  organisational change swamp its intended (or predicted) effects.

In this post I look at the missing element in management models – human intentions –  drawing on this paper by Sumantra Ghoshal which explores  three different modes of explanation that were elaborated by Jon Elster in this book.  My aim in doing this is to highlight the key reason why so many management initiatives fail.

Types of explanations

According to Elster, the nature of what we can reasonably expect from an explanation differs in the natural and social sciences. Furthermore, within the natural sciences, what constitutes an explanation differs in the physical and biological sciences.

Let’s begin with the difference between physics and biology first.

The dominant mode of explanation in physics (and other sciences that deal with inanimate matter) is causal – i.e. it deals with causes and effects as I have described in the introduction. For example, the phenomenon of gravity is explained as being caused by the presence of  matter, the precise relationship being expressed via Newton’s Law of Gravitation (or even more accurately, via Einstein’s General Theory of Relativity).  Gravity is  “explained” by these models because they tell us that it is caused by the presence of matter. More important, if we know the specific configuration of matter in a particular problem, we can accurately predict the effects of gravity – our success in sending unmanned spacecraft to Saturn or Mars depends rather crucially on this.

In biology, the nature of explanation is somewhat different. When studying living creatures we don’t look for causes and effects. Instead we look for explanations based on function. For example,  zoologists do not need to ask how amphibians came to have webbed feet; it is enough for them to know that webbed feet are an adaptation that affords amphibians a survival advantage. They need look no further than this explanation because it is consistent with the Theory of Evolution – that changes in organisms occur by chance, and those that survive do so because they offer the organism a survival advantage. There is no need to look for a deeper explanation in terms of cause and effect.

In social sciences the situation is very  different indeed. The basic unit of explanation in the social sciences is the individual. But an individual is different from an inanimate object or even a non-human organism that reacts to specific stimuli in predictable ways. The key difference is that human actions are guided by intentions, and any explanation of social phenomena ought to start from these intentions.

For completeness I should mention that functional and causal explanations are sometimes possible within the social sciences and management. Typically functional explanations are possible in tightly controlled environments.  For example,  the behaviour and actions of people working within large bureaucracies or assembly lines can be understood on the basis of function. Causal explanations are even rarer, because they are possible only when  focusing on the collective behaviour of large, diverse populations in which the effects of individual intentions are swamped by group diversity. In such special cases, people can indeed be treated as molecules or atoms.

Implications for management

There a couple of interesting implications of restoring intentionality to its rightful place in management studies.

Firstly, as Ghoshal states in his paper:

Management theories at present are overwhelmingly causal or functional in their modes of explanation. Ethics or morality, however, are mental phenomena. As a result they have had to be excluded from our theories and from the practices that such theories have shaped.  In other words, a precondition for making business studies a science as well as a consequence of the resulting belief in determinism has been the explicit denial of any role of moral or ethical considerations in the practice of management

Present day management studies exclude considerations of morals and ethics, except, possibly, as a separate course that has little relation to the other subjects that form a part of the typical business school curriculum. Recognising the role of intentionality restores ethical and moral considerations where they belong – on the centre-stage of management theory and practice.

Secondly, recognizing the role of intentions in determining peoples’ actions helps us see that organizational changes that “start from where people are”  have a much better chance of succeeding than those that are initiated top-down with little or no consultation with rank and file employees. Unfortunately the large majority of organizational change initiatives still start from the wrong place – the top.

Summing up

Most management practices that are taught in business schools and practiced by the countless graduates of these programs are rooted in the belief that certain actions (causes) will lead to specific, desired outcomes (effects). In this article I have discussed how explanations based on cause-effect models, though good for understanding the behaviour of molecules and possibly even mice, are misleading in the world of humans. To achieve sustainable and enduring outcomes  in organisation one has to start from where people are,  and to do that one has to begin by taking their opinions and aspirations seriously.

Free Will – a book review

Did I write this review because I wanted to, or is it because my background and circumstances compelled me to?

Some time ago, the answer to this question would have been obvious to me but after reading Free Will by Sam Harris, I’m not so sure.

In brief: the book makes the case that the widely accepted notion of free will is little more than an illusion because our (apparently conscious) decisions originate in causes that lie outside of our conscious control.

Harris begins by noting that the notion of free will is based on the following assumptions:

1. We could have behaved differently than we actually did in the past.
2. We are the originators of our present thoughts and actions.

Then, in the space of eighty odd pages (perhaps no more than 15,000 words), he argues that the assumptions are incorrect and looks into some of the implications of his arguments.

The two assumptions are actually interrelated:  if it is indeed true that we are not the originators of our present thoughts and actions then it is unlikely that we could have behaved differently than we did in the past.

A key part of Harris’ argument is the scientifically established fact that we are consciously aware of only a small fraction of the activity that takes place in our brains. This has been demonstrated (conclusively?) by some elegant experiments in neurophysiology.   For example:

• Activity in the brain’s motor cortex can be detected 300 milliseconds before a person “decides” to move, indicating that the thought about moving arises before the subject is aware of it.
• Magnetic resonance scanning of certain brain regions can reveal the choice that will be made by a person 7 to 10 seconds before the person consciously makes the decision.

These and other similar experiments pose a direct challenge to the notion of free will: if  my  brain has  already decided on a course before I am aware of it , how can I  claim to be the author of my decisions and, more broadly, my destiny? As Harris puts it:

…I cannot decide what I will think next or intend until a thought or intention arises. What will my next mental state be? I do not know – it just happens. Where is the freedom in that?

The whole notion of free will, he argues, is based on the belief that we control our thoughts and actions.   Harris notes that although we may feel that are in control of the decisions we make, this is but an illusion: we feel that we are free, but this freedom is illusory because our actions are already “decided” before they appear in our consciousness.  To be sure, there are causes underlying our thoughts and actions, but the majority of these lie outside our awareness.

If we accept the above then the role that luck plays in determining our genes, circumstances, environment and attitudes cannot be overstated. Although we may choose to believe that we make our destinies, in reality we don’t.  Some people may invoke demonstrations of willpower – conscious mental effort to do certain things – as proof against Harris’ arguments. However, as Harris notes,

You can change your life and yourself through effort and discipline – but you have whatever capacity for effort and discipline you have in this moment, and not a scintilla more (or less). You are either lucky in this department or you aren’t – and you can’t make your own luck.

Although I may choose to believe that I made the key decisions in my life, a little reflection reveals the tenuous nature of this belief.  Sure,   some decisions I have made resulted in experiences that I would not have had otherwise.   Some of those experiences undoubtedly changed my outlook on life, causing me to do things I would not have done had I not undergone those experiences.  So to that extent, those original choices changed my life.

The question is: could I have decided differently when making those original choices?

Or, considering an even more immediate example:   could I have chosen not to write this review? Or, having written it, could I have chosen not to publish it?

Harris tells us that this question is misguided because you will do what you do. As he states,

…you can do what you decide to do – but you cannot decide what you will decide to do.

We feel that we are free to decide, but the decision we make is the one we make. If we choose to believe that we are free to decide, we are free to do so. However, this is an illusion because our decisions arise from causes that we are unaware of. This is the central point of Harris’ argument.

There are important moral and ethical implications of the loss of free will. For example what happens to the notion of moral responsibility for actions that might harm others? Harris argues that we do not need to invoke the notion of free will in order to see that this is not right – as he tells us, what we condemn in others is the conscious intent to do harm.

Harris is careful to note that his argument against free will does not amount to a laissez-faire approach wherein people are free to do whatever comes to their minds, regardless of consequences for society. As he writes:

….we must encourage people to work to the best of their abilities and discourage free riders wherever we can. And it is wise to hold people responsible for their actions when doing so influences their behavior and brings benefits to society….[however this does not need the] illusion of free will. We need only acknowledge that efforts matter and that people can change. [However] we do not change ourselves precisely – because we have only ourselves with which to do the changing -but we continually influence, and are influenced by, the world around us and the world within us. [italics mine]

Before closing I should mention some shortcomings of the book:

Firstly, Harris does not offer a detailed support for his argument.  Much of what he claims depends on the results of experiments research in neurophysiology that demonstrate the lag between the genesis of a thought in our brains and our conscious awareness of it, yet he describes only a handful experiments detail. That said there are references to many others in the notes.

Secondly, those with training in philosophy may find the book superficial as Harris does not discuss of alternate perspectives on free will.  Such a discussion would have provided much needed balance that some critics have taken him to task for (see this analysis  or this review  for example).

Although the book has the shortcomings I’ve noted, I have to say I enjoyed it because it made me think.  More specifically, it made me think about the way I think.  Maybe it will do the same for you, maybe not  – what happens in your case  may depend on thoughts that are beyond your control.

On the nonlinearity of organisational phenomena

Introduction

Some time ago I wrote a post entitled, Models and Messes – from best practices to appropriate practices, in which I described the deep connection between the natural sciences and 20^th century management.  In particular, I discussed how early management theorists took inspiration from physics. Quoting from that post:

Given the spectacular success of mathematical modeling in the physical and natural sciences, it is perhaps unsurprising that early management theorists attempted to follow the same approach. Fredrick Taylor stated this point of view quite clearly in the introduction to his classic monograph, The Principles of Scientific Management…Taylor’s intent was to prove that management could be reduced to a set of principles that govern all aspects of work in organizations.

In Taylor’s own words, his goal was to “prove that the best management is a true science, resting upon clearly defined laws, rules and principles, as a foundation. And further to show that the fundamental principles of scientific management  are applicable to all human activities…”

In the earlier post I discussed how organisational problems elude so-called scientific solutions because they are ambiguous and have a human dimension.  Now I continue the thread, introducing a concept from physics that has permeated much of management thinking, much to the detriment of managerial research and practice. The concept is that of linearity. Simply put, linearity is a mathematical expression of the idea that complex systems can be analysed in terms of their (simpler) components.  I explain this notion in more detail in the following sections.

The post is organised as follows: I begin with a brief introduction to linearity in physics and then describe its social science equivalent.  Following this, I discuss a paper that points out some pitfalls of linear thinking in organisational research and (by extrapolation) to management practice.

Linearity in physics and mathematics

A simplifying assumption underlying much of classical physics is that of equilibrium or stability. A characteristic of a system in equilibrium is that it tends to resist change.  Specifically, if such a system is disturbed, it tends to return to its original state. Of course, physics also deals with systems that are not in equilibrium – the weather, or  a spacecraft on its way to Mars  are examples of such systems.  In general, non-equilibrium systems are described by more complex mathematical models than equilibrium systems.

Now, complex mathematical models – such as those describing the dynamics of weather or even the turbulent flow of water-  can only be solved numerically using computers.  The key complicating factor in such models is that they consist of many interdependent variables that are combined in complex ways. 19^th  and early 20th century physicists who had no access to computers had to resort to some tricks in order to make the mathematics of such systems tractable. One of the most common simplifying tricks was to treat the system as being  linear.   Linear systems have mathematical properties that roughly translate to the following in physical terms:

1. Cause is proportional effect (or output is proportional to input).  This property is called homogeneity.
2. Any complex effect can be expressed as a sum of a well defined number of simpler effects.  This property is often referred to as additivity, but I prefer the term decomposability.  This notion of decomposability  is also called the principle of superposition.

In contrast, real-life systems (such as the weather) tend to be described by mathematical equations that do not satisfy the above conditions. Such systems are called nonlinear.

Linear systems are well-understood, predictable and frankly, a bit boring –   they hold no surprises and cannot display novel behaviour. The evolution of linear systems is constrained by the equations and initial conditions (where they start from). Once these are known, their future state is completely determined.  Linear systems  cannot display the  range of behaviours that are typical of complex systems. Consequently, when a complex system is converted into a linear one by simplifying the mathematical model, much of the interesting behaviour of the system is lost.

Linearity in organisational theories

It turns out that many organizational theories are based on assumptions of equilibrium (i.e. that organisations are stable) and linearity (i.e. that the socio-economic forces on the organisation are small) . Much like the case of physical systems, such models will predict only small changes about the stable state – i.e. that “business as usual” will continue indefinitely. In a paper published in 1988, Andrew Abbott coined the term General Linear Reality (GLR) to describe this view of reality. GLR is based on the following assumptions:

1. The world consists of unchanging entities which have variable attributes (eg: a fixed organisation with a varying number of employees)
2. Small changes to attributes can have only small effects, and effects are manifested as changes to existing attributes.
3. A given attribute can have only one causal effect – i.e. a single cause has a single effect.
4. The sequence of events has no effect on the outcome.
5. Entities and attributes are independent of each other (i.e. no correlation)

The connection between GLR and linearity in physics is quite evident in these assumptions.

The world isn’t linear

But reality isn’t linear – it is very non-linear as many managers learn the hard way. The problem is that the tools they are taught in management schools do not equip them to deal with situations that have changing entities due to feedback effects and  disproportionately large effects from small causes (to mention just a couple of common non-linear effects).

Nevertheless, management research is catching up with reality. For example, in a paper entitled Organizing Far From Equilibriium: Nonlinear changes in organizational fields,  Allan Meyer, Vibha Gaba and Kenneth Collwell highlight limitations of the GLR paradigm. The paper describes three research projects that were aimed at studying how large organisations adapt to change.  Typically when researchers plan such studies, they tacitly make GLR  assumptions regarding cause-effect, independence etc. In the words of Meyer, Gaba and Collwell:

In accord with the canons of general linear reality, as graduate students each of us learned to partition the research process into sequential stages: conceptualizing, designing, observing, analyzing, and reporting. During the conceptual and design stages, researchers are enjoined to make choices that will remain in effect throughout the inquiry. They are directed, for instance, to identify theoretical models, select units and levels of analysis, specify dependent and independent variables, choose sampling frames, and so forth. During the subsequent stages of observation, analysis, and reporting, these parameters are immutable. To change them on the fly could contaminate data or be interpreted as scientific fraud. Stigma attached to “post hoc theorizing,” “data mining” and “dust-bowl empiricism” are handed down from one generation of GLR researchers to the next.

Whilst the studies were in progress, however, each of the organisations that they were studying underwent large, unanticipated changes: in one case employees went on mass strike; in another, the government changed regulations regarding competition; and in the third boom-bust cycles caused massive changes in the business environment. The important point is that these changes invalidated  GLR assumptions completely.  When such “game-changing” forces are in play, it is all but impossible to define a sensible equilibrium state to which organisations can adapt.

In the last two decades, there is a growing body of research which shows that organizations are complex systems that display emergent behaviour.  Mainstream management practice is yet to catch up with these new developments, but the signs are good: in the last few years there have been articles dealing with some of these issues in management journals which often grace the bookshelves of CEOs and senior executives.

To conclude

Mainstream management principles are based on a linear view of reality, a view that is inspired by scientific management and 19^th century physics.  In reality, however, organisations evolve in ways that are substantially different from those implied by simplistic cause-effect relationships embodied in linear models.  The sciences have moved on, recognizing that most real-world phenomena are nonlinear, but much of organisational research and management practice remains mired in a linear world.  In view of this it isn’t surprising that many management “best” practices taught in business schools don’t work in the real world.

Related posts:

Models and messes – from best practices to appropriate practices

Cause and effect in management

On the origin of power laws in organizational phenomena