Eight to Late

Sensemaking and Analytics for Organizations

All models are wrong, some models are harmful

Introduction

One of the ways in which we attempt to understand and explain natural and social phenomena is by building models of them.  A model is a representation of a real-world phenomenon, and since the real world is messy, models are generally based on a number of simplifying assumptions. It is worth noting that models may be mathematical but they do not have to be –  I present examples of both types of models in this article.

In this post I make two points:

1. That all models are incomplete and are therefore wrong.
2. That certain models  are not only wrong, but  can have harmful consequences if used thoughtlessly. In particular I will  discuss a model of human behaviour that is widely taught and used in management practice, much to the detriment of organisations.

Before going any further I should clarify  that I don’t “prove” that all models are wrong; that is likely an impossible task. Instead, I use an example to illustrate some general features of models which strongly suggest that no model can possibly account for all aspects of a phenomenon. Following that I discuss how models of human behaviour must be used with caution because they can have harmful consequences.

All models are wrong

Since models are based on simplifying assumptions, they can at best be only incomplete representations of reality.  It seems reasonable to expect  that all models will breakdown at some point because they are not reality. In this section, I illustrate this looking at a  real-world example  drawn from the king of natural sciences, physics.

Theoretical physicists build mathematical models that describe natural phenomena. Sir Isaac Newton was a theoretical physicist par excellence.  Among other things, he  hypothesized that the force that keeps the earth in orbit around the sun is the same as the one that keeps our feet firmly planted on the ground.  Based on observational inferences made by Johannes Kepler, Newton  also figured out that the force is inversely proportional to the square of the distance between them.  That is: if the distance between two bodies is doubled, the gravitational force between them decreases four-fold.   For those who are  interested, there is a nice explanation of Newton’s law of gravitation here.

Newton’s law tells us the precise nature of the force of attraction between two bodies.  It is universal in that it applies to all things that have a mass, regardless of the specific material they are made of. It’s utility is well established: among other things, it enables astronomers and engineers to predict the  trajectories of planets, satellites and spacecraft to extraordinary accuracy; on the flip side it also enables war mongers to compute the trajectories of missiles.  Newton’s law of gravitation has been tested innumerable times since it was proposed in the late 1700s, and it has passed with flying colours every time.

Yet, strictly speaking, it is wrong.

To understand why, we need to understand what it means to explain something. I’ll discuss this somewhat philosophical issue by sticking with gravity. Newton’s law enables us to predict the effects of gravity, but it does not tell us what gravity actually is. Yes, it’s a force, but what exactly is this force? How does it manifest itself? What is it that passes between two bodies to make them “aware” of each other’s existence?

Newton is silent on all these questions.

An explanation had to wait for a century and a half. In 1914 Einstein proposed that every body that has mass creates a distortion  of space (actually space and time) around it. He formalised this idea in his General Theory of Relativity which tells us that gravity is a consequence of the curvature of space-time.

This is difficult to visualise, so perhaps an analogy will help. Think of space-time as a flat rubber sheet. A marble on the sheet causes a depression (or curvature) in the vicinity of the marble. Another marble close enough would sense the curvature and would tend to roll towards the original marble.  To an observer who wasn’t aware of the curvature (imagine the rubber sheet to be invisible) the marbles would appear to be attracted to each other. Yet at a deeper level, the attraction is simply a consequence of geometry. In this sense then, Einstein’s theory “explains” gravity at a more fundamental level than Newton’s law does.

Now, one of the predictions of Einstein’s theory is that the force of gravitation is ever so slightly different from that predicted by Newton’s law.  This difference is so small that it is unnoticeable in the case of spacecraft or even planets, but it does make a difference in the case of dense, massive bodies such as black holes. Many experiments have confirmed that Einstein’s theory is more accurate than Newton’s.

So Newton was wrong.

However, the story doesn’t end there because  Einstein was wrong too.  It turns out, that Einstein’s theory of gravitation is not consistent with Quantum Mechanics, the theory that describes the microworld of atoms and elementary particles.  One of the open problems in theoretical physics is the development of a quantum theory of gravity. To be honest, I don’t know much at all about quantum gravity, so if you want to know more about this other  holy grail of physics,  I’ll refer you to Lee Smolin’s excellent book, Three Roads to Quantum Gravity.

Anyway, the point I wish to make is not that these luminaries were wrong but that the limitations of their models were in a sense inevitable. Why? Well, because our knowledge of the real world is never complete, it is forever work in progress. We build models based on what we know at a given time, which in turn is based on our current state of knowledge and the empirical data that supports it. The world, however, is much more complex than our limited powers of reasoning  and observation , even if these are enhanced by instruments. Consequently any models that we construct are necessarily incomplete – and therefore, wrong.

Some models are harmful

The foregoing brings me to the second point of this post.

There’s nothing wrong in being wrong, of course; especially if our understanding of the world is enhanced in the process.  I would be quite happy to leave it there if that was all there was to it. The problem is that there is something more insidious and dangerous: some models are not only wrong, they are positively harmful.

And no, I’m not referring to nuclear weapons; nuclear fission by itself is neither benign nor dangerous, it is what we do with it that makes it so. I’m referring to something far more commonplace, a model that underpins much of modern day management:  it is the notion that  humans are largely rational beings who make decisions based solely on their  narrow self-interest.  According to this view of humans as economic beings,  we are driven by material gain to the exclusion of  all other considerations. This is a narrow, one-dimensional view of humans  but is one that is legitimised by mainstream economics  and has been adopted enthusiastically  by many management schools and their alumni.

Among other things, those who subscribe to  this model believe that:

1. Employees are inherently untrustworthy because they will act in their own personal interests, with no consideration of the greater good. Consequently their performance needs to be carefully “incentivised” and monitored.
2. Management’s goals should be to maximise profits. Consequently they should be “incentivised”  by bonuses that are linked solely to profit earned.

These are harmful because

1. Treating employees like potential shirkers who need to “motivated”  by a carrot and stick policy will only demotivate them.
2. Linking senior management bonuses to financial performance alone encourages managers to follow strategies that boost short term profits regardless of the long term consequences.

The fact of the matter is that humans are not atoms or planets; they can (and will) change their behaviour depending on how they are treated.

To sum up

All models are wrong, but  some models – especially those relating to human behaviour – are harmful. The danger of taking models of human behaviour literally is that they tend become self fulfilling prophecies. As Eliyahu Goldratt once famously said, “Tell me how you measure me and I’ll tell you how I’ll behave.”  Measure managers by the profits they generate and they’ll  work to maximise  profits to the detriment of longer-term sustainability, treat employees  like soulless economic beings and they’ll end up behaving like the self-serving souls the organisation deserves.

Written by K

December 2, 2012 at 5:56 pm

17 Responses

1. The difference between the physics models and the human interaction models are several:
(1) There is no underlying interaction model of the people – no Lagrangian so to speak
(2) There is no way to test the model with an experiment that can be repeated under controlled conditions
(3) There is no way to have the model verified by an unbiased party – blind review
(4) Human models in the end have no control over the externalities of the model or the experience

Models of physical systems from physics to chemistry to probabilistic model of project behaviour have some ability to control these externalities. Human models do not.

Lumping these under the same title puts undo burden on both sides to the determent of the physical models, which are very useful in their narrow domain of probabilistic forecasting

Like

Glen Alleman

December 2, 2012 at 11:17 pm

• Hi Glen,

First, I agree with you – there are several differences between physical and human-interaction models. However, I think the differences are not the issue here.

1. Sure,not all models can be “mathematised”. Nevertheless, there are successful models that are not mathematical – a good example from the biological domain is the theory of evolution. In the social domain too, there are models such as the interactional model of communication and the model of humans as boundedly rational beings, that are useful within their domain of applicability. Physical models too have restricted domains of applicability, so the latter is a limitation of both physico-mathematical and social models.

2. True, one cannnot test human-interaction models (except under rather restrictive conditions). However, it is certainly possible to apply them in real-life situations to see if they work. In the case of the model I describe in the post, the result has been a spectacular fail.

3. Economic models are subject to peer review and are tested too. Unfortunately, all too often they must be tested in the real world, with sometimes disastrous consequences.

4. Externalities will always be present in real world phenomena. In physics we can create artificial environments in which the externalities can be controlled. Nevertheless, in the wild they do exist (which is why experimentalists spend so much effort in eliminating them in the lab). I do agree that externalities are considerably harder to eliminate in the case of social models. However, the difference between the two cases is one of degree rather than principle.

So, yes, there are significant differences between mathematical and non-mathematical model. Nevertheless, there are commonalities that are perhaps more important – the primary one being that they are representations that are intended to improve our understanding of the phenomena under study.

The problem in the social sciences is that models are often reflexive, i.e. that is there is a bidirectional relationship between cause and effect. In particular, humans can change their behaviour in response to being treated in a certain way. One of the consequences of this is that models can become self fulfilling prophecies. And it is this that that makes the model of homo economicus harmful. In short: there is an ethical dimension to models in social sciences that simply does not exist in physics.

I hope this clarifies my position somewhat.

Thanks again for your thoughtful comment – much appreciated!

Regards,

Kailash.

Like

K

December 3, 2012 at 8:28 am

• Thanks for the quick response. I guess what I’m trying to say is attempts to model human behaviour, like modeling economic behaviour are fraught with difficulties and usually don’t produce results that allow the decision makers to make informed decision in the way mathematical model do, even probabilistic model found in cost, schedule and technical performance models

Like

Glen Alleman

December 3, 2012 at 8:37 am

• Hi Glen,

…and there I agree entirely. Economists and management gurus who have popularised these models (and management schools that propagate them) have a lot to answer for.

Regards,

Kailash.

Like

K

December 3, 2012 at 5:23 pm

2. Good points by Glen. In this context, you might be interested in this article, as it touches upon a general theory of modelling and the requirements for the for the successful application of different modelling methods for different modelling tasks.

“Outline for a Morphology of Modelling Methods” (2012)

http://www.amg.swemorph.com/pdf/amg-1-1-2012.pdf

As Glen points out, there is no adequate (causal/predictive) modelling method for “people systems”, inter alia because of the nature of cognitive self-reference.

Regards,

/Tom Ritchey
SweMorph

Like

Tom Ritchey (@swemorph)

December 3, 2012 at 8:00 am

• Hi Tom,

Thanks for your comment and the link to your paper.What an impressive piece of work it is! I will have to read it more carefully when I get a chance, but have seen some great ideas already. I particularly liked your section on synthesis/analysis and the modelling of self-referential (or reflexive) systems. The latter section should be required reading (and understanding!) for those who would build models of human behaviour.

Regards,

Kailash.

Like

K

December 3, 2012 at 5:24 pm

3. Tom,
Thanks for the good paper. While would argue the modeling conversation is too analytical, I work in a domain that lives on models – US DoD program performance management. We’re working on a set of “essential views” using past performance – cost, schedule, technical, risk – that are used to forecast the future performance of those same measurements, with a defined confidence level.
So both the principles and the practices of modeling stochastic networks of work activities that produce measurable outcomes is critical to the success of large complex programs.

Like

Glen Alleman

December 3, 2012 at 8:59 am

4. Of course this all traces back to the work of statistician George P Box (http://en.wikipedia.org/wiki/George_E._P._Box) who is credited with saying “essentially, all models are wrong, but some are useful”. He was referring to errors of fidelity which can only be statistically known.

Other uses of the word ‘model’, as in business model or evolution model or communication model, are speaking metaphorically, raising an image of a physical/math model to explain structure and behavior that have the trappings of repeatability, albeit with generous disorder (entropy)

John Goodpasture

Like

John Goodpasture

December 3, 2012 at 9:52 pm

• Hi John,

Thanks for your comment. I would agree that those who are trained in maths and physics may interpret term model in the sense you mention. However, I reckon others – for example, architects and psychologists – would likely see things differently.

Relevant here, is the etymology of the term, which states that it arose from the notion of “likeness to scale,” suggesting it is more about similarity or even behaviour rather than mathematical representation.

Regards,

Kailash.

Like

K

December 4, 2012 at 5:38 pm

• Kailash: I agree with your point about the similarity definition, but of course that begs the question: similar to what? That’s where things go a bit awry, and bring us back to your original theme of harmful models. I said models without an objective standard (physical or math) are essentially metaphors, ‘representing’ a vision/image/object that is ‘similar to’ or representative of what the metaphor is trying to describe, as in: business model. That then introduces ‘representative bias’ into the frame (See Kahneman and Tversky) and thus may lead to harm. The circle closes.

John

Like

John C Goodpasture

December 4, 2012 at 9:29 pm

• Hi John,

Thanks for your response – this is a good discussion.

Actually, metaphors play a huge role in the creation of models in the physical science – Rutherford used a metaphor of a solar system when coming up with his model of the atom and, before him, Thomson likened an atom to a plum pudding. Examples of metaphorical thinking in the natural sciences abound. Like all metaphors they have their limitations, but I think it is important to recognise the role that metaphor plays in the so-called hard sciences. Much of the process of discovery in science involves metaphorical thinking.

Anyway, all that apart, I think the model of homo economicus, is actually quite objective – even by the standards you define – because it describes humans as maximisers of economic utility. Representing this in mathematical terms is then a matter of finding the appropriate utility function.

Regards,

Kailash.

Like

K

December 4, 2012 at 10:08 pm

• Finding a utility function is no small matter, especially if a similar event to model is not evident. Usually, you have to accumulate a lot of observations/interviews and then synthesize a hypothesis (function), then to be tested or validated by an actual event/circumstance. This brings Thomas Bayes into the frame.

John.

Like

John C Goodpasture

December 5, 2012 at 4:52 am

• Hi John,

Sure, but that makes it akin to the scientific/statistical models you mentioned earlier – involving a sequence of observation, synthesis and validation.

Referring to my earlier comment – there are two points I wanted to make::

1. The use of metaphor is not restricted to non-mathematical models – even scientific models have metaphorical elements.

2. The model of homo economicus can be cast in mathematical terms given a concrete context. It may be difficult to do so, but it can be done in principle. In fact, as you say, Bayes may come into the picture – which kind of brings in statistical issues of the the kind that George Box was talking about.

Thanks so much for taking the time to read and respond.

Regards,

Kailash.

Like

K

December 5, 2012 at 5:23 am

5. Reblogged this on Projects – gathering pace.

Like

Martin Price

December 6, 2012 at 7:12 am

6. K,
Just wondering what the motivation is for modeling the human side of things. It seems such a mess, fraught with conflicting paradigms??

Like

galleman

December 6, 2012 at 7:49 am

• Hi Glen,

That’s a good question. I reckon the motivation for modelling the human side of things is twofold: first, the payoff would be huge if one could actually pull it off; second, it is an inherently interesting problem!

Before going any further, it is perhaps worth making a point that I should have made in the post: not even the most analytically oriented economist or sociologist would claim that it is possible to develop an overarching theory of human behaviour. What most of them try to do is to develop middle range theories – theories that start with empirical phenomena and then attempt to abstract general characteristics that can be verified by data. This works well in some situations – this book by Duncan Watts has some nice examples

The problem, as I have mentioned in the post, is that some of the theories that have resulted from this approach (financial derivatives, for example) have actually had disastrous consequences, primarily because they do not account for bidirectional cause-effect relationships that are inevitable when dealing with humans. That, in my opinion, is the nub of the problem.

So where does that leave us?

I think it is more useful to start from human motivations and intentions- specifically, to understand the things that influence and drive them. In short, one has to start with the human mind. This is more the domain of psychology and even philosophy rather than sociology or economics. There is some good (non-mathematical) modelling work in this area.Two examples that come to mind are Watzlawick’s interactional theory of communication and Winograd’s work on computers and cognition. At a personal level, these works (and others akin to them) have inspired many of the pieces on this blog as well as the stuff I have out in print.

Regards,

Kailash.

Like

K

December 6, 2012 at 9:03 pm

7. Fascinating discussion gentlemen

It is that we continue to explore and in so doing add to our knowledge and understanding.

Throughout this process we establish fixed points in our understanding and construct a conceptual spring board from which to propel our ideas and concepts to the next level of research until such time as we have verified our theory is correct or been challenged in our thinking.

Martin is doing just that by questioning the very core of our understanding of these principles and the possibility they are harmful. What is the new model we should be working to if all models are wrong. It leaves us with a dilemma. How much error can we accept before we are forced to rethink our model.

One to go away and think about

Like

Nigel Wright

December 15, 2012 at 1:37 am