Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘Emergent Design’ Category

To think, to be, to act

leave a comment »

It would have been sometime in late 2013. I was in the midst of exploring the possibility of setting up an analytics development centre for a large, somewhat conservative organization. The location of the centre had yet to be determined, but it was clear it would be a continent and a world away from headquarters.

A senior IT executive from headquarters was visiting our subsidiary. I knew him quite well and we had a good working relationship. He frowned as he caught sight of me across our big open plan area and gestured that he wanted to talk.

Uh oh.

I nodded and walked over to a vacant meeting room on my side.  He followed shortly and closed the door behind him.

Brief pleasantries done, he got to the point. “What’s this I hear about a development centre? What the hell are you up to?”


Despite out best-laid plans, the lives of our projects and the projects of our lives tend to hinge on minor events that we have little control over. Robert Chia stresses this point in his book Strategy without Design:

“Ambitious strategic plans, the ‘big picture’ approach that seeks a lasting solution or competitive advantage through large-scale transformations, often end up undermining their own potential effectiveness because they overlook the fine details of everyday happenings at ‘ground zero’ level.

At one level we know this, yet we act out a large part of our personal and work lives as though this were not so.


In business (and life!) we are exhorted to think before doing. My boss tells me I need to think about my team’s workplan for next year; my wife tells me I need to think about the future. Thinking is at the center of our strategies, blueprints, plans – the things that supposedly propel our lives into an imagined future.  

In brief, we are exhorted to make detailed plans of what we are going to do; we are encouraged not to act without thinking.

As Descartes famously wrote, cogito ergo sum, our thinking establishes our being.

But is that really so?


Gregory Bateson noted the following in his book, Angels Fear:

There is a discrepancy of logical type between “think” and “be”. Descartes is trying to jump from the frying pan of thought, ideas, images, opinions, arguments etc., into the fire of existence and action. But that jump itself is unmapped. Between two such contrasting universes there can be no “ergo” – no totally self-evident link. There is no looking before leaping from “cogito” to “sum”.

The gap between our plans and reality is analogous to the gap between thought and action. There is ample advice on how to think but very little on how to act in difficult situations.

As Bateson wrote elsewhere:

What is lacking is a theory of action within large complex systems, where the active agent is himself a part and a product of the system.

He then goes on to say that Kant’s categorical imperative – “act so to treat humanity, whether in your own person or in another, always as an end and never as only a means – might provide a starting point for such a theory.”

So far, so unsurprising.

But in the very next line, Bateson says something truly intriguing:

It seems also that great teachers and therapists avoid all direct attempts to influence the action of others and, instead, try to provide the settings or contexts in which some (usually imperfectly specified) change may occur.

This line resonated deeply when I read it first because it spelt out something that I had learnt through experience but had not found the words to articulate.


In contentious discussions, it is difficult to change minds using facts and figures alone. Indeed, the current reluctance to be vaccinated against Covid is a case in point (see this article, for example).What one needs in such situations is to reframe the terms of the discussion. In the Covid case that might be to focus on relative risks in terms that people can understand rather than absolute numbers of people who have suffered serious side-effects of the vaccine.

In general, reframing is about changing the way in which people perceive the problematic issue.  The best way to describe how it works is via an example. Here’s one from Paul Watzlawick’s classic book on change

A police officer with a special ability for resolving sticky situations in unusual ways, often involving a disarming use of humour, was in the process of issuing a citation for a minor traffic violation when a hostile crowd began to gather around him. By the time he had given the offender his ticket, the mood of the crowd was ugly and the sergeant was not certain that he would be able to get back to the relative safety of his patrol car. It then occurred to him to announce in a loud voice: “You have just witnessed the issuance of a traffic ticket by a member of your Oakland Police Department” And while the bystanders were busy trying to fathom the deeper meaning of this all too obvious communique, he got into his cruiser and drove off.

The specifics of what one might do depends on the situation, but the general idea is to appreciate the situation from the viewpoint of the other party and act in a way that helps shift that perspective in an indirect or oblique manner.

This is one of the key principles of emergent design – and more about that in a forthcoming piece.


Back to the story I started with:

I realized instinctively that much hinged on what I said and – more importantly – how I said. My interlocutor was clearly upset, and I had to ensure that my words did not infuriate him further. He had the power to stop my fledgling project in its tracks with a word or two in the right ears.

“There is no plan to set up a development centre,” I said, looking him in the eye. “All we have done is hire a couple of people here to help with the workload at headquarters.”

“Who has requested help?”

I told him who. He knew that person well and thought highly of him.

“Where do you plan to go from here?” he demanded.

“Like I said, there is no plan. This is just a pilot to see if we can help improve productivity. The idea is to free people in headquarters so that they can focus on the strategic stuff.”

“Just make sure it doesn’t turn into something bigger.”

“Absolutely,” I responded, mustering what I hoped was a reassuring smile.

“OK,” he nodded and walked out. 

I breathed easier; he seemed to be OK with it for now. But even if not, the conversation was still open. More importantly, I had bought myself some time to pay greater attention to the politics of the project over the coming weeks.


It was only in retrospect that I realized that the interaction described here was pivotal to the success of the project. How so is a story to be told later. For now, the point I wish to make is that the projects of our lives can be planned down to great detail, but their outcomes are often determined by the unplanned micro-actions we take while doing them.


(no identity – courtesy HaPe Gera https://www.flickr.com/photos/hape_gera/2929195528)

Written by K

June 1, 2021 at 6:51 am

Boundaries and horizons

with 2 comments

James Carse once said, “It is the freedom we all know we have that terrifies us.” 

So deep is this terror that we do not want to acknowledge our freedom.  As a result, we play within boundaries defined by fear.


Some time ago, I bumped into a student who had taken a couple of classes that I taught some years ago. Over a coffee, we got talking about his workplace, a large somewhat bureaucratic organisation.

At one point he asked, “We need to change the way we think about and work with data, but I’m not a manager and have no authority to do what needs to be done.”

“Why don’t you demonstrate what you are capable of without waiting for permission?” I replied. “Since you are familiar with your data, it should be easy enough to frame and solve a small problem that makes a difference.”

“My manager will not like that,” he said.

“It is easier to beg forgiveness than seek permission,” I countered.

“He might feel threatened and make life difficult for me.”

“On the other hand, he might appreciate your efforts.”

“You don’t know him,” he replied.

“If you’re not appreciated, you are always free to leave.  Moreover, the skills you have learnt in the last two years should give you confidence to exercise that freedom.”

“I’m comfortable where I am,” he said sheepishly, “with my mortgage and all this uncertainty in the economy, I can ill-afford any risk.”

I didn’t say so at the time, but thought it unfortunate that he had set boundaries for himself.


Boundaries are characteristic of what Carse calls finite games: games that are played with the purpose of winning. These are the games of convention, those that we are familiar with. He contrasts these with infinite games: those whose purpose is the continuation of play.

As a corollary, an infinite game has no winner (or loser) because the game never ends.

Finite games are bounded, both temporally (they last a finite time) and spatially (they are played within a bounded region). As Carse notes in his book, “finite players play within boundaries; infinite players play with boundaries.”

And then a bit later, he tells us how to play with boundaries. “What will undo any boundary is the awareness that it is our vision, and not what we are viewing, that is limited.”


“I’m resigning,” he said, before launching into an explanation. As he talked, the thing that came to mind was the contrast between his attitude and the student’s. 

His explanation was completely unnecessary. I understood.

There comes a tide in the affairs of humans etc…and often that tide is evident only to those who are able and willing to look up and see the possibilities on the distant horizon.


In contrast to boundaries, horizons are not fixed. As you move towards a horizon it moves away from you. As Carse tells us:

One never reaches a horizon. It is not a line; it has no place; it encloses no field; its location is always relative to the view. To move toward a horizon is simply to have a new horizon.

Much of the talk about lifelong learning (which now has its own Wikipedia entry!) is really about taking a horizonal view of life. It has less to do with “staying current” or “learning employable skills” than with gaining new perspectives.

But that does not mean one has to take in the entire vista in one glance. Some new things…no, most new things, take time.


“I can’t handle failure,” she said. “I’ve always been at the top of my class.”

She was being unduly hard on herself. With little programming experience or background in math, machine learning was always going to be hard going.  “Put that aside for now,” I replied. “Just focus on understanding and working your way through it, one step at a time. In four weeks, you’ll see the difference.”

“OK,” she said, “I’ll try.”

She did not sound convinced but to her credit, that’s exactly what she did. Two months later she completed the course with a distinction.

“You did it!” I said when I met her a few weeks after the grades were announced.

“I did,” she grinned. “Do you want to know what the made the difference?”

Yes, I nodded.

“I stopped treating it like a game I had to win,” she said, “and that took the pressure right off.  I then started to enjoy learning.”


For many, success in work…or even in life… is largely a matter of appearances: if one’s career is not marked by a series of increasingly impressive titles then one is likely to be labelled an also-ran, if not an outright failure.

But what is a title? Carse tells us the following:

What one wins in a finite game is a title. A title is the acknowledgment of others that one has been the winner of a particular game. Titles are public. They are for others to notice. I expect others to address me according to my titles, but I do not address myself with them – unless, of course, I address myself as another. The effectiveness of a title depends on its visibility, its noticeability, to others.”

In these lines, Carse makes some important points. Firstly, a title has to be given to us by others.  Secondly, the effectiveness of a title depends on others paying attention to it.  That is, its significance lies in the significance that others give it. This is the reason why titles matter to those who compete for them.


A couple of weeks ago, I invited an ex-student to give a talk to my machine learning class. As I had expected, he did a brilliant job, introducing the class to some tools that they are likely to find useful the future. But the gold lay in something he said in the Q and A session that followed.

“How do you stay up to date in this field?” a student asked.

“Yes, this is a question I struggled with when I started out, ” said Jose. “Data science is a rapidly expanding field and it is impossible to keep pace with it…but let me show you something.” He navigated to his LinkedIn profile and started scrolling through his list of certifications.

It was a long list.

“When I started out,” he continued, “I constantly felt this fear that I was missing out. So, what did I do? I tried to learn everything I could, collecting a bunch of certifications that I kept adding to my profile. One day, I woke up feeling burnt out and asked myself why I was doing this. The only honest answer was because others seemed to think it necessary and even important. That shook me. I started thinking deeply about what I thought was important for myself, what my purpose is. I realised I did not have one; I was running like crazy down a path set by others, not my own.  That realisation changed everything for me.”


Life’s too short to play games and chase titles that others deem important. We are free to play our own game and keep playing it as long as we wish to.

Yes, that can be terrifying.

It can also be liberating.


Written by K

May 3, 2021 at 7:58 pm

The elusive notion of context

with 13 comments

It would have been my second or third day in this country. There was a lot going in my head at the time: the worry of a small bank balance, the tension of finding a job, the assorted insecurities of finding one’s way in a strange land. I was walking back to my motel in the evening, lost in thought  when, quite unexpectedly, I caught the unmistakable aroma of eucalyptus and wattle, a scent I had grown up with in my childhood years in the Nilgiri Hills.

A view of the Niligiri Hills (Wikimedia)

It was strangely comforting. A place that felt unfamiliar moments earlier seemed much less so, a small step towards a sense of belonging…


A few years later, I was working on a research problem relating to a phenomenon that is easily demonstrated in a kitchen sink. If you are interested, make your way to the nearest sink and do the following:

Turn the tap on slowly until water starts to flow out as a cylindrical jet. You will notice that the jet breaks up into near spherical droplets a short distance from the mouth of the tap.

This phenomenon is called jet breakup. Instead of describing it further, I will follow the advice that a picture is worth several words.

Jet breakup


The problem of jet breakup was first studied by Lord Rayleigh in 1878 but had enjoyed a renaissance in the 1970s and 80s due to applications such as inkjet printing. My project was to apply the principle to the production of shot from jets of molten metal. I was not too enthused by the problem as it was, at best, a minor variation of a theme that had already been done to death. However, I was being paid to do research so I worked on it as diligently as I could, making desultory progress.

One October evening, about two years into my project, I was washing up after dinner when I noticed a curious wave-like structure on the thin jet that emerged from the kitchen sink tap and fell onto a plate an inch or two below the tap (the dishes had piled up that day). The wave pattern was stationary and rather striking.

Stationary waves on a water jet

The phenomenon is one that countless folks have seen. Indeed, I had noticed it before but never paid it much attention until that October evening when I saw the phenomenon with new eyes.  Being familiar with the work of Rayleigh and others, I realised, at once, that the pattern had the same underlying cause as jet breakup. Wondering if anyone had published papers on it, I dashed off to the library to do a literature search (Google Scholar and decent search engines were still a few years away). Within a few hours I realised I’d stumbled on a new context that would change the direction of my research.


Years and a couple of career changes later, I was working in a large multinational. Through sheer luck of being at the right place at the right time, I got to do some well-regarded work in business intelligence and analytics.  One of the things I realized during these projects is that the technical aspects of the work are the easiest. The hardest part is figuring out what to do…and this is hard because of people, not technology.

It was my first encounter with problem wickedness, the notion that problems involving diverse groups of individuals are socially complex. I started reading and thinking about practical ways to deal with social complexity in organizations, an effort that led to my collaborative work with Paul Culmsee. The key point, elaborated at length in our work, is that progress on tackling such problems depends critically on building a common context, one that all stakeholder groups can relate to.


There is a lot of published nonsense about building productive teams, much of it circulating on the internet. I will refrain from singling out specific articles because there are so many, but you can find them for yourself using search phrases such as “building teams”.   The fact of the matter is that there are no special levers one can pull to make good teams. As JR Hackman, a well-known researcher on team behaviour stated in a review article entitled, From causes to conditions in group research:

Influences on group behavior and performance do not come in separate, distinguishable packages. They come, instead, in complex tangles of redundant features and forces. To try to partial out and assess the causal effects of each component can be an exercise in frustration because each ingredient of what may be a spicy stew loses its zest when studied separately from the others.

Hackman urges researchers (and by implication, practitioners) to forget about causes and focus on creating the right conditions for a team to do good work. The real work of a leader is to set the right context and get out of the way. As David Snowden tells us in this talk, “Leadership is a property of the way you interact with people, not a property of status or training…[it] is about creating the right context for good things to happen.”

 If you are interested in finding out more about building context, check out David Snowden’s writings on the Cognitive Edge blog or the books I have linked to above. For a much longer and very enlightening series of discourses on the importance of context, I highly recommend Gregory Bateson’s magnum opus, Steps to an Ecology of Mind. The work is a collection of writings that deal with things far removed from management and organisations, but you will slowly start to see its relevance as you digest his many-splendoured essays on anthropology, evolution, animal behaviour, art, psychology, information, communication and more.


David Snowden’s Cynefin Framework offers a useful way to think about organizational problems. As an example, I have found the complicated / complex distinction useful in triaging problems frequently encountered in organizational settings.  In brief, complex problems tend to have a much higher degree of social complexity compared to complicated problems. They are, in other words, wicked problems. Most organizational problems that matter are complex, not complicated. 

Quite apart from the utility of the framework, I find the origin of the name Cynefin enlightening. The term is commonly (and poorly) translated as habitat. A better translation is the phrase, a place of belonging.  This resonates with the following passage excerpted from a paper Snowden wrote about the origins of the term:

For Christmas 1998 my parents followed a long tradition of giving me a book related to Welsh culture. That year it was The Land and the Sea, a collection of the works of Kyffin Williams, published to celebrate his eightieth year. I have admired his work from when I first saw a portrait painted by him, of a much-loved headmaster a quarter of a century earlier. He is an artist who has a profound understanding of place, both in his landscape work and his formal and informal portraits of the people of his native land, the Brythonic Kingdom of Gwynedd which dominated what is now Wales from the fifth to the thirteenth centuries. The Romans knew it as Venedotia, or the land of Venus, and I had the privilege of growing up there, walking its hills and sailing its coast. To sit on the summit of Tryfan as the sun descends over the Glyder ridge and Y Garn, casting shadows on to the precipitous slopes of Pen yr Ole Wen is a profound experience, not just of the aesthetic beauty of the landscape, but of one’s place, one’s identity, one’s place of belonging. There is a welsh word, Cynefin, which means all of that and more and has no equivalent in the English language, where it is crudely translated as habitat or place. In his preface to Kyffin Williams’book, Nicholas Sinclair connects the word Cynefin to the interaction between human beings and their environment that is the essence of the author’s work. I took that word as the name for this model created to understand the different types of system within which we operate: ordered, complex and chaotic.

Williams, Kyffin; Deserted Farm, Llanrhuddlad; Southend Museums Service; http://www.artuk.org/artworks/deserted-farm-llanrhuddlad-2750

Snowden’s words resonate with my professional experiences in a wide variety of organisations as well as my personal experiences of dealing with the unfamiliar.  Until I feel I belong and can relate to others in that environment, I cannot make any meaningful progress.  This is a prerequisite to initiating and achieving successful change of any kind, be it an organizational restructure, a large-scale IT project or even settling into a new country.

The sense of belonging is particularly important for work that involves different groups of stakeholders, particularly those that have conflicting interests and values. In such cases it is critical to achieve a shared understanding of the problem one is trying to solve before attempting to solve it. Such an understanding is often assumed at the start but is then found to be illusory when the “solution” is implemented which, of course, is much too late. The work of building a shared understanding is essentially an act of collaborative problem formulation or sensemaking. It is, in other words, the creation of a common context.


Years later still, I was asked to move to a regional role in the same multinational. My role there was to look after IT development activities in the region. Since much of this was driven from the corporate head office, much of my work was about resolving the gaps between what various offices scattered across the region needed and what the corporate-mandated solution offered. My experiences in sensemaking helped, but the official aspects of the role were – not to put too fine a point on it – excruciatingly boring. 

There was, however, a saving grace. When I was hired, my boss told me that he wanted me to explore the possibility of setting up a regional development centre for analytics that could serve the entire organization. There was a clear cost argument in favour of such centre. However, given the sharply divided opinions around offshoring, the public airing of such a proposal would cause all kinds of reactions, many of which would be negative. The first problem was to address those upfront.

Oh, and I forgot to mention, my boss told me I had zero budget to do this.

I talked to the usual suspects, a few big outsourcers and consultancies, but soon realized that their aims were not congruent with mine. Everything they told me pointed to high costs and potential conflicts down the line, not the least being vendor lock in and expensive contract variations. As I have pointed out elsewhere, the hidden costs of outsourcing are much too high.

Around that time, I came across the notion of emergent design. The essential idea is to start from where people are and take small steps, each of which lead to demonstrable improvement. This generally requires some trial and error, but since the investment at each step is small and the benefit demonstrable, it is not hard to convince the folks who sign cheques. Moreover, this enables one to continually adjust one’s approach based on feedback from the previous step, much like nature does in the process of evolution.  Actions are based on a given context, but the context itself changes because of the action and thus necessitates recalibrating subsequent actions.

To begin with, I needed to develop a small use case that would clearly demonstrate the benefit of hiring developers in my region to do work for people located in the head office, but in a way that alleviated concerns about job losses. As it happened, I had a colleague in head office who led a database team that was struggling to keep up with their workload. They simply did not have enough people and lacked the budget to hire contractors locally. I had a phone conversation with my colleague and suggested hiring someone who would sit in my office and work for him remotely. He was enthusiastic once he heard the costs (much lower than local consultants!) and committed funding for year.

The fledgling centre thus made its first hire.  A few months forward, my colleague was happy with the result. Consequently, he did a lot of evangelizing on my behalf about how this arrangement did not take away jobs, but augmented capacity.

Requests started trickling in; a few short months later they were pouring in. Three years later, there were over fifty developers in the centre. It had become a thing.


In recent years, advances in computing have made it possible to process vast volumes of data.  It is now possible for data scientists to apply machine learning techniques to large datasets containing detailed information about individuals.  There seems to be a widespread belief that, given the right kind of data, it is possible to abdicate decision-making on social matters to machine learning models. Of course, there are the warnings of data ethicists and others about the dark side of data science, but these tend to be swamped by the voices of data optimists. Data is so revered that large datasets tend to take on a life of their own, stripped of the context in which they were originally gathered.  There are enough examples that illustrate why this is problematic. If one could save not just a dataset but also details on how it was collected and why, the limitations of the data would be more apparent thus making it is less likely that it would be used in inappropriate ways.  The proposal on datasheets for datasets by Timnit Gebru and collaborators is a good start towards addressing the issue of context in data science.

One must keep in mind that cause-effect relationships are not obvious in large social systems. There are myriad plausible mechanisms of change and, more often than not, several of them will interact and operate in tandem. Given this, any decision based on data alone is likely to be misleading. Instead, success is more likely to come from an emergent approach based on developing multiple perspectives on the data – i.e. by building context around it.


Building context in social systems is hard because cause-effect relationships are hard to disambiguate and pin down. Consequently, it is impossible to know, with certainty, what is relevant and what is not. Indeed, if it were possible at all, one would include all details, leaving nothing out. The model of the world would then be the world itself; the map would be the entire territory.  But as Borges so eloquently noted:

… In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast map was Useless…

Building context is a never-ending quest to understand the territory.


…and so I have been here for many years now. The smell of eucalyptus and wattle still evokes memories of home, but I do not know which one. What I do know now, but did not know then, is that being at home in a place or a set of ideas is the work of a lifetime.

Written by K

March 2, 2021 at 9:57 pm

%d bloggers like this: