What is Emergent Design?

Last week I had the opportunity to talk to a data science team about the problems associated with building a modern data capability in a corporate environment. The organisation’s Head of Data was particularly keen to hear about how the Emergent Design approach proposed in my recent book might help with some of the challenges they are facing. To help me frame my talk, he sent me a bunch of questions, which included the following two:

1. Can you walk us through the basic steps of emergent design in data science?
2. How does emergent design work with other data science methodologies, such as CRISP-DM or Agile?

On reading these questions, I realized I had a challenge on my hands: Emergent Design is about cultivating a mindset or disposition with which to approach problematic situations, rather than implementing a canned strategic framework or design methodology. Comparing it to other methodologies would be a category error – like comparing pizza to philosophy. (Incidentally, I think this is the same error that some (many?) Agile practitioners make when they mistake the rituals of Agile for the mindset required to do it right…but that’s a story for another time.)

The thing is this: the initial task of strategy or design work is about a) understanding what ought to be done, considering the context of the organisation and b) constructing an appropriate approach to doing it.  In other words, it is about framing the (strategic or any other) problem and then figuring out what to do about it, all the while keeping in mind the specific context of the organization. Emergent Design is a principled approach to doing this.

–x–

So, what is Emergent Design?

A good place to start is with the following passage from the PhD thesis of David Cavallo, the man who coined the term and developed the ideas behind it:

“The central thrust of this thesis is the presentation of a new strategy for educational intervention. The approach I describe here resembles that of architecture, not only in the diversity of the sources of knowledge it uses but in another aspect as well – the practice of letting the design emerge from an interaction with the client. The outcome is determined by the interplay between understanding the goals of the client; the expertise, experience, and aesthetics of the architect; and the environmental and situational constraints of the design space. Unlike architecture where the outcome is complete with the artifact, the design of [such initiatives] interventions is strengthened when it is applied iteratively. The basis for action and outcome is through the construction of understanding by the participants. I call this process Emergent Design.”

Applied to the context of building a data capability, Emergent Design involves:

1. Having conversations across the organization to understand the kinds of problems people are grappling with. The problems may or may not involve data. The ones that do not involve data give you valuable information about the things people worry about.  In general the conversations will cover a lot of territory and that is OK – the aim is to get a general feel for the issues that matter.
2. Following the above, framing a handful – two or three – concrete problems that you can solve using skills and resources available on hand. These proof-of-concept projects will help you gain allies and supporters for your broader strategic efforts.

While doing the above, you will notice that people across the organization have wildly different perspectives on what needs to be done and how. Moreover, they will have varying perspectives on what technologies should be used. As a technology strategist, your key challenge is around how to reconcile and synthesise these varied viewpoints. This is what makes the problem of strategy a wicked problem. For more on the wicked elements of building data capabilities, check out the first chapter of my book which is available for free here.

Put simply then, Emergent Design is about:

1. Determining a direction rather than constructing a roadmap
2. Letting the immediate next steps be determined by what adds the greatest value (for your stakeholders)

With that said for the “what,” I will now say a few words about the “how.”

–x–

In the book we set out eight guidelines or principles for practicing Emergent Design. I describe them briefly below:

Be a midwife rather than an expert: In practical terms, this involves having conversations with key people in business units across the organisation, with the aim of understanding their pressing problems and how data might help in solving them (we elaborate on this at length in Chapter 4 of the book). The objective at this early stage is to find out what kind of data science function your organisation needs. Rather than deep expertise in data science, this requires an ability to listen to experts in other fields, and translate what they say into meaningful problems that can potentially be solved by data science. In other words, this requires the strategist to be a midwife rather than an expert.

Use conversations to gain commitment: In their ground- breaking book on computers and cognition, Winograd and Flores observed that “organisations are networks of commitments.” between people who comprise the organisation. It is through conversations that commitments between different groups of stakeholders are established and subsequently acted on. In Chapter 3 of the book, we offer some tips on how to have such conversations. The basic idea in the above is to encourage people to say what they really think, rather than what they think you want them to say. It is crucial to keep in mind that people may be unwilling to engage with you because they do not understand the implications of the proposed changes and are fearful of what it might mean for them.

Understand and address concerns of stakeholders who are wary of the proposed change: In our book, The Heretic’s Guide to Management, Paul Culmsee and I offer advice on how to do this in specific contexts using what we call “management teddy bears”. These involve offering reassurance, advice, or opportunities that reduce anxiety, very much akin to how one might calm anxious children by offering them teddy bears or security blankets. Here are a few examples of such teddy bears:

• A common fear that people have is that the new capability might reduce the importance of their current roles. A good way to handle this is to offer these people a clear and workable path to be a part of the change. For example, one could demonstrate how the new capability (a) enriches their current role or (b) offers opportunities to learn new skills or (c) enhances their effectiveness. We could call this the “co- opt teddy bear”. In Chapter 7 of our data science strategy book, we offer concrete ways to involve the business in data science projects in ways that makes the projects theirs.
• It may also happen that some stakeholder groups are opposed to the change for political reasons. In this case, one can buy time by playing down the significance of the new capability. For example, one could frame the initiative as a “pilot” project run by the current data and reporting function. We could call this the “pilot teddy bear.” See the case study towards the end of Chapter 3 of the data science strategy book for an example of a situation in which I used this teddy bear.

Frame the current situation as an enabling constraint: In strategy development, it is usual to think of the current situation in negative terms, a situation that is undesirable and one that must be changed as soon as practicable. However, one can flip this around and look at the situation from the perspective of finding specific things that you can change with minimal political or financial cost. In other words, you reframe the current situation as an enabling constraint (see this paper by Kauffman and Garre for more on this). The current situation is well defined, but there are an infinite number of possible next steps. Although the actual next step cannot be predicted, one can make a good enough next step by thinking about the current situation creatively in order to explore what Kauffman calls the adjacent possible – the possible future states that are within reach, given the current state of the organisation (see this paper by Kauffman). You may have to test a few of the adjacent possible states before you figure out which one is the best. This is best done via small, safe- to- fail proof of con­cept projects (discussed at length in Chapter 4 of our data science strategy book).

Consider long- term and hidden consequences: It is a fact of life that when choosing between different approaches, people will tend to focus on short- term gains rather than long- term consequences. Indeed, one does not have to look far to see examples that have global implications (e.g., the financial crisis of 2008 and climate change). Valuing long- term results is difficult because the distant future is less salient than the present or the immediate future. A good way to look beyond immediate concerns (such as cost) is to use the “solution after next principle” proposed by Gerald Nadler and Shozo Hibino in their book entitled Breakthrough Thinking. The basic idea behind the principle is to get people to focus on the goals that lie beyond the immediate goal. The process of thinking about and articulating longer term goals can often provide insights into potential problems with the current goals and/ or how they are being achieved. We discuss this principle and another approach to surfacing hidden issues in Chapters 3 and 4 of the data science strategy book.

Create an environment that encourages learning: Emergent Design is a process of experimentation and learning. However, all learning other than that of the most trivial kind involves the possibility of error. So, for it to work, one needs to create an environment of psychological safety – i.e., an environment in which employees feel safe to take risks by trialling new ideas and processes, with the possibility of failure. A key feature of learning organisations is that when things go wrong, the focus is not on fixing blame but on fixing the underlying issue and, more importantly, learning from it so that one reduces the chances of recurrence. It is interesting to note that this focus on the system rather than the individual is also a feature of high reliability organisations such as emergency response agencies.

Beware of platitudinous goals: Strategies are often littered with buzzwords and platitudes – cliched phrases that sound impressive but are devoid of meaning. For example, two in- vogue platitudes at the time our book was being written are “digital transformation” and “artificial intelligence.” They are platitudes because they tell you little about what exactly they mean in the specific context of the organisation.

The best way to deconstruct a platitude is via an oblique approach that is best illustrated through an example. Say someone tells you that they want to implement “artificial intelligence” (or achieve a “digital transformation” or any other platitude!) in their organisation. How would you go about finding out what exactly they want? Asking them what they mean by “artificial intelligence” is not likely to be helpful because the answer you will get is likely to be couched in generalities such as data- driven decision making or automation, phrases that are world- class platitudes in their own right! Instead, it is better to ask them how artificial intelligence would make a difference to the organisation. This can help you steer the discussion towards a concrete business problem, thereby bringing the conversation down from platitude- land to concrete, measurable outcomes.

Act so as to increase your choices: This is perhaps the most important point in this list because it encapsulates all the other points. We have adapted it from Heinz von Foerster’s ethical imperative which states that one should always act so as to increase the number of choices in the future (see this lecture by von Foerster). Keeping this in mind as you design your data science strategy will help you avoid technology or intellectual lock in. As an example of the former, when you choose a product from a particular vendor, they will want you to use their offerings for the other components of your data stack. Designing each layer of the stack in a way that can work with other technologies ensures interoperability, an important feature of a robust data technology stack (discussed in detail in Chapter 6 of the strategy book). As an example of the latter, when hiring data scientists, hire not just for what they know now but also for evidence of their curiosity and proclivity to learn new things – a point we elaborate on in Chapter 5 of the data strategy book.

You might be wondering why von Foerster called this the “ethical imperative.”  An important aspect of this principle is that your actions should not constrain the choices of others. Since the predictions of your analytical models could well affect the choice of others (e.g. whether or not they are approved for a loan or screened out for a job), you should also cast an ethical eye on your work. We discuss ethics and privacy at length in Chapter 8 of the data strategy book.

–x–

These principles do not tell you what to do in specific situations. Rather, they are about cultivating a disposition that looks at technology through the mul­tiple, and often conflicting, perspectives of those whose work is affected by it. A technical capability is never purely technical; within the context of an  organisation it becomes a sociotechnical capability.

In closing: Emergent Design encourages practitioners to recognise that building and embedding a data capability (or any other technical capability such as program evaluation) is a process of organisational change that is best achieved in an evolutionary manner. This entails starting from where people are – in terms of capability, culture, technology, and governance – and enabling them to undertake a collective journey in an agreed direction.

…and, yes, all the while keeping in mind that it is the journey that matters, not the destination.