Archive for June 2011
An enterprise IT initiative generally affects a range of stakeholders groups, each with their own take on why the project is being undertaken and what the result should look like. This diversity of views is no surprise: an organisation-wide effort affects many divisions and departments, so there are bound to be differing – even conflicting – views regarding the initiative and its expected outcome.
The existence of many irreconcilable viewpoints is one of the main symptoms of a wicked problem – a problem that is hard to define, let alone solve. Paul Culmsee has written about the inherent wickedness of projects that involve collaborative platforms such as SharePoint. In this post I discuss how another class of enterprise scale initiatives – efforts to consolidate and harmonise organizational data for analytical and reporting purposes (so-called data warehouse projects) – display characteristics of wickedness. I also briefly discuss a couple of approaches that can be used to manage this issue.
As some of my readers may not be familiar with the terms data warehouse or wicked problem, I’ll start with a short introduction to the two terms in order to set the stage for the main topic.
A data warehouse is a repository of data that a business deems important for reporting and analysis. Ideally, a data warehouse integrates data from multiple sources – for example, CRM and financial systems – thereby serving as an authoritative source for management reports (often referred to as a “single point of truth”). There are at least a couple of different design philosophies for data warehouses, but I won’t go into these as they are not relevant to the discussion. What’s interesting is that most of the literature on data warehousing deals with its technical aspects – things such as data modelling and extract-transform-load processes – yet, as anyone who has been involved in an enterprise-scale data warehousing effort will tell you, the biggest challenges are political, not technical. To be fair, this was recognized a while ago – Marc Demarest wrote an article on the politics of data warehousing in 1997. However, it is worth revisiting this issue because there are techniques to handle it that weren’t widely known at the time Demarest wrote his article. I discuss these briefly later, but first let’s look at what wickedness means and its relevance to data warehouse projects.
The term wicked problem was coined by Horst Rittel and Melvin Webber in a now-classic paper entitled Dilemmas in a General Theory of Planning. The paper is essentially a critique of the traditional approach to social planning, wherein decisions are made by experts who, by virtue of their specialist knowledge and training, are assumed to know best. Such an approach often doesn’t work because it ends up alienating stakeholders who are adversely affected by the “solution.” This is a symptom of social complexity – messiness and conflict arising from diverse opinions as to what the problem is and how it should be solved. Those involved in enterprise-scale IT initiatives – whether as users, managers or technical specialists – would have had first-hand experience of this social complexity.
How do we know that a problem is socially complex (or wicked)? That’s easy: In the paper, Rittel and Webber describe ten criteria for wickedness – so a problem is wicked if it satisfies some or all of the Rittel-Webber criteria. We’ll take a look at the criteria and their relevance to data warehousing next.
The wickedness of data warehouse initiatives
To support my claim about the wickedness of data warehousing initiatives, I’ll simply list the ten Rittel-Webber criteria (in their original form) along with a brief commentary on how they can crop up in data warehouse projects. Here we go:
- There is no definitive formulation of a wicked problem: Those who have worked on organisation-wide efforts at integrating data will know that the first problem is to decide “what’s in and what’s out” – that is, what data sources are considered in scope for integration. The problem arises because different business stakeholders have different views on what is important. For example, data that is critical to HR may not be a priority for the marketing function.
- Wicked problems have no stopping rule: Data warehouse initiatives are never definitively completed: there are always new data sources that need to be integrated; old ones to be turned off; business rules to be changed and so on. Any stopping rule that one might define will need to be revised as new business requirements come up and new data sources are revealed.
- Solutions to wicked problems are not true or false, but better or worse: This is simply an expression of the truism that there is no right or wrong way to build a data warehouse. There are a range of different architectures and approaches that can be chosen, each with their pros and cons (see this paper for a comparison of the two most popular approaches). The problem is that one often cannot tell beforehand which approach is going to be best for a particular situation.
- There is no immediate or ultimate test of a solution to a wicked problem: This is a statement of the fact that one cannot tell whether or not a particular implementation can completely solve the problem of data integration. As Rittel and Webber put it, “…any solution, after being implemented, will generate waves of consequences over an extended – virtually unbounded – period of time. Moreover, the next day’s consequences of the solution may yield utterly undesirable consequences…” Although these words are somewhat over-the-top, the message isn’t: for example, I have seen situations where programming errors that have remained undetected for years (yes, years) have lead to incorrect data being used in reports.
- Every solution to a wicked problem is a “one-shot” operation; because there is no opportunity to learn by trial and error, every attempt counts significantly: Because of the high costs of implementation, enterprise-scale IT initiatives tend to be one-shot affairs. Another limiting factor is that there is usually a very short window of time in which the project must be completed – as the cliché goes, “users need these reports yesterday.” Among other things, this precludes the option of learning by trial and error.
- Wicked problems do not have an enumerable (or exhaustively describable) set of potential solutions, nor is there a set of well-describable options that may be incorporated into the plan: This point may seem like it doesn’t apply to data warehousing initiatives – all data warehousing projects have a plan, right? Nevertheless, those who have worked on such projects will attest to the fact that the plan – such as it is – needs frequent revision because of surprises that crop up along the way. Iterative/incremental development approaches can address these issues to some extent, but cannot eliminate them completely. Because of time constraints, it is inevitable that solutions to unexpected roadblocks occur through improvisation rather than planning.
- Every wicked problem is essentially unique: This one is easy to see: every organisation is unique, and so are its data integration requirements. Methodologists and consultants may try to convince you otherwise, and tempt you into following generic approaches – but don’t be fooled, generic approaches will come unstuck. Your data is unique, treat it with the respect and seriousness it deserves.
- Every wicked problem can be considered to be a symptom of another problem: One of the key drivers of data warehouse projects is that organizations tend to have the same (or similar) data residing in multiple databases. As a consequence there are several different “sources of truth” for reports. These different sources of truth arise because systems used in different departments may have different definitions of the same business entity. For example, a customer might be defined in one way within the financial system but in another way in a CRM system. Seen in this light, the problem of multiple sources of truth is actually a symptom of lack of communication between different departments, what is sometimes called silo mentality.
- The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem’s resolution: As discussed in the previous point, the discrepancy in the case of a data integration problem is the lack of congruency between different data sources. There can be a range of explanations for the discrepancy. For example, one explanation may be that the data is actually different – a customer in the CRM system is not the same as the customer in the finance system; another explanation may be that the two entities are the same but their definitions differ because the systems were developed independently of each other. The data integration solution in the two cases will differ –in other words, the solution to the problem depends on which explanation is seen as the correct one.
- The planner has no right to be wrong: The data warehouse designer is in a difficult position: he or she may have to reconcile contradictory requirements. Following from the example of the previous point, whatever design decisions the designer makes regarding the definition of a customer, there will be some parties that will not be happy: if she goes with the finance definition, sales will be ticked off; if she chooses the sales definition, finance will not be happy; if she chooses to define a single common entity, neither will be pleased. Yet, her mandate is to satisfy all business requirements. This criterion is essentially an expression of the political aspect of data warehouse projects.
I find it quite amazing that criteria that were framed in the context of social planning problems can apply word-for-word to data consolidation initiatives.
Managing wickedness in data warehousing
As should be evident from the above, wicked problems can’t be solved in the usual sense of the word, but they can be managed. Although there are many techniques to manage wickedness, they all focus on the same end: to help all stakeholder groups reach a shared understanding of the problem and make a shared commitment to action. Such a shared understanding is absolutely critical because business and IT folks often have differing views on what a data warehouse ought to be.
One approach that I have used to help stakeholders get to a shared understanding in data warehouse projects is dialogue mapping, a facilitation technique that maps out the conversation between stakeholders as it occurs. Dialogue mapping uses the Issue-Based Information System (IBIS) notation which was invented by Rittel as a means to document the different facets of a wicked problem. See this post for a data warehouse related example of dialogue mapping and this one for more on the IBIS notation.
Shared understanding and commitment to action is well and good, but in the end success is measured by deliverables: the data warehouse and accompanying reports must be built. One of the challenges with a data warehouse initiative is that customers have to wait a long (very long!) time before they see any tangible benefits. Agile approaches to data warehousing offer a way to address this issue. For those interested in the nuts and bolts of agile data warehousing, I recommend Ralph Hughes’ book, which discusses how Scrum can be adapted for data warehousing projects.
Although the juxtaposition of the terms “agile” and “data warehouse” may sound oxymoronic to some, there is evidence that it works (see this case study, for example). Of course, no approach is a silver bullet; those who want to read about potential problems may want to look at this thesis for a research-based view of the pros and cons of an agile approach to data warehousing.
In the end, though, one has to keep in mind that no development technique – agile or otherwise – will succeed unless all stakeholders have a shared understanding of what the data warehouse is intended to achieve. The biggest issues are organisational rather than technical.
As we have seen, corporate data integration problems satisfy many – if not all – of the criteria for wickedness. The main implication of this is that data consolidation at an enterprise level is not just a difficult technical problem it is also a socially complex one. Although tackling this requires skills and techniques that are outside of the standard repertoire of technical staff and managers, these skills can be learnt. What’s more, they are critical for success: those who undertake data warehouse projects without an understanding of the conflicting agendas of stakeholder groups may fail for reasons that have nothing to do with technology.
The other day a friend and I were talking about a failed process improvement initiative in his organisation. The project had blown its budget and exceeded the allocated time by over 50%, which in itself was a problem. However, what I found more interesting was that the failure was in a sense planned – that is, given the way the initiative was structured, failure was almost inevitable. Consider the following:
- Process owners had little or no input into the project plan. The “plan” was created by management and handed down to those at the coalface of processes. This made sense from management’s point of view – they had an audit deadline to meet. However, it alienated those involved from the outset.
- The focus was on time, cost and scope; history was ignored. Legacy matters – as Paul Culmsee mentions in a recent post, “To me, considering time, cost and scope without legacy is delusional and plain dumb. Legacy informs time, cost and scope and challenges us to look beyond the visible symptoms of what we perceive as the problem to what’s really going on.” This is an insightful observation – indeed, ignoring legacy is guaranteed to cause problems down the line.
The conversation with my friend got me thinking about planned failures in general. A feature that is common to many failed projects is that planning decisions are based on dubious assumptions. Consider, for example, the following (rather common) assumptions made in project work:
- Stakeholders have a shared understanding of project goals. That is, all those who matter are on the same page regarding the expected outcome of the project.
- Key personnel will be available when needed and – more importantly – will be able to dedicate 100% of their committed time to the project.
The first assumption may be moot because stakeholders view a project in terms of their priorities and these may not coincide with those of other stakeholder groups. Hence the mismatch of expectations between, say, development and marketing groups in product development companies. The second assumption is problematic because key project personnel are often assigned more work than they can actually do. Interestingly, this happens because of flawed organisational procedures rather than poor project planning or scheduling – see my post on the resource allocation syndrome for a detailed discussion of this issue.
Another factor that contributes to failure is that these and other such assumptions often come in to play during the early stages of a project. Decisions that are based on these assumptions thus affect all subsequent stages of the project. To make matters worse, their effects can be amplified as the project progresses. I have discussed these and other problems in my post on front-end decision making in projects.
What is relevant from the point of view of failure is that assumptions such as the ones above are rarely queried, which begs the question as to why they remain unchallenged. There are many reasons for this, some of the more common ones are:
- Groupthink: This is the tendency of members of a group to think alike because of peer pressure and insulation from external opinions. Project groups are prone to falling into this trap, particularly when they are under pressure. See this post for more on groupthink in project environments and ways to address it.
- Cognitive bias: This term refers to a wide variety of errors in perception or judgement that humans often make (see this Wikipedia article for a comprehensive list of cognitive biases). In contrast to groupthink, cognitive bias operates at the level of an individual. A common example of cognitive bias at work in projects is when people underestimate the effort involved in a project task through a combination of anchoring and/or over-optimism (see this post for a detailed discussion of these biases at work in a project situation). Further examples can be found in in my post on the role of cognitive biases in project failure, which discusses how many high profile project failures can be attributed to systematic errors in perception and judgement.
- Fear of challenging authority: Those who manage and work on projects are often reluctant to challenge assumptions made by those in positions of authority. As a result, they play along until the inevitable train wreck occurs.
So there is no paradox: planned failures occur for reasons that we know and understand. However, knowledge is one thing, acting on it quite another. The paradox will live on because in real life it is not so easy to bell the cat.
Much of the knowledge that exists in organisations remains unarticulated, in the heads of those who work at the coalface of business activities. Knowledge management professionals know this well, and use the terms explicit and tacit knowledge to distinguish between knowledge that can and can’t be communicated via language. Incidentally, the term tacit knowledge was coined by Michael Polanyi – and it is important to note that he used it in a sense that is very different from what it has come to mean in knowledge management. However, that’s a topic for another post. In the present post I look at a related issue that is common in organisations: the fact that much of what people know can be made explicit, but isn’t. Since the discipline of knowledge management is in dire need of more jargon, I call this inexplicit knowledge. To borrow a phrase from Polanyi, inexplicit knowledge is what people know, but won’t tell. Below, I discuss reasons why potentially explicit knowledge remains inexplicit and what can be done about it.
Why inexplicit knowledge is common
Most people would have encountered work situations in which they chose “not to tell” – remaining silent instead of sharing knowledge that would have been helpful. Common reasons for such behaviour include:
- Fear of loss of ownership of the idea: People are attached to their ideas. One reason for not volunteering their ideas is the worry that someone else in the organisation (a peer or manager) might “steal” the idea. Sometimes such behaviour is institutionalised in the form of an “innovation committee” that solicits ideas, offering monetary incentives for those that are deemed the best (more on incentives below). Like most committee-based solutions, this one is a dud. A better option may be to put in place mechanisms to ensure that those who conceive and volunteer ideas are encouraged to see them through to fruition.
- Fear of loss of face and/or fear of reprisals: In organisational cultures that are competitive, people may fear that their ideas will be ridiculed or put down by others. Closely related to this is the fear of reprisals from management. This happens often enough, particularly when the idea challenges the status quo or those in positions of authority. One of the key responsibilities of management is to foster an environment in which people feel psychologically safe to volunteer ideas, however controversial or threatening the ideas may be.
- Lack of incentives: Some people may be willing to part with their ideas, but only at a price. To address this, organisations may offer extrinsic rewards (i.e. material items such as money, gift vouchers etc) for worthwhile ideas. Interestingly, research has shown that non-monetary extrinsic rewards (meals, gifts etc.) are more effective than monetary ones. This makes sense – financial rewards are more easily forgotten; people are more likely to remember a meal at a top-flight restaurant than a 500$ cheque. That said, it is important to note that extrinsic rewards can also lead to unintended side effects. For example, financial incentives based on quantity of contributions might lead to a glut of low-quality contributions. See the next point for a discussion of another side effect of extrinsic rewards.
- Wrong incentives: As I have discussed at length in my post on motivation in knowledge management projects, people will contribute their hard earned knowledge only if they are truly engaged in their work. Such people are intrinsically motivated (i.e. internally motivated, independent of material rewards); their satisfaction comes from their work (yes, such people do exist!). Consequently they need little or no supervision. Intrinsic rewards are invariably non-material and they cannot be controlled by management. A surprising fact is that, intrinsically motivated people can actually be turned off – even offended – by material rewards.
Psychological safety and incentives are important factors, but there is an even more important issue: the relationships between people who make up the workgroup.
Knowledge sharing and the theory of cooperative action
The work of Elinor Ostrom on collective (or cooperative) action is relevant here because knowledge sharing is a form of cooperation. According to the theory of cooperative action, there are three core relationships that promote cooperation in groups: trust, reciprocity and reputation. Below I take a look at each of these in the context of knowledge sharing:
Trust: In the end, whether we choose to share what we know is largely a matter of trust: if we believe that others will respond positively – be it through acknowledgement or encouragement via tangible or intangible rewards – then the chances are that we will tell what we know. On the other hand, if the response is likely to be negative, we may prefer to remain silent.
Reciprocity: This refers to strategies that are based on treating people in the way we believe they would treat us. We are more likely to share what we know with others if we have reason to believe that they would be just as open with us.
Reputation: This refers to the views we have about the individuals we work with. Although such views may be developed by direct observation of peoples’ behaviours, they are also greatly influenced by opinions of others. The relevance of reputation is that we are more likely to be open with people who have a good reputation.
According to Ostrom, these core relationships can be enhanced by face-to-face communication and organisational rules/ norms that promote openness. See my post on Ostrom’s work and its relevance to project management for more on this.
One of the key challenges that organisations face is to get people working together in a cooperative manner. Among other things this includes getting people to share their knowledge; to “tell what they know.” Unfortunately, much of this potentially explicit knowledge remains inexplicit, locked away in peoples’ heads, because there is no incentive to share or, even worse, there are factors that actively discourage people from sharing what they know. These issues can be tackled by offering employees the right incentives and creating the right environment. As important as incentives are, the latter is the more important factor: the key to unlocking inexplicit knowledge lies in creating an environment of trust and openness.
The discussion of risk in presented in most textbooks and project management courses follows the well-trodden path of risk identification, analysis, response planning and monitoring (see the PMBOK guide, for example). All good stuff, no doubt. However, much of the guidance offered is at a very high level. Among other things, there is little practical advice on what not to do. In this post I address this issue by outlining some of the common pitfalls in project risk analysis.
1. Reliance on subjective judgement: People see things differently: one person’s risk may even be another person’s opportunity. For example, using a new technology in a project can be seen as a risk (when focusing on the increased chance of failure) or opportunity (when focusing on the opportunities afforded by being an early adopter). This is a somewhat extreme example, but the fact remains that individual perceptions influence the way risks are evaluated. Another problem with subjective judgement is that it is subject to cognitive biases – errors in perception. Many high profile project failures can be attributed to such biases: see my post on cognitive bias and project failure for more on this. Given these points, potential risks should be discussed from different perspectives with the aim of reaching a common understanding of what they are and how they should be dealt with.
2. Using inappropriate historical data: Purveyors of risk analysis tools and methodologies exhort project managers to determine probabilities using relevant historical data. The word relevant is important: it emphasises that the data used to calculate probabilities (or distributions) should be from situations that are similar to the one at hand. Consider, for example, the probability of a particular risk – say, that a particular developer will not be able to deliver a module by a specified date. One might have historical data for the developer, but the question remains as to which data points should be used. Clearly, only those data points that are from projects that are similar to the one at hand should be used. But how is similarity defined? Although this is not an easy question to answer, it is critical as far as the relevance of the estimate is concerned. See my post on the reference class problem for more on this point.
3. Focusing on numerical measures exclusively: There is a widespread perception that quantitative measures of risk are better than qualitative ones. However, even where reliable and relevant data is available, the measures still need to based on sound methodologies. Unfortunately, ad-hoc techniques abound in risk analysis: see my posts on Cox’s risk matrix theorem and limitations of risk scoring methods for more on these. Risk metrics based on such techniques can be misleading. As Glen Alleman points out in this comment, in many situations qualitative measures may be more appropriate and accurate than quantitative ones.
4. Ignoring known risks: It is surprising how often known risks are ignored. The reasons for this have to do with politics and mismanagement. I won’t dwell on this as I have dealt with it at length in an earlier post.
5. Overlooking the fact that risks are distributions, not point values: Risks are inherently uncertain, and any uncertain quantity is represented by a range of values, (each with an associated probability) rather than a single number (see this post for more on this point). Because of the scarcity or unreliability of historical data, distributions are often assumed a priori: that is, analysts will assume that the risk distribution has a particular form (say, normal or lognormal) and then evaluate distribution parameters using historical data. Further, analysts often choose simple distributions that that are easy to work with mathematically. These distributions often do not reflect reality. For example, they may be vulnerable to “black swan” occurences because they do not account for outliers.
6. Failing to update risks in real time: Risks are rarely static – they evolve in time, influenced by circumstances and events both in and outside the project. For example, the acquisition of a key vendor by a mega-corporation is likely to affect the delivery of that module you are waiting on –and quite likely in an adverse way. Such a change in risk is obvious; there may be many that aren’t. Consequently, project managers need to reevaluate and update risks periodically. To be fair, this is a point that most textbooks make – but it is advice that is not followed as often as it should be.
This brings me to the end of my (subjective) list of risk analysis pitfalls. Regular readers of this blog will have noticed that some of the points made in this post are similar to the ones I made in my post on estimation errors. This is no surprise: risk analysis and project estimation are activities that deal with an uncertain future, so it is to be expected that they have common problems and pitfalls. One could generalize this point: any activity that involves gazing into a murky crystal ball will be plagued by similar problems.