Eight to Late

Introduction

Over the last two decades or so, it has been recognized that creativity and innovation tend to thrive in organisations  where employees have a say in decisions that affect their work.  This has lead to the notion of a post-bureaucratic organisation –  an organisation in which decisions are made collectively through dialogue and consensus, and where  the hierarchy is flat.  Although a number of workgroups within large organizations function this way (and with some success too),  a post-bureaucratic organisation is generally seen as a utopian and academic ideal; one that is  unlikely to work in the real world. Those who manage  organizations, departments or workgroups  are generally uncomfortable with employees working autonomously,  even though this might lead to the generation of novel ideas and products.  This is understandable: it is ultimately the responsibility of managers to ensure that  organisational or departmental goals are achieved. How else to do this but through the time-tested command and control approach to management?

In response to the question posed above, project management is often touted as a means to manage creative and innovative efforts in organisations (see some of the articles in this issue of PM Network, for example).  The claim seems a reasonable one: project management (by definition) provides a means to manage  collective, goal oriented endeavours. Further, many projects – especially those involving  new product or software development – have a creative/innovative component. In practice, though, project management tends to be a bureaucratic affair;  involving plans that must be followed, schedules that must be adhered to and regular progress reports that must be made. Even so (or perhaps, because this is so) many organizations see project management as a means to manage all creative work in a post-bureaucratic setting.  Implicit in this view is the assumption that the implementation of project management processes will  enable managers to control and direct creative work without any adverse side-effects.  An article by Damian Hodgson entitled Project Work: The Legacy of Bureaucratic Control in the Post-Bureaucratic Organisation, explores the tensions and contradictions presented by this notion. Although the article was written a while ago (in 2003), I believe the ideas explored in it are ever more relevant today, particularly in view of the increasing projectisation of organisations and the work carried out within them.  Hence my motivation to  summarise and review  the paper.

Background – setting the stage

To be fair, many organizations recognise that a “light hand on the rudder” is needed in order to encourage creativity and innovation.  In these organisations, project management is often seen as a means to achieve this.  But  how well does it work in practice? Hodgson’s paper aims to provide some insight into this question via a case study of an organization in which a project-based form of management was implemented as a means to balance the  requirements of creativity and control. In his words:

In response to the challenges of the post-bureaucratic form, project management has  been put forward by many as a ‘tried-and-tested’ package of techniques able to cope with discontinuous work, expert labour and continuous and unpredictable change while delivering the levels of reliability and control of the traditional bureaucracy. In this article I explore some of the contradictions and tensions within a department where such a ‘hybrid’ mode of control is implemented, embodying both bureaucratic and post-bureaucratic logics. In particular, I focus upon the discursive tactics employed to sell ‘rebureaucratization’ as ‘debureaucratization’, and the complex employee responses to this initiative. I argue that the tensions evident here cast significant doubt on the feasibility of a seamless integration of bureaucracy and the post-bureaucratic [organization].

The “discursive tactics” that Hodgson mentions to are (seemingly reasonable and rational) arguments that an organization might use to “sell” the idea that the methods and approaches of project management are consistent with the ideals of a post-bureaucratic organization. An example of such an argument goes as follows: project management is routinely used to manage new product development projects, so it is eminently suited to managing creative work (Incidentally,  this isn’t quite right – see this paper review for more on why)

Post bureaucracy vs. bureaucracy

Before moving on, it is worth comparing the characteristics of bureaucratic and post-bureaucratic organizations. Hodgson provides the following comparison, drawn from Hekscher’s work:

Bureacracy	Post-bureaucracy
Consensus through Acquiescence to Authority	Consensus through Institutionalized Dialogue
Influence based on Formal Position	Influence through Persuasion/PersonalQualities
Internal Trust Immaterial	High Need for Internal Trust
Emphasis on Rules and Regulations	Emphasis on Organizational Mission
Information Monopolised at Top of Hierarchy	Strategic Information shared in Organization
Focus on Rules for Conduct	Focus on Principles Guiding Action
Fixed (and Clear) Decision Making Processes	Fluid/Flexible Decision Making Processes
Network of Specialized FunctionalRelationships	Communal Spirit/Friendship Groupings
Hierarchical Appraisal	Open and Visible Peer Review Processes
Definite and Impermeable Boundaries	Open and Permeable Boundaries
Objective Rules to ensure Equity of Treatment	Broad Public Standards of Performance
Expectation of Constancy	Expectation of Change

The aim of the case study is to highlight some of the inconsistencies and contradictions that result from applying a bureaucratic mechanism (project management) to manage the work of a group that was very good at doing creative work, but was used to a more free-wheeling, hands-off management style (i.e. a group which approximates the idealised post-bureaucratic environment described above).

Why project management?

Project management has its roots in classical management (ala Taylor and Fayol), so it is perhaps  surprising that it is considered as a means to manage work in a post-bureaucratic setting. In Hodgson’s words:

I would argue that what distinguishes project management as of particular relevance to 21stcentury organizations is its rediscovery of a very 19th-century preoccupation with comprehensive planning, linked to a belief in the necessity of tight managerial discipline.

Project management tools and techniques support managerial discipline by providing means to decompose the project into manageable bits – by using a WBS say – and then assigning these bits to individuals (or small teams). The work assigned can then be tracked and controlled tightly – which is a good thing.  If the matter rested there it wouldn’t be too bad, but very often management also prescibes how the work should be done. This level of control results in a loss of autonomy (and motivation)  for team members whose job it is to do the work. The loss of motivation can have negative effects, especially in projects with a large creative component.  To counter this criticism, in recent years project management has started to focus on the creative aspects of projects – what’s needed to motivate teams, how to foster creativity (in new product development work, say) etc. As Hodgson puts it,

The linking of project management and change management has increased project management’s influence across industries, such that now the largest professional organization in project management includes special interest groups in areas as diverse as healthcare, retail, media, marketing, and hospitality. As a consequence the last decade has been a time of particularly rapid expansion for project management, as issues of change, knowledge management, and constant innovation emerged as central themes in popular management discourse.

So, project management offers two (seemingly contradictory) benefits: the ability to maintain tight control of work and the ability to foster innovation and creativity:

…project management can be seen as an essentially bureaucratic system of control, based on the principles of visibility, predictability and accountability, and operationalized through the adherence to formalized procedure and constant written reporting mechanisms. At the same time, however, project management draws upon the rhetoric of empowerment, autonomy and self-reliance central… In principle, then, project management offers a system which attempts to integrate bureaucratic control and a form of responsible autonomy more in keeping with the interdisciplinary, knowledge-intensive nature of much project work in teams.

Seen in this light, it is perhaps not so surprising that project management is viewed as a means to manage creative work.

The case study

With the above background done, I now move on to a discussion of the case study.  In Hodgson’s words, the study:

…focused upon  a telephone bank in northern England which I have referred to under the pseudonym Buzzbank. In the late 1980s, Buzzbank had been set up by a major UK bank, which I will call TN Banking, and represented one of several success stories in the retail banking sector over this period. Through reduced overheads and the extensive use of new technology in the form of sophisticated marketing techniques and call-centre technology, Buzzbank had expanded rapidly in terms of market share and turnover, developing into a key component of TN Banking’s global operations. My interest in particular centred on Buzzbank senior management’s identification of project management as the prime ‘critical success factor’ for the organization; the development of project management expertise throughout the organization was seen as a key priority to maintain performance into the next decade. To an extent, the project teams researched could scarcely be more ‘cutting edge’, representing highly-trained ‘knowledge workers’ developing innovative high technology applications and solutions in a new sector of an enormously profitable industry

Hodgson conducted interviews and observed operations within the IT department of Buzzbank over a period of two years. During this period, the organization was implementing a “strategic plan” aimed at formalizing innovation and creative work using project management processes. The idea, in the words of a couple of senior IT managers was to “bring a level of discipline” and “bring an idea of professional structuring” to the work of the highly successful unit. The structuring and discipline was to be achieved by implementing project management processes.

The main rationale used to sell project management to the Buzzbank IT team is a familiar one: the need to ensure predictability and repeatability of work done whilst ensuring that innovation and creativity would not be impeded. Another justification offered by management was that the size of the organization (which had grown considerably in the years prior to the implementation of the strategic plan) meant that  the existing “ad-hoc” work culture would no longer be successful. That is,  the size of the organization necessitated a degree of formalization,  ergo bureaucratic procedures had to be put in place. This was rationalised  (by senior management) as a natural and inevitable consequence of growth:

…The organization was therefore portrayed by senior management in IT as approaching its ‘next stage of evolution’. The immediate benefit of such a metaphor for those members of senior management charged with rebureaucratizing the organization is that it carries a very strong sense of inevitability. As such, it casts opposition to such changes as irrational and futile, standing in the way of natural ‘evolution’.

Further, managers in the organization dubbed any employee resistance as “natural growing pains” – like those of an adolescent, say. Cast in this light, dissenting viewpoints were portrayed as natural and unavoidable – and possibly even necessary – but ultimately without any validity.

Another interesting aspect that Hodgson highlights is the way in which old practices (the successful but “bad” ones) were subsumed in the new (formal) framework. For example, in the old world, employees were given the freedom to experiment, and many considered this as a strength not a weakness. In the new world, however, such a practice was seen as a threat; it was considered more important to capture how to do things correctly so that things became repeatable (ala best practice) and experimentation would not be necessary. As one manager put it:

If we capture how we do things right, at least it makes things repeatable, and we can record the improvement required when things don’t go right, which doesn’t happen in a rapidly-expanding, gung-ho environment.

Hodgson notes that the terms rapidly-expanding and gung-ho, which are used in a negative sense, could just as well be cast in positive terms such as  flexible, proactive etc. The point being that management framed the existing situation in terms that made the implementation of the new procedures seem like a logical and reasonable next step. The processes were touted as a means to achieve change (i.e. be flexible), but in a controlled way. So, management went to great lengths to avoid use of terms that would be perceived as being negative – for example, the term “structure” was used instead of “bureaucracy” or “formalization.” In this way, management attempted to assimilate the existing values of Buzzbank into the strategic plan.

So, how well did it work? Here’s what Hodgson says about the end result:

The impression given [by senior management] was that of an organizational change which was inevitable, which gave rise to some understandable but irrational resistance, and which had now been effectively completed, for the good of the organization as a whole.

On the other hand, the impression Hodgson got from speaking to lower level employees was very different:

However, in the time spent by myself in the organization, the tone and target of much of the humour, as well as much stronger reactions, appeared to throw doubt on the extent to which this discourse had permeated among the general employees, particularly within the IT department. Humour was commonplace in the everyday banter both within teams and between teams in the IT division at Buzzbank, and the increasing levels of bureaucratization was the butt of most of the humour, particularly at the lower levels of the hierarchy. The main experience of project management as reported by many  Buzzbank employees was one of intensified bureaucratic surveillance…

A key example is the reaction of employees to managerial jargon that was used in company circulars and literature intended to promote the strategic plan.

Typically, comments were provoked by the circulation of literature on the strategic plan, and again, excerpts of the document were read out by members of staff, adding ironic comments to underline the gap between the document and their experience of life and work in the department.

Hodgson notes that employees often appeared to comply with the new regulations, but not in the way intended by management:

At other times, the employees appeared to comply with the formal requirements of the new system, in terms of filling in the necessary forms, reporting in at given times, completing the necessary work-logs and so on. Even here, despite the relative sophistication of senior management’s re-articulation of key discourses, compliance on the part of Buzzbank employees in many cases bore all the hallmarks of instrumental behaviour, accompanied by insubordinate statements and humour ranging from the cynical to the confrontational. At other times, assurances were given to senior management and immediately contravened, fictionalized accounts of project activities were submitted late, or else procedures were observed meticulously to the detriment of deadlines and other constraints. The emergent organizational order was a precarious negotiation between alienated compliance and an autonomous disregard for bureaucratic demands…

In short: there was a clear gap between the perceptions of management and employees as to the success of the newly implemented processes.

It is clear that Buzzbank managers saw  project management as  a means to control and direct creative / innovative work in a way that would have no negative effect on employee morale and motivation.   The challenge, of course, lay in achieving employee buy-in.  Management used many creative (!) tactics to “sell” project management to staff. These included:

1. References to a “natural process of evolution” and the consequent “growing pains” that the organization would experience. This made the pain of the change natural and inevitable, but necessary in the interest of future gain.
2. Manipulation of terminology to make the changes seem more palatable – e.g. using the word “structure” instead of “formalization” or “bureaucracy”.
3. Co-opting terminology of post-bureaucratic organizations into literature designed to promote the new structure. For example, claiming that project management processes would enable the organization to be even more responsive to change (i.e. flexible), through change management processes.

From employees’ perspective, such techniques were plainly seen for what they were: methods to “sell the unsellable”. The negative reactions of employees were manifested through sarcastic humour and (often minor) acts of insubordination. The case study thus highlights the difficulties in using project management as a means to control work in a post-bureaucratic work environment.

Wrapping-up: reflections and summary

Hodgson sees the case study as exemplifying the problem of control vs. autonomy in emerging post-bureaucratic organizations:  managers view project management as a means to address the risks inherent in post-bureaucratic work, whereas employees view it as a unnecessary and unjustified imposition. Management was looking for the “best of both worlds”, a hybrid model that incorporated the best elements of a post bureaucratic model and a traditional command and control approach. The case study casts doubt on whether such a hybrid is possible solely through the implementation standard project management techniques and processes. It does so by exposing some of the tensions and differences in perceptions that can occur when such a model is implemented.

So where does this leave managers? Is there a way to manage creative work without destroying employee morale and motivation?

Looking over the complaints of the Buzzbank employees, it is clear that most of the problems arose from the loss of autonomy that they had enjoyed prior to the implementation of the new processes. This being the case, any measure to increase autonomy should improve the situation. A couple of possibilities come to mind – both of which I have discussed in earlier pieces. These are:

1. Empower employees to make decisions that affect their work. This means allowing them the freedom to decide the best approach to solving problems (within limits specified by organisational and resource constraints).
2. In situations where (1) isn’t possible, one could use collaborative techniques such as dialogue mapping to achieve employee buy-in. Of course, management has to be prepared to engage in true dialogue, and be willing to act upon (reasonable) suggestions made by employees.

The key message is simple and obvious: the more of say employees have in making work-related decisions, the more engaged and motivated they’ll be. This is not just a warm and fuzzy notion, but one that is backed up by research on motivation (see this paper review, for example). Yes, this does mean letting go of the “reins of control” to an extent but it is clear, as highlighted by Hodgson’s work, that holding the reins tightly might cause more problems that it solves. What’s called for, above all, is a degree of flexibility:  use project management processes by all means, but be open to employee input as to what’s working well and what’s not.

To sum up:  Hodgson’s case study suggests that inflexible project management based approaches to managing creative work  may not work as well as advertised by purveyors of frameworks and methodologies. As an alternative,  it might be worth taking a step towards the utopian ideal of a post-bureaucratic organisation  by using techniques that encourage employee input in organisational decision making.

It would have been a couple of weeks after the kit tracking system was released that Therese called Mike to report the problem.

“How’re you going, Mike?” She asked, and without waiting to hear his reply, continued, “I’m at a site doing kit allocations and I can’t find the screen that will let me allocate sub-kits.”

“What’s a sub-kit?” Mike was flummoxed; it was the first time he’d heard the term. It hadn’t come up during any of the analysis sessions, tests, or any of the countless conversations he’d had with end-users during development.

“Well, we occasionally have to break open kits and allocate different parts of it to different sites,”  said Therese.  “When this happens, we need to keep track of which site has which part.”

“Sorry Therese, but this never came up during any of the requirements sessions, so there is no screen.”

“What do I do? I have to record this somehow.” She was upset, and understandably so.

“Look,” said Mike, “could you make a note of the sub-kit allocations on paper – or better yet, in Excel?

“Yeah, I could do that if I have to.”

“Great. Just be sure to record all the kit identifier and which part of the kit is allocated to which site.  We’ll have a chat about the sub-kit allocation process when you are back from your site visit. Once I understand the process, I should be able to have it programmed in a couple of days. When will you be back?”

“Tomorrow,” said Therese.

“OK, I’ll book something for tomorrow afternoon.”

The conversation concluded with the usual pleasantries.

After Mike hung up he wondered how they could have missed such an evidently important requirement. The application had been developed in close consultation with users.  The requirements sessions had  involved more than half the user community. How had they forgotten to mention such an important requirement and, more important, how had he and the other analyst not asked the question, “Are kits ever divided up between sites?”

Mike and Therese had their chat the next day. As it turned out, Mike’s off-the-cuff estimate was off by a long way. It took him over a week to add in the sub-kit functionality, and another day or so to import all the data that users had entered in Excel (and paper!) whilst the screens were being built.

The missing requirement turned out to be a pretty expensive omission.

—-

The story of Therese and Mike may ring true with those who are involved with software development. Gathering requirements is an error prone process: users forget to mention things, and analysts don’t always ask the right questions.  This is one reason why iterative development is superior to BDUF approaches: the former offers many more opportunities for interaction between users and analysts, and hence many more opportunities to catch those elusive requirements.

Yet, although Mike had used a joint development approach, with plenty of interaction between users and developers, this important requirement had been overlooked.

Further, as Mike’s experience corroborates, fixing issues associated with missing requirements  can be expensive.

Why is this so? To offer an answer, I can do no better than to quote from  Robert Glass’ book, Facts and Fallacies of Software Engineering.

Fact 25 in the book goes: Missing requirements are the hardest requirements errors to correct.

In his discussion of the above, Glass has this to say:

Why are missing requirements so devastating to problem solution? Because each requirement contributes to the level of difficulty of solving a problem, and the interaction among all those requirements quickly escalates the complexity of the problem’s solution. The omission of one requirement may balloon into failing to consider a whole host of problems in designing a solution.

Of course, by definition, missing requirements are hard to test for. Glass continues:

Why are missing requirements hard to detect and correct? Because the most basic portion of the error removal process in software is requirements-driven. We define test cases to verify that each requirement in the problem solution has been satisfied. If a requirement is not present, it will not appear in the specification and, therefore, will not be checked during any of the specification-driven reviews or inspections; further there will be no test cases built to verify its satisfaction. Thus the most basic error removal approaches will fail to detect its absence.

As a corollary to the above fact, Glass states that:

The most persistent software errors – those that escape the testing process and persist into the production version of the software – are errors of omitted logic. Missing requirements result in omitted logic.

In his research, Glass found that 30% of persistent errors were errors of omitted logic! It is pretty clear why these errors persist – because it is difficult to test for something that isn’t there. In the story above, the error would have remained undetected until someone needed to allocate sub-kits – something not done very often. This is probably why Therese and other users forgot to mention it. Why the analysts didn’t ask is another question: it is their job to ask questions that will catch such elusive requirements. And before Mike reads this and cries foul, I should admit that I was the other analyst on the project, and I have absolutely no defence to offer.

Introduction

A couple of months ago I wrote an article highlighting some of the pitfalls of using risk matrices. Risk matrices are an example of scoring methods , techniques which use ordinal scales to assess risks. In these methods,  risks are ranked by some predefined criteria such as impact or expected loss, and the ranking  is then used as the basis for  decisions on how the risks should be addressed. Scoring methods are popular because they are easy to use. However,  as Douglas Hubbard points out in his critique of current risk management practices, many commonly used scoring techniques are flawed. This post – based on Hubbard’s critique and research papers quoted therein -  is a brief look at some of the flaws of risk scoring techniques.

Commonly used risk scoring techniques and problems associated with them

Scoring techniques fall under two major categories:
 

1. Weighted scores: These use several ordered scales which are weighted according to perceived importance. For example: one might be asked to rate financial risk, technical risk and organisational risk on a scale of 1 to 5 for each, and then weight then by factors of 0.6, 0.3 and 0.1 respectively (possibly because the CFO – who happens to be the project sponsor – is more concerned about financial risk than any other risks ).
2. Risk matrices: These rank risks along two dimensions – probability and impact – and assign them a qualitative ranking of high, medium or low depending on where they fall.  Cox’s theorem shows such categorisations are internally inconsistent because the category boundaries are arbitrarily chosen.

 Hubbard makes the point that, although both the above methods are endorsed by many standards and methodologies (including those used in project management), they should be used with caution because they are flawed. To quote from his book:

Together these ordinal/scoring methods are the benchmark for the analysis of risks and/or decisions in at least some component of most large organizations. Thousands of people have been certified in methods based in part on computing risk scores like this. The major management consulting firms have influenced virtually all of these standards. Since what these standards all have in common is the used of various scoring schemes instead of actual quantitative risk analysis methods, I will call them collectively the “scoring methods.” And all of them, without exception, are borderline or worthless. In practices, they may make many decisions far worse than they would have been using merely unaided judgements.

 What is the basis for this claim? Hubbard points to the following:

1. Scoring methods do not make any allowance for flawed perceptions of analysts who assign scores – i.e. they do not consider the effect of cognitive bias. I won’t dwell on this as I have  previously written  about the effect of cognitive biases in project risk management -see this post and this one, for example.
2. Qualitative descriptions assigned to each score are understood differently by different people. Further, there is rarely any objective guidance as to how an analyst is to distinguish between a high or medium risk. Such advice may not even help: research by Budescu, Broomell and Po shows that there can be huge variances in understanding of qualitative descriptions, even when people are given specific guidelines what the descriptions or terms mean.
3. Scoring methods add their own errors.  Below are brief descriptions of some of these:
   1. In his paper on the risk matrix theorem, Cox mentions that “Typical risk matrices can correctly and unambiguously compare only a small fraction (e.g., less than 10%) of randomly selected pairs of hazards. They can assign identical ratings to quantitatively very different risks.” He calls this behaviour “range compression” – and it applies to any scoring technique that uses ranges.
   2. Assigned scores tend to cluster around the mid-low high range. Analysis by Hubbard shows that, on a 5 point scale, 75% of all responses are 3 or 4. This implies that changing a score from 3 to 4 or vice-versa can have a disproportionate effect on classification of risks.
   3. Scores implicitly assume that the magnitude of the quantity being assumed is directly proportional to the scale. For example, a score of 2 implies that the criterion being measured is twice as large as it would be for a score of 1. However, in reality, criteria are rarely linear as implied by such a scale.
   4. Scoring techniques often presume that the factors being scored are independent of each other independence – i.e. there are no correlations between factors. This assumption  is rarely tested or justified in any way. 

Many project management standards advocate the use of scoring techniques.  To be fair, in many situations they are adequate as long as they are used with an understanding of their limitations. Seen in this light, Hubbard’s book is  an admonition to standards and textbook writers to be more critical of the methods they advocate, and a warning to practitioners that an uncritical adherence to standards and best practices is not the best way to manage project risks .

Scoring done right

Just to be clear, Hubbard’s criticism is directed against  scoring methods that use arbitrary, qualitative scales which are not justified by independent analysis. There are other techniques which, though superficially similar to these flawed scoring methods, are actually quite robust because they are:

1. Based on observations. 
2. Use real measures (as opposed to arbitrary ones – such as “alignment with business objectives” on a scale of 1 to 5, without defining what ”alignment” means.) 
3. Validated after the fact (and hence refined with use).

As an example  of a sound scoring technique, Hubbard quotes this paper by Dawes, which presents evidence that linear scoring models are superior to intuition in clinical judgements. Strangely, although the weights themselves can be obtained through intuition, the scoring model outperforms clinical intuition. This happens because human intuition is good at identifying important factors, but not so hot at evaluating the net effect of several, possibly competing factors. Hence simple linear scoring models can outperform intuition. The key here is that the models are validated by checking the predictions against reality.

Another class of techniques use axioms based on logic to reduce inconsistencies in decisions. An example of such a technique is multi-attribute utility theory. Since they are based on logic, these methods can also be considered to have a solid foundation unlike those discussed in the previous section.

 Conclusions

 Many commonly used scoring methods in risk analysis are based on flaky theoretical foundations – or worse, none at all. To compound the problem, they are often used without any validation.  A particularly ubiquitous example is the well-known and loved risk matrix.  In his paper on risk matrices,  Tony Cox  shows how risk matrices can sometimes lead to decisions that are worse than those made on the basis of a coin toss.   The fact that this is a possibility – even if only a  small one – should worry anyone who uses risk matrices  (or other flawed scoring techniques) without an understanding of their limitations.

Introduction 

Conventional approaches to knowledge management on projects focus on  the cognitive (or thought-related) and mechanical  aspects of knowledge creation and capture. There is alternate view, one which considers knowledge as being created through interactions between people who –  through their interactions -  develop mutually acceptable interpretations of theories and facts in ways that suit their particular needs. That is, project knowledge is socially constructed. If this is true, then project managers need to pay attention to the environmental and social factors that influence knowledge construction.  This is the position taken by Paul Jackson and Jane Klobas in their paper entitled, Building knowledge in projects: A practical application of social constructivism to information systems development, which presents a  knowledge creation / sharing process model based social constructivist theory. This article is a summary and review of the paper.

 A social constructivist view of knowledge

Jackson and Klobas begin with the observation that engineering disciplines are founded on the belief that knowledge can be expressed in propositions that correspond to a reality which  is independent of human perception.  However, there is an alternate view that knowledge is not absolute, but relative – i.e.  it depends on the mental models and beliefs used to interpret facts, objects and events. A  relevant example is how a software product is viewed by business users and software developers. The former group may see an application in terms of its utility whereas the latter may see it as an instance of a particular technology. Such perception gaps can also occur within seemingly homogenous groups – such as teams comprised of software developers, for example. This can happen for a variety of reasons such as the differences in the experience and cultural backgrounds of those who make up the group. Social constructivism looks at how such gaps can be bridged.

The authors’ discussion relies on the work of Berger and Luckmann, who described how the gap between perceptions of different individuals can be overcome to create a socially constructed, shared reality. The phrase “socially constructed” implies that reality (as it pertains to a project, for example) is created via a common understanding of issues, followed by mutual agreement between all the players as to what comprises that reality. For me this view strikes a particular chord because of it is akin to the stated aims of dialogue mapping, a technique that I have described in several earlier posts (see this article for an example relevant to projects).

 Knowledge in information systems development as a social construct

 First up, the authors make the point that information systems development (ISD) projects are:

 …intensive exercises in constructing social reality through process and data modeling. These models are informed with the particular world-view of systems designers and their use of particular formal representations. In ISD projects, this operational reality is new and explicitly constructed and becomes understood and accepted through negotiated agreement between participants from the two cultures of business and IT

 Essentially, knowledge emerges  through interaction and discussion   as the project proceeds.  However, the methodologies used in design are typically founded on an engineering approach, which takes a positivist view rather than a social one. As the authors suggest,

Perhaps the social constructivist paradigm offers an insight into continuing failure, namely that what is happening in an ISD project is far more complex than the simple translation of a description of an external reality into instructions for a computer. It is the emergence and articulation of multiple, indeterminate, sometimes unconscious, sometimes ineffable realities and the negotiated achievement of a consensus of a new, agreed reality in an explicit form, such as a business or data model, which is amenable to computerization.

With this in mind, the authors aim to develop a model that addresses the shortcomings of the traditional, positivist view of knowledge in ISD projects. They do this by representing Berger and Luckmann’s theory of social constructivism in terms of a knowledge process model. They then identify management principles that map on to these processes. These principles form the basis of a survey which is used as an operational version of the process model. The operational model is then assessed by experts and tested by a project manager in a real-life project.

The knowledge creation/sharing process model

The process model that Jackson and Klobas describe is based on Berger and Luckmann’s work.

The model  describes how personal knowledge is created – personal knowledge being what an individual knows. Personal knowledge is built up using mental models of the world – these models are frameworks that individuals use to make sense of the world.

According to the Jackson-Klobas process model, personal knowledge is built up through a number of process including:

 Internalisation: The absorption of knowledge by an individual

 Knowledge creation: The construction of new knowledge through repetitive performance of tasks (learning skills) or becoming aware of new ideas, ways of thinking or frameworks. The latter corresponds to learning concepts and theories, or even new ways of perceiving the world. These correspond to a change in subjective reality for the individual.

 Externalisation: The representation and description of knowledge using speech or symbols so that it can be perceived and internalized by others. Think of this as explaining ideas or procedures to other individuals.

 Objectivation: The creation of a shared constructs that represent a group’s understanding of the world. At this point, knowledge is objectified – and is perceived as having an existence independent of individuals.

 Legitimation: The authorization of objectified knowledge as being “correct” or “standard.”

 Reification: The process by which objective knowledge assumes a status that makes it difficult to change or challenge. A familiar example of reified knowledge is any procedure or process that is “hardened” into a system – “That’s just the way things are done around here,” is a common response when such processes are challenged.

  The links depicted in the figure show the relationships between these processes.

 Jackson and Klobas suggest that knowledge creation in ISD projects is a social process, which occurs through continual communication between the business and IT. Sure, there are other elements of knowledge creation – design, prototyping, development, learning new skills etc. – but these amount to nought unless they are discussed, argued, agreed on and communicated through social interactions. These interactions occur in the wider context of the organization, so it is reasonable to claim that the resulting knowledge takes on a form that mirrors the social environment of the organization.

 Clearly, this model of knowledge creation is very different from the usual interpretation of knowledge having an independent reality, regardless of whether it is known to the group or not.

 An operational model

 The above is good theory, which makes for interesting, but academic, discussions. What about practice? Can the model be operationalised?  Jackson and Klobas describe an approach to creating to testing the utility (rather than the validity) of the model.  I discuss this in the following sections.

 Knowledge sharing heuristics

 To begin with, they surveyed the literature on knowledge management to identify knowledge sharing heuristics (i.e. experience-based techniques to enable knowledge sharing).  As an example, some of the heuristics associated with the externalization process were:

• We have standard documentation and modelling tools which make business requirements easy to understand
• Stakeholders and IS staff communicate regularly through direct face-to-face contact
• We use prototypes

 The authors identified more than 130 heuristics. Each of these was matched with a process in the model. According to the authors, this matching process was simple: in most cases there was no doubt as to which process a heuristic should be attached to. This suggests that the model provides a natural way to organize the voluminous and complex body of research in knowledge creation and sharing. Why is this important? Well, because it suggests that the conceptual model (as illustrated in Fig. 1) can form the basis for a simple means to assess knowledge creation / sharing capabilities in their work environments, with the assurance that they have all relevant variables covered.

 Validating the mapping

 The validity of the matching was checked using twenty historical case studies of ISD projects. This worked as follows: explanations for what worked well and what didn’t were mapped against the model process areas (using the heuristics identified in the prior step). The aim was to answer the question:   “is there a relationship between project failure and problems in the respective knowledge processes or, conversely, between project success and the presence of positive indicators?” 

 One of the case studies the authors use is the well-known (and possibly over-analysed) failure of the automated dispatch system for the London Ambulance Service.  The paper has a succinct summary of the case study, which I reproduce below: 

The London Ambulance Service (LAS) is the largest ambulance service in the world and provides accident and emergency and patient transport services to a resident population of nearly seven million people. Their ISD project was intended to produce an automated system for the dispatch of ambulances to emergencies. The existing manual system was poor, cumbersome, inefficient and relatively unreliable. The goal of the new system was to provide an efficient command and control process to overcome these deficiencies. Furthermore, the system was seen by management as an opportunity to resolve perceived issues in poor industrial relations, outmoded work practices and low resource utilization. A tender was let for development of system components including computer aided dispatch, automatic vehicle location, radio interfacing and mobile data terminals to update the status of any call-out. The tender was let to a company inexperienced in large systems delivery. Whilst the project had profound implications for work practices, personnel were hardly involved in the design of the system. Upon implementation, there were many errors in the software and infrastructure, which led to critical operational shortcomings such as the failure of calls to reach ambulances. The system lasted only a week before it was necessary to revert to the manual system.

 Jackson and Klobas show how their conceptual model maps to knowledge-related factors that may have played a role in the failure project. For example, under the heading of personal knowledge, one can identify at least two potential factors: lack of involvement of end-users in design and selection of an inexperienced vendor. Further, the disconnect between management and employees suggests a couple of factors relating to reification: mutual negative perceptions and outmoded (but unchallenged) work practices.

 From their validation, the authors suggest that the model provides a comprehensive framework that explains why these projects failed. That may be overstating the case – what’s cause and what’s effect is hard to tell, especially after the fact. Nonetheless, the model does seem to be able to capture many, if not all, knowledge-related gaps that could have played a role in these failures. Further, by looking at the heuristics mapped to each process, one might be able to suggest ways in which these deficiencies could have been addressed. For example, if externalization is a problem area one might suggest the use of prototypes or encourage face to face communication between IS and business personnel.

 Survey-based tool

 Encouraged by the above, the authors created a survey tool which was intended to evaluate knowledge creation/sharing effectiveness in project environments. In the tool, academic terms used in the model were translated into everyday language (for example, the term externalization was translated to knowledge sharing – see Fig 1 for translated terms). The tool asked project managers to evaluate their project environments against each knowledge creation process (or capability) on a scale of 1 to 10.   Based on inputs, it could recommend specific improvement strategies for capabilities that were scored low. The tool was evaluated by four project managers, who used it in their work environment over a period of 4-6 weeks. At the end of the period, they were interviewed and their responses were analysed using content analysis to match their experiences and requirements against the designed intent of the tool.  Unfortunately, the paper does not provide any details about the tool, so it’s difficult to say much more than paraphrase the authors comments.

 Based on their evaluation, the authors conclude that the tool provides:

1. A common framework for project managers to discuss issues pertaining to knowledge creation and sharing.
2. A means to identify potential problems and what might be done to address them.

 Field testing 

One of the evaluators of the model tested the tool in the field. The tester was a project manager who wanted to identify knowledge creation/sharing deficiencies in his work environment, and ways in which these could be addressed.  He answered questions based on his own evaluation of knowledge sharing capabilities in his environment and then developed an improvement plan based on strategies suggested by the tool along with some of his own ideas.  The completed survey and plan were returned to the researchers.

 Use of the tool revealed the following knowledge creation/sharing deficiencies in the project manager’s environment:

1. Inadequate personal knowledge.
2. Ineffective externalization
3. Inadequate standardization (objectivation)

Strategies suggested by the tool include:

1. An internet portal to promote knowledge capture and sharing. This included discussion forums, areas to capture and discuss best practices etc.
2. Role playing workshops to reveal how processes worked in practice (i.e. surface tacit knowledge).

Based on the above, the authors suggest that:

1. Technology can be used to promote support knowledge sharing and standardization, not just storage.
2. Interventions that make tacit knowledge explicit can be helpful.
3. As a side benefit, they note that the survey has raised consciousness about knowledge creation/sharing within the team.

Reflections and Conclusions

In my opinion, the value of the paper lies not in the model or the survey tool, but the conceptual framework that underpins them – namely, the idea knowledge depends on, and is shaped by, the social environment in which it evolves. Perhaps an example might help clarify what this means. Consider an organisation that decides to implement project management “best practices” as described by <fill in any of the popular methodologies here>. The wrong way to do this would be to implement practices wholesale, without regard to organizational culture, norms and pre-existing practices. Such an approach is unlikely to lead to the imposed practices taking root in the organisation. On the other hand, an approach that picks the practices that are useful and tailors these to organizational needs, constraints and culture is likely to meet with more success. The second approach works because it attempts to bridge gap between the “ideal best practice” and social reality in the organisation. It encourages employees to adapt practices in ways that make sense in the context of the organization. This invariably involves modifying practices, sometimes substantially, creating new (socially constructed!) knowledge in the bargain. 

Another interesting point the authors make is that several knowledge sharing heuristics (130, I think the number was) could be classified unambiguously under one of the processes in the model. This suggests that the model is a reasonable view of the knowledge creation/sharing process. If one accepts this conclusion, then the model does indeed provide a common framework for discussing issues relating knowledge creation in project environments. Further, the associated heuristics can help identify processes that don’t work well.

  I’m unable to judge the usefulness of the survey-based tool developed by the authors because they do not provide much detail about it in the paper. However, that isn’t really an issue;  the field of project management has too many “tools and techniques” anyway.  The key message of the paper, in my opinion, is the that every project has a unique context, and that the techniques used by others have to be interpreted and applied in ways that are meaningful in the context of the particular project. The paper is an excellent counterpoint to the methodology-oriented practice of knowledge management in projects; it should be required reading for methodologists and  project managers who believe that things need to be done by The Book, regardless of social or organizational context.

Introduction

In my previous post I  demonstrated the use of a Monte Carlo technique in simulating  a single project task with completion times  described by a triangular distribution.  My aim in that article was to:  a) describe a Monte Carlo simulation procedure in enough detail for someone interested to be able to reproduce the calculations and b) show that it gives sensible results in a situation where the answer is known.  Now it’s time to take things further.  In this post, I present simulations for two tasks chained together in various ways.  We shall see that, even with this small increase in complexity (from one task to two), the results obtained can be surprising.  Specifically, small changes in inter-task dependencies can have a huge effect on the overall (two-task) completion time distribution. Although, this is something that that most project managers have experienced in real life, it is rarely taken in to account by traditional scheduling techniques. As we shall see, Monte Carlo techniques predict such effects as a matter of course.

Background

The three simulations discussed here are built on the example that I used in my previous article, so it’s worth spending a few lines for a brief recap of that example.  The task simulated in the example was assumed to be described by a triangular distribution with  minimum completion time () of 2  hours,   most likely completion time () of 4 hours and   a maximum completion time () of 8 hours.   The resulting triangular probability distribution function (PDF),  –  which gives the probability of completing the task at time t – is shown in Figure 1.

Figure 2 depicts the associated cumulative distribution function (CDF),   which gives the probability that a task will be completed by time t (as opposed to the PDF which specifies the probability of completion at time t). The value of the CDF at t=8 is 1 because the task must finish within 8 hrs.

The equations describing the PDF and CDF are listed in equations 4-7 of my previous article.  I won’t rehash them here as they don’t add anything new  to the discussion – please see the article for all the gory algebraic details and formulas.   Now, with that background, we’re ready to move on to the examples.

Two tasks in series with no inter-task delay

As a first example, let’s look at two tasks that have to be performed sequentially – i.e. the second task starts as soon as the first one is completed. To simplify things, we’ll also assume that they have identical (triangular) distributions as described earlier and shown in Figure 1  (excepting , of course,  that the distribution is shifted to the right for the second task  – since it starts after the first one finishes).  We’ll also  assume that the second task begins right after the first one is completed (no inter-task delay) – yes, this is unrealistic, but bear with me.  The simulation algorithm for the  combined tasks is very similar to the one for a single task (described in detail in my previous post). Here’s the procedure:

1. For each of the two tasks, generate a set of N random numbers. Each random number generated corresponds to the cumulative probability of completion for a single task on that particular run.
2. For each random number generated, find the time corresponding to the cumulative probability by solving equation 6 or 7 in my previous post.
3. Step 2 gives N sets of completion times. Each set has two completion times – one for each tasks.
4. Add up the two numbers in each set to yield the comple. The resulting set corresponds to N simulation runs for the combined task.

I then divided the time interval from t=4 hours  (min possible completion time for both tasks) to t=16 hours (max possible completion time for both tasks) into bins of 0.25 hrs each, and then assigned each combined completion time to the appropriate bin. For example, if the predicted completion time for a particular run was 9.806532 hrs, it was assigned to the bin corresponding to 0.975 hrs.  The resulting histogram is shown in Figure 3 below (click on image to view the full-size graphic).

[An aside:  compare the histogram in Figure 3 to the one for a single task (Figure 1):  the distribution for the single task is distinctly asymmetric (the peak is not at the centre of the distribution) whereas the two task histogram is nearly symmetric.  This surprising result is a consequence of the Central Limit Theorem (CLT) – which states that the sum of many identical distributions tends to resemble the Normal (Bell-shaped) distribution, regardless of the shape of the individual distributions.   Note that the CLT holds even though the two task distributions are shifted relative to each other – i.e. the second task begins after the first one is completed.]

The simulation also enables us to compute the cumulative probability of completion for the combined tasks (the CDF). This value of the cumulative probability at a particular bin equals the sum of the number of simulations runs in every bin up to (and including) the bin in question, divided by the total number of simulation runs. In mathematical terms this is:

where   is the cumulative probability at the time corresponding to the  th  bin, , the number of simulation runs in the th  bin and    the total number of simulation runs. Note that this formula is an approximate one because time is treated as a constant within each bin. The approximation can be improved by making the bins smaller (and hence increasing the number of bins).

The resulting cumulative probability function is shown in Figure 4. This allows us to answer questions such as:  “What is the probability that the tasks will be completed within 10 days?”. Answer:  .698, or approximately 70%. (Note:  I obtained this number by interpolating between values obtained from equation (1), but this level of precision is uncalled for, particularly because the formula is an approximate one)

Many project scheduling techniques compute average completion times for component tasks and then add them up to get the expected completion time for the combined task. In the present case the average works out to 9.33 hrs (twice the average for a single task). However, we see from the CDF that there is a significant probability (.43) that we will not finish by this time – and this in a “best-case ” situation where the second task starts immediately after the first one finishes!

[An aside: If one applies the  well-known PERT formula   to each of the tasks, one gets an expected completion time  of 8.66 hrs for the combined task.  From the CDF one can show that there is a  probability of non-completion of 57%  by t=8.66 hours (see Figure 4) - i.e. there’s a greater than even chance of not finishing by this time!]

As interesting as this case is, it is somewhat unrealistic because successor tasks seldom start the instant the predecessor is completed. More often than not, there is a cut-off time before which the successor cannot start – even if there are no explicit dependencies between the two tasks.  This observation is a perfect segue into my next example, which is…

Two tasks in series with a fixed earliest start for the successor

Now we’ll introduce a slight complication: let’s assume, as before, that the two tasks are done one after the other but that the earliest the second task can start is at hours  (as measured from the start of the first task). So, if the first task finishes within 6 hours, there will be a delay between its completion and the start of the second task. However, if the first task takes longer than 6 hours to finish, the second task will start soon after the first one finishes.  The simulation procedure is the same as described in the previous section excepting for the last step – the completion time for the combined task is given by:

, for t   6 hrs and , for t < 6 hrs

I divided the time interval from t=4hrs to t=20 hrs into bins of 0.25 hr duration (much as I did before) and then assigned each combined completion time to the appropriate bin. The resulting histogram is shown in Figure 5.

Comparing Figure 5 to Figure 3, we see that the earliest possible finish time now increases from 4 hrs to 8 hrs. This is no surprise, as we built this in to our assumption.  Further, as one might expect, the distribution is distinctly asymmetric – with a minimum completion time of 8 hrs, a most likely time between 10 and 11 hrs and a maximum completion time of about 15 hrs.

Figure 6 shows the cumulative probability of completion for this case.

Because of the delay condition, it is impossible to calculate the average completion from the formulas for the triangular distribution – we have to obtain it from the simulation.  The average can be calculated from the simulation adding up all completion times and dividing by the total number of simulations, . In mathematical terms this is:

where is the average time,    the completion time for the th simulation run and    the total number of simulation runs.

This procedure gave me a value of about 10.8  hrs for the average.  From the CDF in Figure 6 one sees that the probability that the combined task will finish by this time is only 0.60 – i.e. there’s only a 60% chance that the task will finish by this time.  Any naïve estimation of time would do just as badly unless, of course, one is overly pessimistic and assumes a completion time of 15 – 16 hrs.

From the above it should be evident that the simulation allows one to associate an uncertainty (or probability) with every estimate. If management imposes a time limit of 10 hours,  the project manager can refer to the CDF in Figure 6 and figure out the probability of completing the task by that time (there’s a 40 % chance of completion by 10 hrs).  Of course, the reliability of the numbers depend on how good the distribution is. But  the assumptions that have gone into the model are known -  the optimistic, most likely and pessimistic times and the form of the distribution - and these can be refined as one gains experience.

Two tasks in parallel

My final example is the case of two identical tasks performed in parallel. As above, I’ll assume the uncertainty in each task is characterized by a triangular distribution with ,   and  of 2, 4 and 8 hours respectively. The simulation procedure for this case is the same as in the first example, excepting the last step. Assuming the simulated completion times for the individual tasks are and , the completion time for the combined tasks is given by the greater of the two – i.e. the combined completion time is given by .

To plot the histogram shown in Figure 7 , I divided the interval from t=2 hrs to t=8 hrs into bins of 0.25 hr duration each (Warning: Note the difference in the time axis scale  from Figures 3 and 5!).

It is interesting to compare the above  histogram with that for an individual task with the same parameters (i.e. the example that was used in my previous post). Figure 8 shows the histograms for the two examples on the same plot (the combined task in red and the single task in blue). As one might expect, the histogram for the combined task is shifted to the right, a consequence of the nonlinear condition on the completion time.

What about the average? I calculated the average as before, by using equation (2) from the previous section. This gives an average of 5.38 hrs (compared to 4.67 hrs for either task, taken individually).   Note that the method to calculate the average is the same regardless of the form of the distribution. On the other hand,  computing the average from the equations would be a complicated affair, involving a stiff dose of algebra with an optional  sprinkling of  calculus.  Even worse – the calculations would vary from distribution to distribution. There’s no question that simulations are much easier.

The CDF for the overall completion time is also computed easily using equation (1). The resulting plot is shown in Figure 9  (Note the difference in the time axis scale  from Figures  4 and 6!). There are no surprises here – excepting how easy it is to calculate once the simulation is done.

Let’s see what time corresponds to a 90% certainty of completion. A rough estimate for this number can be obtained from Figure 9 – just find the value of t (on the x axis) corresponding to a cumulative probability of 0.9 (on the y axis).  This is the graphical equivalent of solving the CDF for time, given the cumulative probability is 0.9. From Figure 9, we get a time of approximately 6.7 hrs. [Note: we could get a more accurate number by fitting the points obtained from equation (1) to a curve and then calculating the time corresponding to ]. The interesting thing is that the 90% certain completion time is not too different from that of a single task (as calculated from equation 7 of my previous post) – which works out to 6.45 hrs.

Comparing the two histograms in Figure 8, we expect the biggest differences in cumulative probability to occur at about the t=4 hour mark, because by that time the probability for the individual task has peaked whereas that for the combined task is yet to peak. Let’s see if this is so: from Figure 8, the cumulative probability for t=4  hrs is about .15 and from the CDF for the triangular distribution (equation 6 from my previous post), the cumulative probability at t=4 hours  (which is the most likely time) is .333 – double that of the combined task.  This, again, isn’t too surprising (once one has Figure 8 on hand). The really nice thing is that we are able to attach uncertainties to all our estimates.

Conclusion

Although the examples discussed above are simple – two identical tasks with uncertainties described by a triangular distribution – they serve to illustrate some of the non-intuitive outcomes when tasks have dependencies.   It is also worth noting that although the distribution for the individual tasks is known, the only straightforward way to obtain the distributions for the combined tasks (figures 3, 5 and 7) is through simulations. So, even these simple examples are a good demonstration of the utility of Monte Carlo techniques. Of course, real projects are way more complicated, with diverse tasks distributed in many different ways.   To simplify simulations in such cases,  one could  perform  coarse-grained simulations on a small number of high-level tasks,  each consisting of a number of  low-level, atomic tasks. The high-level tasks could be constructed in such a way as to focus attention on areas of greatest complexity, and hence greatest uncertainty.

As I have mentioned several times in this article and the previous one: simulation results are only as good as the distributions on which they are based. This begs the question: how do we know what’s an appropriate distribution for a given situation? There’s no one-size-fits-all answer to this question. However, for project tasks there are some general considerations that apply. These are:

1. There is a minimum time () before which a task cannot cannot be completed.
2. The probability will increase from 0 at to a maximum at a “most likely” completion time, . This holds true for most atomic tasks – but may be not for composite tasks which consist of many smaller tasks.
3. The probability decreases as time increases beyond ,  falling to 0 at a time much larger than .   This is simply another way of saying that the distribution has a long (but not necessarily infinite!) tail.

Asymmetric triangular distributions (such as the one used in my examples) are the simplest distributions that satisfy these conditions. Furthermore, a three point estimate is enought to specify a triangular distribution completely – i.e. given a three point estimate there is only one triangular distribution that can be fitted to it. That said, there are several other distributions that can be used; of particular relevance are certain long-tailed distributions.

Finally, I should mention that I make no claims about the efficiency or accuracy of the method presented here:  it should be seen  as  a demonstration rather than a definitive technique.  The many commercial Monte Carlo tools available in the market probably offer far more comprehensive, sophisticated and reliable algorithms (Note:  I ‘ve never used any of them, so I can’t make any recommendations!).  That said, it is always helpful to know the principles behind such tools,  if for no other reason than to understand how they work and, more important,  how to use them correctly.  The material discussed in this and the previous article came out of my efforts to develop an understanding Monte Carlo techniques and how they can be applied to various aspects of project management (they can also be applied to cost estimation, for example).  Over the last few weeks  I’ve spent many enjoyable evenings developing and running these simulations, and learning from them.  I’ll  leave it here with the hope that you find my articles helpful in your own explorations of the wonderful world of Monte Carlo simulations.