Eight to Late

Sensemaking and Analytics for Organizations

Value judgements in system design

leave a comment »

Introduction

How do we choose between competing design proposals for information systems? In principle this should be done using an evaluation process based on objective criteria. In practice, though, people generally make choices based on their interests and preferences. These can differ widely, so decisions are often made on the basis of criteria that do not satisfy the interests of all stakeholders. Consequently, once a system becomes operational, complaints from stakeholder groups whose interests were overlooked are almost inevitable (Just think back to any system implementation that you were involved in…).

The point is: choosing between designs is not a purely technical issue; it also involves value judgements – what’s right and what’s wrong – or even, what’s good and what’s not. Problem is, this is a deeply personal matter – different folks have different values and, consequently, differing ideas of which design ideal is best (Note: the term ideal refers to the value judgements associated with a design). Ten years ago, Heinz Klein and Rudy Hirschheim addressed this issue in a landmark paper entitled, Choosing Between Competing Design Ideals in Information Systems Development. This post is a summary of the main ideas presented in their paper.

Design ideals and deliberation

A design ideal is not an abstract, philosophical concept. The notion of good and bad or right and wrong can be applied to the technical, economic and social aspects of a system. For example, a choice between building and buying a system has different economic and social consequences for stakeholder groups within and outside the organisation. What’s more, the competing ideals may be in conflict – developers employed in the organisation would obviously prefer to build rather than buy because their employment depends on it; management, however, may have a very different take on the issue.

The essential point that Hirschheim and Klein make is that differences in values can be reconciled only through honest discussion. They propose a deliberative approach wherein all stakeholders discuss issues in order to come to an agreement. To this end, they draw on the theories of argumentation and communicative rationality to come up with a rational framework for comparing design ideals.  Since these terms are new, I’ll spend a couple of paragraphs in describing them briefly.

Argumentation is essentially reasoned debate – i.e. the process of reaching conclusions via arguments that use informal logic – which, according to the definition in the foregoing link, is the attempt to develop a logic to assess, analyse and improve ordinary language (or “everyday”) reasoning. Hirschheim and Klein use the argumentation framework proposed by Stephen Toulmin, to illustrate their approach.

The basic premise of communicative rationality is that rationality (or reason) is tied to social interactions and dialogue. In other words, the exercise of reason can  occur only through dialogue.  Such communication, or mutual deliberation, ought to result in a general agreement about the issues under discussion.  Only once such agreement is achieved can there be a consensus on actions that need to be taken.  See my article on rational dialogue in project environments for more on communicative rationality.

Obstacles to rational dialogue and how to overcome them

The key point about communicative rationality is that it assumes the following conditions hold:

  1. Inclusion:  includes all stakeholders.
  2. Autonomy: all participants should be able to present and criticise views without fear of reprisals.
  3. Empathy: participants must be willing to listen to and understand claims made by others.
  4. Power neutrality: power differences (levels of authority) between participants should not affect the discussion.
  5. Transparency: participants must not indulge in strategic actions (i.e. lying!).

Clearly these are idealistic conditions, difficult to achieve in any real organisation.  Klein and Hirschheim acknowledge this point, and note the following barriers to rationality in organisational decision making:

  1. Social barriers: These include inequalities (between individuals) in power, status, education and resources.
  2. Motivational barriers: This refers to  the psychological cost of prolonged debate.  After a period of sustained debate, people will often cave in just to stop arguing even though they may have the better argument.
  3. Economic barriers: Time is money: most organisations cannot afford a prolonged debate on contentious issues.
  4. Personality differences: How often is it that the most charismatic or articulate person gets their way, and the quiet guy in the corner (with a good idea or two) is completely overlooked?
  5. Linguistic barriers: This refers to the difficulty of formulating arguments in a way that makes sense to the listener. This involves, among other things, the ability to present ideas in a way that is succinct, without glossing over the important issue – a skill not possessed by many.

These barriers will come as no surprise to most readers. It will be just as unsurprising that it is difficult to overcome them. Klein and Hirschheim offer the usual solutions including:

  1. Encourage open debate – They suggest the use of technologies that support collaboration. They can be forgiven for their optimism given that the paper was written a decade ago, but the fact of the matter is that all the technologies that have sprouted since have done little to encourage open debate.
  2. Implement group decision techniques –  these include arrangements such as quality circles, nominal groups and constructive controversy. However, these too will not work unless people feel safe enough to articulate their opinions freely.

Even though the barriers to open dialogue are daunting, it behooves system designers to strive towards reducing or removing them.  There are effective ways to do this, but that’s a topic I won’t go into here as it has been dealt with at length elsewhere.

Principles for arguments about value judgements

So, assuming the environment is right, how should we debate value judgements?  Klein and Hirschheim recommend using informal logic supplemented with ethical principles. Let’s look at these two elements briefly.

Informal logic is a means to reason about human concerns. Typically, in these issues there is no clear cut notion of truth and falsity. Toulmin’s argumentation framework (mentioned earlier in this post) tells us how arguments about such issues should be constructed. It consists of the following elements:

  1. Claim:   A statement that one party asks another to accept as true. An example would be my claim that I did not show up to work yesterday because I was not well.
  2. Data (Evidence): The basis on which one party expects to other to accept a claim as true. To back the claim made in the previous line, I might draw attention to my runny nose and hoarse voice.
  3. Warrant:  The bridge between the data and the claim.  Again, using the same example, a warrant would be that I look drawn today, so it is likely that I really was sick yesterday.
  4. Backing: Further evidence, if the warrant should prove insufficient. If my boss is unconvinced by my appearance he may insist on a doctor certificate.
  5. Qualifier: These are words that express a degree of certainty about the claim. For instance, to emphasise just how sick I was, I might tell my boss  that I stayed in bed all day because I had  high fever.

This is all quite theoretical: when we debate issues we do not stop to think whether a statement is a warrant or a backing or something else; we just get on with the argument. Nevertheless, knowledge of informal logic can help us construct better arguments for our positions. Further, at the practical level there are computer supported deliberative techniques such as argument mapping and dialogue mapping which can assist in structuring and capturing such arguments.

The other element is ethics: Klein and Hirschheim  contend that moral and ethical principles ought to be considered when value judgements are being evaluated. These principles include:

  1. Ought implies can – which essentially means that one morally ought to do something only if one can do it (see this paper for an interesting counterview of this principle). Taking the negation of this statement, one gets “Cannot implies ought not” which means that a design can be criticised if it involves doing something that is (demonstrably) impossible – or makes impossible demands on certain parties.
  2. Conflicting principles – This is best explained via an example. Consider a system that saves an organisation money but involves putting a large number of people out of work. In this case we would have an economic principle coming up against a social one. According to the principle, a design ideal based on an economic imperative can be criticised on social grounds.
  3. The principle of reconsideration – This implies reconsidering decisions if relevant new information becomes available. For example, if it is found that a particular design overlooked a certain group of users, then the design should be reconsidered in the light of their needs.

They also mention that new ethical and moral theories may trigger the principle of reconsideration. In my opinion, however, this is a relatively rare occurrence:  how often are new ethical or moral theories proposed?

Summarising

The main point made by the authors is that system design involves value judgements. Since these are largely subjective,  open debate using the principles of informal logic is the best way to deal with conflicting values. Moreover, since such issues are not entirely technical, one has to use ethical principles to guide debate.  These principles – not asking people to do the impossible; taking everyone’s interests into account and reconsidering decisions in the light of new information – are reasonable if not self-evident. However, as obvious as they are, they are often ignored in design deliberations. Hirschheim and Klein do us a great service by reminding us of their importance.

Written by K

March 31, 2011 at 10:16 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: