Abbe Boulah’s Weblog

An effort to clarify the role of deliberative evaluation in the planning and policy-making process.

Thorbjoern Mann

JUDGMENT SCALES

Differences of judgments are expressed  on  s c a l e s  — ‘yardsticks’ on which to locate different judgment ‘visually’.  There are many different kinds of judgment scales.

Which kind of scale should be chosen and agreed upon in a specific evaluation situation depends on the purpose of the task: whether a decision for action (e.g. acceptance or rejection) about a single plan is called for, or a selection among a number of competing proposals or options; or a general expression of  appraisal (e.g. goodness for some purpose, perhaps to guide design in a ‘program’ sense or to improve a proposed plan).  For some purposes ,  like the ‘acceptance/ rejection’ decision, the scale needed will  be ‘binary’ , — have only two ‘values’ or at most three:  ‘Yes, No, Undecided’.  For other purposes, such as comparison between alternative proposals, scales with more such ‘values’  are needed.

Traditionally (as in science), four types of scales are distinguished: 

‘N o m i n a l’,  ‘O r d i n a l’ ,   ‘D i f f e r e n c e’  (or  ‘I n t e r v a l’)  and ‘R a t i o‘  scales.

The Nominal scale (not really an ‘ordered’ scale since its values can be in any order) contains just ‘names’ of different kinds of objects or options must be distinguished. Architectural examples are ‘single-family detached home’, ‘duplex’, ‘row house’, ‘multistory  apartment’ etc. 

Ordinal scales , as the name suggests, put its items in a distinct order — for example ‘first place, second place, third place’ etc. in sports — but without any information about how much faster the winner of the race was than the runner-up. Just rank order.   

Difference or Interval scales offer more detailed ‘quantitative’ information. They specify      u n i t s — but these are arbitrary entities on a scale with an arbitrary location of ‘zero’. The temperature scales are examples: the Fahrenheit scale sets the‘zero’ degrees point (the temperature on the coldest day Mr. Fahrenheit had ever experienced and could not imagine being any lower) at what in the Celsius scale becomes  ~minus 17.8‘degrees. Celsius ‘zero’ is the temperature of water freezing. Fahrenheit’s 100 degrees is the approximate temperature of human blood, a mere 37.8 degrees Celsius, where 100 degrees is the temperature of water boiling, which in Fahrenheit becomes 212 degrees.

The Ratio scale has a ‘natural’  zero point (e.g. human height) but also arbitrary units — it can be measured in feet and inches or centimeters, etc.. 

The more ‘o b j e c t i v e’ , ‘factual’  and ‘scientific’  judgments we wish to express, the more we will tend to use difference scales a least and at best ratio scales that allow precise         m e a s u r e m e n t  rather than  s u b j e c t i v e  opinions, guesses and  estimates.

For judgment purposes, this means that scales for expressing ‘goodness’ and similar judgments, the scales will be at the difference scale level at best. Agreements are needed about whether judgment scales are to be ‘one-directional’  e.g.  from 0 upwards, or ‘bi-directional’, that is, showing both positive and negative ‘d i s c r e t e’  or ‘c o n t i n u o u s’  values. They can be ‘unbounded’  (going to infinity) or ‘bounded‘  with a distinct  upper ‘couldn’t be better‘ or lower (‘couldn’t be worse’) number. 

All these choices, and the m e a n i n g of the points on a chosen scale,  must be agreed upon for any particular  task, to avoid misunderstanding and conflicts.

The map above shows distinctions of judgment scales that perhaps gives a wrong impression that these are  mutually exclusive. In reality, they are combined even when judgments are ‘atomic’ (that is, one judgment applied to one object, and especially when two or more judgments are expressed in ‘compound‘ judgments where several evaluation judgments are applied to the object. This variety of possible judgment types and scales are better shown in a ‘Zwicky Box’ type  or ‘morphological analysis’ diagram, as in the following diagram where the type categories are shown as ‘parameters’ and the members of the categories are parameter values, and each particular scale is described by the profile linking several values:

The profiles show two typical ‘measurement’ scales  and two common judgment scales: A: the temperature scale and  B  the movement speed in miles per hour; C,  the common academic grading scale, and D, the bidirectional ‘goodness’ judgment scale according to Musso and Rittel.

–o–