Archive for November, 2019


Thorbjoern Mann

An effort to clarify the role of deliberative evaluation in the planning and policy-making process.

Many aspects of evaluation-related tasks in familiar approaches and practice, call for some re-assessment and improvement even for practical applications in current situations. These will be discussed in more detail in sections addressing requirements and tools for practical application. Others are more significant in that they end up questioning the entire concept of deliberative evaluation in planning on a ‘philosophical’ level, or resist adopting smaller detail improvements of the first (practical) kind because they may mean abandoning familiar habits based on tradition and even constitutional provisions.

The very concept of deliberative evaluation — as materialized in procedures and practices that look too cumbersome, bureaucratic and elitist ‘expert-model‘ to many — is an example of a fundamental issue that can significantly flavor and complicate planning discourse. The desire to do without such ‘methods’ is theoretically and emotionally supported by concepts such as the civic, patriotic, call and need for consensus, unity of purpose and even ideas such as swarm behavior or ‘wisdom of the crowds’ that claim to more effortlessly produce ‘good’ solutions and community behavior. A related example is the philosophy behind Christopher Alexander’s ‘Pattern Language’ . Does its claim that using patterns declared ‘valid’ and ‘good’ (having ‘Quality Without a Name — ‘QWAN’) in developing plans and solutions, e.g. for buildings and neighborhoods, will produce overall solutions that will ‘automatically’ be valid / good etc. and thus require no evaluation ‘method’ at all to validate it?

A related issue is the one about ‘objective’ measurement, fact, ‘laws’ (akin to natural laws) as opposed to ‘subjective’ opinion. Discussion, felt to consist mainly of the latter, ‘mere opinions’, difficult to measure and thus lacking reliable tools for resolution of disagreement is seen as too unreliable a basis for important decisions.

On a more practical level, there is the matter of ‘decision criteria’ that are assumed to legitimize decisions. Simple tools such as voting ratios — even of votes following the practice of debating the pros and cons of proposed plans: the practice (accepted as eminently ‘democratic’ even by authoritarian regimes as a smokescreen) in reality results in the concerns of significant parts of affected populations (the minority) to be effectively ignored. Is the call for reaching decisions better and more transparently based on the merit of discourse contributions and ‘due consideration’ of all aspects promising but needing different tools? What would they look like?

An understanding of ‘deliberation’ as the process of making overall judgment (of the good, value, acceptability etc.) a function of partial judgment raises questions of ‘aggregation’: how do or should we convert the many partial judgments into overall judgments? How should the many individual judgments of members of a community be‘aggregated’ into overall ‘group’ judgments or indicators of the distribution of individual judgments that can guide the community’s decision on an issue? Here, to, traditional conventions need reconsiderations.

These issues and controversies need to be examined not only individually but also how they relate to one another and how they should guide evaluation procedures in the planning discourse. The diagram shows a number of them and some relationships adding to the complexity, there are probably more that should be added to the list.

Additions, connections, comments?


An effort to clarify the role of deliberative evaluation in the planning and policy-making process
Thorbjoern Mann


The purpose of this effort is drawn from evaluation issues encountered in the planning discourse platform project. What does it take to reach decisions based on the merit of discourse contributions? It first seemed enough to identify the various evaluation tasks we find throughout the planning process, in different disguises, to find common denominators and a coordinated vocabulary between these. Would this mean adopting one common perspective of the role and purpose of evaluation — among the several not universally shared perspectives, or a new one? If the platform aims at openness to all such perspectives, how should this discrepancy be dealt with?

More specifically: if new IT tools are used to facilitate evaluation tasks, is there a danger that these could be to narrowly based on one such perspective, become entrenched and end up excluding other views? It seemed necessary to go back to re-assess the basic concept of evaluation and its role in the planning and policy-making discourse. Whether that re-assessment could resolve the differences between different philosophical positions and lead to common ground, or identify incompatible approaches among which participants must choose in each specific planning situation, — open at this point — would set the direction for the platform design.

Within each perspective this effort will have to consider, there are several issues that should be examined for better solutions. New technology may offer solutions to problems traditionally dealt with by inconsistent means and shortcuts. So a more limited and modest aim of the project is to explore new ideas and possibilities for their constructive application in the planning discourse.

Comments? Suggestions? Wrong question?



Thorbjoern Mann


I have spent some effort on the task of exploring what a better discourse, process, and platform for public planning would look like. A platform for ‘unprecedented’ planning projects, crises, emergencies and policies that affect people in several or many traditional governance units (e.g. countries). Projects that call for more intensive public participation and for which traditional decision-making customs are increasingly inapplicable. One of the insights that have become increasingly clear is that evaluation will play a key role in these processes. Not only for the task of evaluating a proposed plan, for example, to decide whether to reject or adopt it for implementation, but all through the process, in various forms.

The assumption is that public planning discourse should be aiming at decisions that can be seen as based on the quality and merit of contributions to the discourse, rather than mere voting results that give no indication of the extent to which voters were adequately informed, nor of the quality, completeness and reliability of the information they relied on.

There are well-intended techniques for assessing that quality that tried to make better use of new information technology tools, but efforts that left me unconvinced, for various reasons. They led me to the suggestion that the nature and role of evaluation in the planning discourse deserves a more comprehensive review. This should be an effort involving many disciplines. Lacking funding and institutional support for a more comprehensive project, I can currently do no more than to sketch a kind of overview of the various facets of evaluation, a context map in which I can place some of my own ideas for discussion, but perhaps become the draft of an agenda for further work?

I intend to post a series of blog entries showing an ‘agenda map’ of various aspects, partial tasks, and controversial questions, for discussion, starting with a first rough grouping of themes and questions that will be explored in more detail in subsequent posts.

There are serious approaches to planning and problem-solving ‘on the market‘ of new ideas that can be seen as questioning the very need for systematic, thorough evaluation procedures. So it may be necessary to examine my hypothesis — that improved evaluation is necessary — by raising it for discussion. One topic will have to deal with the basic concept, understanding and and rationale for evaluation. A second set of questions will provide an overview of concepts, tools, and other requirements for carrying out evaluation activities, using a few examples of current approaches. This will serve as the source for closer examination of a number of controversial issues and new ideas that arise in the discussion of evaluation and for which more research and better solutions are needed. Finally, an attempt will be made to formulate a collection of preliminary insights from the discussion, and suggestions for what one might actually do ‘today’ in an actual planning or policy-making case, given our current knowledge and tools — of course, also for discussion.