Monthly Archives: April 2012

213 – The environmental planning fallacy

Psychologists studying the way that people plan projects have found that they are often way too optimistic – they think the project will take less time, or cost less, or achieve greater change, or that the change is worth more, relative to what is realistic based on experience with other similar projects. We have observed exactly these tendencies in environmental planning, and seen that they create serious (but often unrecognised) problems for managers wishing to prioritise environmental projects. 

Psychologists Daniel Kahneman and Amos Tversky made very successful careers out of studying the biases that commonly occur in people’s thinking, culminating in Kahneman being awarded the Nobel Prize for Economics in 2002 (Tversky would surely have shared the award if he had still been alive). Kahneman’s recent book for a general audience, ‘Thinking, Fast and Slow’ (Kahneman, 2011), includes very enlightening and entertaining information about the various biases.

Daniel Kahneman

The bias towards optimistic planning they called the ‘planning fallacy’. In his book, Kahneman provided a variety of striking examples. Here’s one that’s not unusual:

‘A 2005 study examined rail projects undertaken worldwide between 1969 and 1998. In more than 90% of the cases, the number of passengers projected to use the system was overestimated. Even though these passenger shortfalls were widely publicized, forecasts did not improve over those 30 years; on average planners overestimated how many people would use the new rail projects by 106%, and the average cost overrun was 45%’ (Kahneman, 2011).

Although we know these things occur, understanding of the reasons why they occur is not so clear-cut. Suggestions include that:

  • planners focus on the most optimistic scenario, rather than using their full experience;
  • simple wishful thinking;
  • biased interpretation of poor past results;
  • underestimating or overlooking the variety of risks that could affect the project.

When people plan environmental projects, exactly the same sorts of things occur. In observing existing environmental plans, or helping people to develop new ones, my collaborators in the INFFER project and I (Pannell et al., 2012) have often observed the phenomenon. It’s so common that we’ve developed our own term for it: the ‘culture of hope’. Unfortunately, overly optimistic claims about projects tend not to get picked up by funding programs when they are evaluating project proposals (e.g., Pannell and Roberts, 2010).

To illustrate the problem, we often observe that people who develop an environmental project frequently seem to be overly positive in their perceptions about the following variables.

  • The value of the environmental assets that the project will protect or enhance
  • The level of cooperation and behaviour change by landholders
  • The various risks that might cause the project to fail (some of which tend to be ignored completely, not just understated)
  • The cost of the project in the short term
  • The longer-term costs, beyond the initial project (also commonly ignored)

With the combined effects of these biases and omissions, it’s common for the assumptions in the plan for an environment project to make it look dramatically better than it really is. I reckon the implied benefit: cost ratio could be exaggerated by a factor of 10 or more in many cases we’ve seen. So the likelihood that decision making will be messed up, with adverse consequences for the environment, is very high.

Although the biases might sometimes result from people trying to game the system to get support for their pet project, that is certainly not the only cause. The psychologists have shown that optimistic biases occur even when people are not trying to distort things. It’s human nature.

From an economist’s perspective, this matters because it is not possible to judge whether an environmental project is worth investing in if the assumptions made about it are inaccurate. From an environmentalist’s perspective, there is a serious risk that funders will end up supporting those projects that have been exaggerated the most, rather than the projects that will really deliver the most valuable environmental outcomes.

Having observed this phenomenon from the earliest days of developing INFFER (before we had even heard of the ‘planning fallacy’), we developed several strategies to try to counter it.

Explicit questions about negative factors that tend to be ignored. When people are completing our Project Assessment Form, we require responses to questions about a range of specific risks, and about long-term funding, so that they can be factored into the project assessment.

Logical consistency checks. We ask them to check whether the answers to some later questions are logically consistent with their answers to specific earlier questions. This helps to flush out some biased responses.

Review of assumptions by independent experts. Kahneman and Tversky found that we are much better at being realistic when judging other people’s projects, rather than our own. Our system includes a facility for independent reviewers to provide comments on particular assumptions, and we use this extensively when supporting people to develop projects.

Feasibility assessment. For large projects, we recommend that a feasibility assessment should be included as the first phase of project funding, with further funding depending on the results of the feasibility assessment. This process should involve collection of additional information about those aspects of the project that were most uncertain in the project-assessment phase, followed by revision of the original assessment.

Even with all of these measures, I don’t think we eliminate the planning fallacy problem, but I do think we reduce its impacts quite a bit.

It’s interesting to observe that most environmental managers are quite oblivious to the problem. Some of them view our measures to try to counter it as unnecessary inconveniences. The measures mean that our process is more involved than the simpler processes many environmental managers are used to, so there is some resistance to using it.

Despite this, our experience shows that it’s crucial to include these measures to counter the planning fallacy. Without them, the project plans developed are of little value to decision makers who genuinely wish to support projects that will deliver the best environmental outcomes with the available resources.

Further reading

Kahneman, D. (2011). Thinking, Fast and Slow, Farrar, Straus and Giroux, New York.

Pannell, D.J. and Roberts, A.M. (2010). The National Action Plan for Salinity and Water Quality: A retrospective assessment, Australian Journal of Agricultural and Resource Economics54(4): 437-456. Journal web site here ♦ IDEAS page for this paper

Pannell, D.J., Roberts, A.M., Park, G., Alexander, J., Curatolo, A. and Marsh, S. (2012). Integrated assessment of public investment in land-use change to protect environmental assets in Australia, Land Use Policy 29(2): 377-387. Journal web site ♦ IDEAS page for this paper

212 – Wildlife corridors: the next big thing?

There seems to be a lot of interest in wildlife corridors as a strategy for nature conservation at the moment. The push is for very large corridors, hundreds of km long in some cases. Reflecting this interest, the Australian Government put together a committee that has developed a ‘Draft National Wildlife Corridors Plan’. Having read the Plan, I can see that the corridors idea has appeal, but I think there is a need for caution when the Government decides how enthusiastically to embrace it.

Why are corridors good? According to the draft Plan, because ‘in order to rebuild connected, functioning landscapes and maximise the benefits provided by the system of protected areas, it is necessary to strategically link protected areas with areas of remnant habitat and ecological value across all lands’. It is claimed that ‘wildlife corridors are one of the most effective conservation tools available for protecting biodiversity and preparing landscapes for climate change.’ Some species might be able to move along corridors as climate changes, so that they can remain in enviroments that suit them. Also, ‘naturally connected landscapes and ecosystems are generally healthier and can store carbon more effectively than degraded landscapes.’

Despite these claims, and other similar ones in the document, I found the case for corridors made in the Plan to be underwhelming. The claimed benefits seem rather rhetorical and non-specific. I would have liked to get clearer and more specific information about the environmental benefits of corridors. It’s about ‘connectivity’, which ‘is a fundamental requirement of healthy, productive landscapes’, apparently, but what, specifically, does connectivity help with? Which types of flora and fauna, or which types of ecosystems, are expected to do better if we invest in corridors? How much better would they do? Are these organisms and ecosystems particularly important, compared to others? If resources are concentrated in a large corridor project, is the whole really better than the sum of the parts? What is the evidence?

The absence of this sort of information from the Plan is liable to make one wonder if the information is actually available.

It appears that ecologists are not uniformly enthusiastic about the corridors idea. Decision Point, the magazine on research about environmental decision making, has presented a range of views on the subject. Hugh Possingham (in DP#26) seems to be an agnostic, suggesting that we don’t yet have the evidence that spatially connected investments deliver a greater outcome than those investments in isolation, but arguing that it’s worth testing.

Hodgeson and several colleagues (in DP#35), summarising an article they published in the Journal of Applied Ecology, caution against making connectivity the top priority of conservation decisions. They argue that uncertainty about the benefits of connectivity is generally higher than uncertainty about the benefits of increasing habitat area and quality. They propose that ‘before investing in connectivity projects, conservation practitioners should analyse the benefits expected to arise from increasing connectivity and compare them with alternative investments, to ensure as much biodiversity conservation and resilience to climate change as possible within their budget.

On the other hand, Carina Wyborn (in DP#58) argues that, even if ‘connectivity provides an uncertain metric for effective conservation’, the corridors idea is worthwhile because it inspires and motivates people to get involved.

Suppose it’s true that a whole corridor is better than the sum of its parts, and that the intuitive appeal of the corridor concept helps to bring more landholders and other stakeholders on side. Are these things sufficient to be able to say that corridors should be a high priority for public investment in the environment?

They help, obviously, but they are not sufficient without a broader assessment of the issues. As argued by Hodgeson and colleagues, investment in a corridor would need to be weighed up against investment in the best alternative approach that could be purchased with the same resources.

The fact that corridors are large, operating at the ‘landscape scale’, is put forward as one of their advantages.

That’s fair enough, but there are also disadvantages from concentrating a lot of conservation eggs into one large basket in this way.

The most important disadvantage, in my judgement, is that by concentrating resources into a particular corridor project, there may be an increased risk of spending a share of the money on changes that provide relatively low environmental benefits. Areas may get resources because they are in the right place to connect to the corridor, rather than because they help address an important and tractable environmental problem.

On the other hand, if we are spending money on many well-chosen smaller projects, we are able to cherry pick projects that will generate the greatest conservation benefits per dollar, and they may or may not be projects that do most to enhance connectivity. (The ‘well-chosen’ is important. It implies a significant effort to analyse project benefits and costs and select the best ones – a greater effort than we commonly see in Australian Government programs.)

The spatial heterogeneity of environmental problems, values and solutions is so high that I am very confident in the benefits of cherry picking. Can we be confident that the benefits arising from connectivity are greater than the benefits of cherry picking? From what I’ve read so far, such confidence seems unlikely to be justified in general. Maybe it is justified in specific cases, but this needs assessment to decide which cases.

Looking at it that way, there is a question that worries me. The benefits of cherry picking are big and pretty obvious, so if the benefits of connectivity were even bigger, they ought to be obvious too. It seems to me that it’s not at all clear that connectivity generates benefits that could be big enough to compete with cherry picking. Does this mean that the benefits of connectivity aren’t all that big?

I am not arguing that corridors are bad. I am arguing that their benefits need to be big enough to outweigh the alternatives. Whether the benefits are big enough is not self-evident, and there are reasons to suspect that they might not be, so careful assessment (and perhaps further research) is needed before we should spend a lot of public money on them.

211 – Water policy book

I am a contributor to a new book on water policy, just published by Edward Elgar. It is called ‘Water Policy Reform: Lessons in Sustainability from the Murray–Darling Basin’, edited by John Quiggin , Thilak Mallawaarachchi , Sarah Chambers.

The book results from a workshop held in Brisbane in October 2010, just after the release of the infamous ‘Guide to the Murray-Darling Basin Plan’, which caused so much unnecessary controversy among rural communities in the Basin. I commented at the time (in PD176) that it was a very good workshop, so I have high expectations for the book.

Included within it are chapters on the history of water policy in the Basin, why the guide failed, what could be done to fix it, predicted economic impacts of the plan, and some of the social issues involved. For details, or to order the book, go to  the publisher’s web page.

My contribution arose from being asked to provide a summary of the key points at the end of the workshop. I decided to do this in verse, with stanzas in limerick form. The editors liked this, and asked to include it at the end of the book. After a bit more work to polish it (thanks to Leona Theis for her help), I provided the final version which has been included. As an ad for the book’s contents, here  it is.

 

The Murray-Darling Basin Plan:
Economic and Community Perspectives

 The river was filling and flowing
The pastures and crops were a-growing
The drought then began
And now comes this Plan
The farmers feel all hope is going

A fight for ideals has exploded
On one side the farmers feel goaded
But others say please
More water for trees
The public debate has eroded

Some say many jobs will be wiped
Economists say, “That’s been hyped
The doomsday is twaddle
Just look at our model”
But no-one believes what we’ve typed

Some folks want the cash for construction
For leaks in the old infrastruction
But we’ve said for ages
“The costs are outrageous
They outweigh the job-loss reduction”

The market’s the way to get water
Just buy it, we think that you oughta
It should be sufficient
To know it’s efficient
The cost will be less than a quarter

If we can contain people’s jitters
We soon will have water for critters
The fight must continue
With courage and sinew
Economists shouldn’t be quitters