237 – Ranking environmental projects 3: With vs without
Episode 3 in this series on principles to follow when ranking environmental projects. This one discusses the “with versus without” principle for estimating the project benefits.
Through the series, we will cover a number of points about the estimation of benefits from an environmental project. Initially, to keep things simple, I’ll talk about the case where there is a single type of benefit being generated by an environmental project (e.g. a threatened species is being made safer). In later posts I’ll talk about cases with multiple types of benefits from the same project.
This first point is deceptively simple. It is that the benefit of an environmental project is the change in value of the environmental asset as a result of the project. In other words, it is a difference: the difference between the environmental value with the project and without the project.
So, to estimate the benefits of a project, you need two pieces of information: the environmental values with the project and the values without the project. Usually, when we are evaluating a project, the project has not yet been implemented. In that case, both of the required pieces of information have to be predicted. You can’t observe them, because they are in the future.
Note that comparing environmental values “with versus without” the project is not the same as comparing values “before versus after” the project. The reason is that the condition of the asset would probably not be static in the absence of the project. For example, it may be that the asset would degrade in the absence of the project, but its condition would be improved by the project (relative to its current condition). This is illustrated in Figure 1.
The graph illustrates a case where the asset currently has a value of 57 [labelled (1)]. (The 57 is just some measure of value – we’ll discuss values more in later posts.) Without the proposed project, the value is expected to decline steadily, to a score of 37 after 25 years [labelled (3)]. With the project, value would increase to a score of 76 after 25 years [labelled (2)].
Clearly, in this example, the benefits of the project grow over time (the two lines diverge in Figure 1). Ideally, we would estimate the benefits in each year after the project is implemented and add them up (after allowing for discounting, which we’ll cover in a later post). A practical simplification is to estimate the environmental benefits based on the difference in the asset value with and without the project in a particular future year. For example, we might choose to focus on 25 years in the future, and estimate values at that date with and without the project. In doing this, we need to be careful that we deal appropriately with time (see a later post for details).
Assuming we go with that simplified approach (focusing on benefits at year 25), the relevant measure of project benefits for ranking projects is (2) minus (3). I have seen ranking systems which use (1) minus (3), (2) minus (1), (1) alone or (2) alone, and sometimes more than one of these in the same ranking system, but they are all irrelevant. If you include (2) minus (3) you should not include any of the others listed. To do so will just make the rankings worse.
Because of the “with versus without” principle, a project can generate benefits even if it does not completely prevent degradation of the environmental asset. As long as it slows or reduces degradation, this should be measured as a benefit. Figure 2 shows an illustration of this. In this example, future asset condition with the project (2) is below the initial asset condition (1), but is above future asset condition without the project (3). Since the project benefit is (2) minus (3), the benefit is positive.
On the other hand, a project that superficially appears to generate large benefits may actually not do so, because those benefits would have been generated even without the project. In other words, the benefits are not ‘additional’ to what would have happened anyway. The without-project line in the graph would be almost as high as the with-project line, so the difference between them (= the benefit of the project) would be minimal (Figure 3).
For example, suppose that a proposed project encourages farmers to adopt a new type of environmentally beneficial crop, where that crop is much more profitable than farmers’ existing crops. If the private benefits are large enough, it’s a safe bet that the farmers would have adopted the new crop even without the project. It would have been promoted by word of mouth and by private farm business consultants. Adoption of the crop for commercial reasons would have generated environmental benefits as a spin-off.
Making good predictions about the “without project” scenario can be quite difficult, requiring good knowledge of the environment, the relevant management practices and the people whose behaviour matters. Weak thinking about the “without” scenario for environmental projects is a common failing, sometimes leading to exaggerated estimates of the benefits.
Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research 40(2), 126-133. Journal web page ♦ Pre-publication version at IDEAS
2 thoughts on “237 – Ranking environmental projects 3: With vs without”
I like the way you explain the concept step by step and very clearly.
It may be worth mentioning that this issue occurs in many disciplines where eveluation of intervention(s) is necessary. Readers of your work may need to know that in some disciplines doing the evlaution requires knowing the “counterfactual(s)” [i.e., the without-project scenario].
Viewed: 27/02/2013, Article dated: August 29, 2012 at 6:28 pm, Author: Austin Frakt
The only time I had lunch with Jon Gruber (Professor of Health Economics at MIT USA) he told me the biggest problem in policy debates is a lack of clear understanding of the counterfactual. Since then, I’ve been paying more attention, and he’s right that it is not well understood. Since I used the concept repeatedly in the comments on a post earlier today, I thought I should probably explain what I (and Gruber) mean. …….
The most important point is that almost nobody is explicit about this in policy debates, even when the policy is crucially important. Will health reform cause employers to drop coverage or not? Well, we only have one world. The counterfactual needs to be constructed. It can’t simply be assumed to be the pre-reform world, because employers have been dropping coverage for years. There’s a trend. Other things may change (the economy, the nature of the labor market, etc.), so we’d want to control for those. And so forth.
This is worth thinking about. Next time you’re involved in a policy debate, ask your opponent, what (s)he is taking as the counterfactual? If (s)he doesn’t even know what you’re talking about, you’ve already won, even if (s)he won’t admit it.
In PD242 there is an interesting comment by Peter Vesk, and a reply by me, that relates to this post.