Episode 3 in this series on principles to follow when ranking environmental projects. This one discusses the “with versus without” principle for estimating the project benefits. 

Through the series, we will cover a number of points about the estimation of benefits from an environmental project. Initially, to keep things simple, I’ll talk about the case where there is a single type of benefit being generated by an environmental project (e.g. a threatened species is being made safer). In later posts I’ll talk about cases with multiple types of benefits from the same project.

This first point is deceptively simple. It is that the benefit of an environmental project is the change in value of the environmental asset as a result of the project. In other words, it is a difference: the difference between the environmental value with the project and without the project.

So, to estimate the benefits of a project, you need two pieces of information: the environmental values with the project and the values without the project. Usually, when we are evaluating a project, the project has not yet been implemented. In that case, both of the required pieces of information have to be predicted. You can’t observe them, because they are in the future.

Note that comparing environmental values “with versus without” the project is not the same as comparing values “before versus after” the project. The reason is that the condition of the asset would probably not be static in the absence of the project. For example, it may be that the asset would degrade in the absence of the project, but its condition would be improved by the project (relative to its current condition). This is illustrated in Figure 1.

The graph illustrates a case where the asset currently has a value of 57 [labelled (1)]. (The 57 is just some measure of value – we’ll discuss values more in later posts.) Without the proposed project, the value is expected to decline steadily, to a score of 37 after 25 years [labelled (3)]. With the project, value would increase to a score of 76 after 25 years [labelled (2)].

Figure 1.

Clearly, in this example, the benefits of the project grow over time (the two lines diverge in Figure 1). Ideally, we would estimate the benefits in each year after the project is implemented and add them up (after allowing for discounting, which we’ll cover in a later post). A practical simplification is to estimate the environmental benefits based on the difference in the asset value with and without the project in a particular future year. For example, we might choose to focus on 25 years in the future, and estimate values at that date with and without the project. In doing this, we need to be careful that we deal appropriately with time (see a later post for details).

Assuming we go with that simplified approach (focusing on benefits at year 25), the relevant measure of project benefits for ranking projects is (2) minus (3). I have seen ranking systems which use (1) minus (3), (2) minus (1), (1) alone or (2) alone, and sometimes more than one of these in the same ranking system, but they are all irrelevant. If you include (2) minus (3) you should not include any of the others listed. To do so will just make the rankings worse.

Because of the “with versus without” principle, a project can generate benefits even if it does not completely prevent degradation of the environmental asset. As long as it slows or reduces degradation, this should be measured as a benefit. Figure 2 shows an illustration of this. In this example, future asset condition with the project (2) is below the initial asset condition (1), but is above future asset condition without the project (3). Since the project benefit is (2) minus (3), the benefit is positive.

Figure 2.

On the other hand, a project that superficially appears to generate large benefits may actually not do so, because those benefits would have been generated even without the project. In other words, the benefits are not ‘additional’ to what would have happened anyway. The without-project line in the graph would be almost as high as the with-project line, so the difference between them (= the benefit of the project) would be minimal (Figure 3).

Figure 3.

For example, suppose that a proposed project encourages farmers to adopt a new type of environmentally beneficial crop, where that crop is much more profitable than farmers’ existing crops. If the private benefits are large enough, it’s a safe bet that the farmers would have adopted the new crop even without the project. It would have been promoted by word of mouth and by private farm business consultants. Adoption of the crop for commercial reasons would have generated environmental benefits as a spin-off.

Making good predictions about the “without project” scenario can be quite difficult, requiring good knowledge of the environment, the relevant management practices and the people whose behaviour matters. Weak thinking about the “without” scenario for environmental projects is a common failing, sometimes leading to exaggerated estimates of the benefits.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS

Episode 2 in this series on principles to follow when ranking environmental projects. This one discusses one aspect of the metric used to rank projects: how to include costs.

Suppose you manage an environmental program that has a budget available for spending on environmental projects and there is not enough money to fund every proposed project. You have to decide which projects to fund. How should you do it?

The first principle is that projects should be ranked using a metric (a formula) that consists of a measure of project benefits divided by a measure of project costs. Economists call this metric a Benefit: Cost Ratio (BCR).

There are plenty of project ranking metrics out there in actual use that don’t do this. Some subtract costs instead of dividing them, and some (remarkably) ignore costs entirely. These are mistakes that are costly to the environment.

To illustrate, consider the following three hypothetical projects, with the indicated benefits (B) and costs (C). Because the budget is limited, the first project we should choose is the one with the highest benefits per unit cost (the highest BCR) = project 1. But if we rank according to B – C the top ranked project seems to be project 2, while ranking according to B (ignoring costs) tells us that project 3 is best.

The loss of environmental values from using the wrong metric (i.e., ranking according to B – C or B) depends on how tight the budget is. Assuming that the budget is enough to fund 10% of projects, the loss of environmental benefits is 12% for B-C and 19% for B (based on simulating 1000 funding rounds with 100 potential projects in each).

In other words, fixing up the formula is like increasing the program budget by 14% or 23%. It’s much easier to fix the formula than to increase the budget!

In the examples above, I’ve assumed that we know what the benefits and costs would be for each project. Later posts in this series will deal in detail with how we should estimate the benefits and costs. For now I’ll just make these two observations.

The benefits used in the ranking metric should be the benefits of the proposed intervention or project, not the total benefits of the environmental asset. What difference can be made by the intervention or project, and how important is that difference?

The costs should also represent the costs of the intervention or project. If this project did not go ahead, what level of resources could be diverted to other uses?

p.s. (9 May 2013). A slightly more technical issue: it is sometimes claimed that BCRs are flawed because they can be manipulated by transferring costs from the denominator to the numerator. For example, suppose that a proposed project has benefits of $10m, program costs of $2m (requested from the funding program) and other costs of $1m (from other sources, such as the private sector). We could potentially calculate the BCR as 10/(2+1) = 3.3, or else as (10-1)/2 = 4.5. However, there is no ambiguity about the correct way to do this: what should go into the denominator are the costs that are in limited supply from the perspective of the decision makers in the funding program. They are trying to choose the projects that generate the most net benefits per dollar that they have to allocate. So the correct procedure is to subtract the other costs from the benefits, meaning that the correct BCR for this project would be 4.5.

Things get a bit tricky, however, if projects also require ongoing maintenance funding beyond the current project, and the budget for maintenance funding is expected to be fully allocated. This is realistic for many (probably most) projects. In this case, there are actually two constraints that must be satisfied: the current program budget and the long-term maintenance budget. Strictly, in this situation, projects cannot be ranked using a single formula as a metric. The program would need a mathematical programming model to select which projects deliver the most benefits while satisfying both constraints. In practice, after testing various approaches, I believe that a reasonable approximation is to add up both costs (short-term program costs and long-term maintenance costs) and include the total as the denominator in the single formula.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS