Monthly Archives: July 2013

248 – Ranking environmental projects 14: Private costs

Episode 14 in this series on principles to follow when ranking environmental projects. It is about how to account for the costs that are borne by private individuals and businesses in the course of their participation in the project. 

People or businesses who participate in environmental projects may bear costs. For example, people may contribute their time, their land, or other inputs to the project.

The question of whether and how to consider these costs when ranking projects is not as straightforward as for the other costs we’ve discussed.

First consider the case where private costs are contributed voluntarily. Just as I argued for leaving out private benefits (PD244), my advice is to leave out these voluntary private costs.

vic_view2The reason is that, since people are doing it voluntarily, if they actually do it, it must be something they want to do. For them, personally, the private benefits (broadly defined) must be sufficient to outweigh the private costs. In other words, overall, there are no net private costs to include. If there were private net costs from participation (i.e. if private costs exceeded private benefits), then they would not participate, and no private costs would be incurred.

[Just to be clear, in those comments about the balance between private costs and private benefits, I don’t just mean private financial benefits and costs, but also all of the psychological and social benefits and costs that people get from contributing to an environmental project.]

Things are different, however, when the private costs are not borne voluntarily, but are imposed on people against their will, such as through enforcement of a regulation. In that case, it’s quite possible that private costs exceed private benefits and we have “compliance costs” that should be accounted for. My simple advice is to estimate them, discount them if necessary, and add them in to the denominator of the BCR, like this:

pd247e1

where E represents total compliance costs.

The compliance costs you need to estimate are the losses incurred by the relevant people, aggregated over the whole group of people, and aggregated over the relevant time frame (including discounting if the time frame is more than a few years).

Compliance costs may continue on longer than the initial project costs if people are required to continue doing something that they wouldn’t otherwise choose to do. Over what time frame should they be counted? See the next post on maintenance costs.

Compliance costs should be calculated as the change in private net benefits with and without the project. They should include any direct and indirect losses, and any offsetting benefits, if relevant. Estimating compliance costs can be quite challenging unless you have good survey information or an economic model to use, but it’s better to make an educated guess than to leave them out.

The advice to add compliance costs to the denominator is another case of weighing up the benefits of practical simplicity with the costs of an approximation. If you’re not a stickler or an enthusiast for theoretical detail, you might not want to bother with the rest of this post where I present my justification for this simple advice. If you are interested, read on.

As I’ve said before, putting costs in the denominator of the formula used to rank projects is the right approach when there is a limited pool of resources to be allocated amongst projects. The limited pool might be held within the organisation delivering the projects, or in a separate funding organisation.

On the other hand, if supply of a costly input is not in limited supply, then in principle its costs does not belong in the denominator. Rather, the cost should be subtracted from the numerator.

Given that, what should we do with compliance costs? Should they be added to the denominator or subtracted from the numerator? If there is effectively a constraint on the total level of compliance costs the community will bear, it is reasonable to add them to the denominator. (This approximates the effect of optimising for an additional constraint, just as it did for the organisation’s in-kind costs.)

If there is no such constraint, then in theory they should be subtracted from the numerator. But what if the numerator is not measured in dollars? I suppose you could apply a weight to the compliance costs (with another subjectively determined weight) and subtract the weighted cost, but I feel that’s likely to be even more of an approximation than adding them to the denominator, so I’d do the latter.

That leaves us with compliance costs being subtracted from the numerator only in the case where benefits are measured in dollars and there is no constraint on the level of compliance costs. However, if you’re dealing with a variety of different projects, including compliance costs differently in different cases might be more confusing than it’s worth. If we were to simplify the system by always adding compliance costs to the denominator, the error would usually be small, in my judgement. That’s why this is my “simple advice” on how to include compliance costs: add them to the denominator.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS

247 – Ranking environmental projects 13: In-kind costs

Episode 13 in this series on principles to follow when ranking environmental projects. It is about how to account for the in-kind costs that are contributed by the organisation that is responsible for delivering the project. 

If the cash funds provided for a project are not sufficient, it’s common for the organisation that’s running the project to make up the difference. For example, they may cover some or all of the costs of: project staff salaries, administrative support, office space, stationary, telephone calls, and so on.

These ‘in-kind’ costs provided by the organisation responsible for the project are real costs and should be accounted for when projects are being ranked.

vic_view_with_carGenerally, organisations have a limited level of resources that they can use to provide in-kind support to projects. From a theoretical perspective, this creates a slight problem, because there are now two limited pools from which different types of costs (cash and in-kind) are being drawn.

Strictly speaking, this means that you cannot rank projects based on a single formula. The BCR only ranks projects correctly if there is only one constraint on costs (e.g. on the level of funds in the pool of new cash). If there are two constraints, it is impossible to specify a formula that is guaranteed to rank projects correctly. In theory, what one would have to do is build a constrained optimisation model with separate constraints for the two types of funding (e.g. using the technique of mathematical programming).

In practice, that is almost never going to happen, because of the additional time and expertise required, and the loss of transparency and intuitiveness in the results. Fortunately, treating all costs as if they come from one large pool (i.e. including them all in the denominator) is usually a very good approximation of the theoretically correct approach. (The loss of environmental benefits was less than 1% in the examples I looked at.) So that’s the approach I recommend – use …

pd245e2

where K is in-kind costs provided by the organisation.

As was the case with cash costs, if in-kind costs are borne for more than a few years, then they should be discounted before adding them up to calculate K (PD242).

There is one combination of circumstances when it would be OK to ignore in-kind costs. This is when (a) in-kind costs are close to being the same proportion of cash costs for every project, and (b) you only care about the ranking of projects, not in assessing whether their overall benefits exceed their overall costs. Situation (b) is reasonably common, but I don’t think you would often find situation (a), especially if projects from different organisations are to be compared when funding decisions are made.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS

 

246 – Ranking environmental projects 12: Cash costs

Episode 12 in this series on principles to follow when ranking environmental projects. It is about how to account for the cash costs of projects. 

Remarkably, some systems for prioritising environmental projects only look at the benefits and ignore the costs. In certain special cases, this can be OK, but in most cases it’s a serious mistake. Ironically, although it might feel like focusing on the environmental benefits should be good for the environment, neglecting costs results in a (usually large) reduction in the total environmental benefits generated from a given pool of funding.

Even where costs are included in the ranking process, people tend to think that the cost aspect is simple and straightforward. In fact there are some tricky aspects to be aware of.

There are several different types of costs related to environmental projects that may need to be considered when ranking projects. They are: the cost of the project itself (cash costs to the funder, in-kind costs to the lead organisation and private costs to participants), ongoing costs to maintain the benefits generated by the project, and the opportunity cost of the funds invested in the projects. The last one is different in nature from the others, and we have already dealt with it in PD242 (on time lags and discounting). This time I’ll cover project cash costs.

For an environmental project to proceed, money is needed for various purposes, potentially including: the time of people employed to implement the project or provide support to the project, cars, fuel, machinery and equipment, payments to people to encourage behaviour change, legal costs, office space, telephones, insurance, publicity, printing, and so on.

vic_view

Whether an organisation is allocating its own cash resources amongst projects or applying to an external funder, the cash allocated to a project may not be enough to pay for all associated costs. Often the lead organisation bears some of the costs out of existing salary or operating budgets (in-kind costs), or people in the community voluntarily contribute time or other resources (private costs). I’ll cover these non-cash costs in the next two posts.

Accounting for new cash allocated to a project is straightforward. It counts as a cost, and should be included in the denominator of the Benefit: Cost Ratio.

pd245e1

where C is total project cash costs.

In theory, to calculate C, costs in future years should be discounted to their present values (see PD242) before adding them up. How important it is to do this discounting depends on the number of years over which the funding will be rolled out. If it is only a short project (e.g., three years), the error in total costs if you ignore discounting of costs is only a few percent, so it doesn’t matter much. Leaving cash costs undiscounted might even help to offset the common tendency for excessive optimism about projects (PD213).

On the other hand, if the project funding is longer in duration, discounting probably matters. For example, if the project involves constant funding over 10 years, failing to discount would exaggerate total costs by around 25% (assuming a 5% discount rate). For some projects, this would affect whether benefits exceed costs (which can only be seen if benefits are being measured in dollar terms).

Discounting costs matters most if some of the projects being considered involve funding over much longer time frames than others, because in those cases it can affect the ranking of projects.

Cash costs unambiguously go in the denominator, because the level of cash available for funding environmental actions within a program is always limited. Whenever projects require a resource that is limited in availability, the right ranking is calculated by dividing project benefits by the quantity of that resource required by each project.

I’ve seen a ranking metric in actual use that put a weight on cash costs and subtracted them from the estimate of project benefits. This is slightly better than ignoring costs entirely, but only slightly. There’s no reason not to get this right – it’s trivially easy.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS

245 – Ranking environmental projects 11: Scoring variables

Episode 11 in this series on principles to follow when ranking environmental projects. It covers a few issues related to the scoring of variables – taking information about a variable and converting it into a number that can be put into the equation for benefits.  

Let’s return to the benefits part of the equation for ranking environmental projects (the simple version with only one benefit).

Expected benefit = [V(P1) – V(P0)] × A × (1 – R) / (1+r)L

or

Expected benefit = V(P’) × W × A × (1 – R) / (1+r)L

where V is the value of the asset (in whatever system of measurement makes sense), W is the effectiveness of works, A is adoption, R is risk and L is the time lag in years. (For the purpose of this discussion, ignore r. It doesn’t vary between projects. See PD242.)

There are two distinct groups of variables in these equations. There are two variables that can take any value greater than zero: V and L. And there are three that can take any value between zero and one: W, A and R.

All of them are “continuous” variables – they change smoothly and can take any value within their feasible ranges. If you had the information, you would plug their exact values into the equation.

However, you never have exact information. A common approach in systems that collect information for ranking projects is to present a discrete number of options for the value of the variable, and ask participants to select the value that seems to be nearest to the correct value. For example, here is a question of this type about technical risk.

What is the probability that the benefits generated by the project would fall well short of expectations due to technical factors? (Rt)

    • 0-5% Very low risk of project failure due to poor technical feasibility. (Rt = 0.03)
    • 6-10% (Rt = 0.08)
    • 11-15% (Rt = 0.13)
    • 16-20% (Rt = 0.18)
    • 21-100% High risk of long-term project failure due to poor technical feasibility. (Rt = 0.60)

I don’t have a problem with this approach, as long as the response options are chosen thoughtfully. The quality of information available is usually not so high that this sort of approximation causes any significant reduction in the quality of the resulting rankings.

In the above example, I haven’t spaced out the response options for Rt equally between zero and one, because I judged that most projects have values between zero and 20%. I have indicated the mid-point of the range for each response option, and that is what I would plug into Rt in the benefits equation.

canadian_lakeSometimes people convert the responses from this type of question into a number from an ad hoc scoring system, rather than using a scale that is more natural for the variable. For example, in the above case they might assign a score of 1 to the first response, 2 to the second response, and so on (instead of probabilities of 0.03, 0.08 and so on). This can potentially be OK, but there are a few traps to avoid.

Firstly, in a case like the one above where the response options are not equally spaced, the scores assigned should not be equally spaced either. They should be spaced out consistent with the values in the response options.

Secondly, if one of the response options represents zero, the score assigned to that option should be zero. For example, if a response option for adoption is zero adoption, it should get a score of zero so that when it is multiplied into the equation, the overall score is zero. (Obviously, if there is zero adoption of the actions being promoted by the project, there would be no environmental benefits attributable to the project.) In that case, using scores of 1, 2, 3, 4 or 5 is no good. If you must use scores instead of probabilities, use 0, 1, 2, 3 or 4 (assuming that the first response option is zero adoption).

Thirdly, even if you are using an ad hoc scoring system, you still have to multiply the variables, as shown in the above equations. Weighting them and adding them up, as done in many systems, produces much inferior rankings.

If benefits aren’t being measured in dollars, then a scoring system can work, as long as it satisfies the above requirements. However, my advice is to use the correct ranges when scoring each variable (i.e. between zero and one for W, A and R) rather than some ad hoc system. Why not do that? It is no more difficult, it makes it easy to meet all the above requirements, and it makes the meaning of each variable clearer.

If the benefits are being measured in dollars, then you don’t have an option. You have to assign values that correspond to the meanings of the variables rather than using an ad hoc scoring system. Otherwise you lose the benefit of being able to assess whether the benefits exceed the costs.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS

244 – Ranking environmental projects 10: Private benefits

Episode 10 in this series on principles to follow when ranking environmental projects. It is about how to account for private benefits when assessing the overall benefits from an environmental project.

Suppose that an environmental project in a government program requires some private citizens to change their behaviour or business management. An example would be a project that aims to encourage farmers to convert crop land into native forest. As well as generating environmental benefits and public costs (the cost of running the project), this project may generate benefits and costs to the private citizens whose behaviour would need to change. I’ll call these “private” benefits and costs.

Environmental projects are generally not set up with the aim of generating private benefits, but some do so as a spinoff from the real aim of generating environmental benefits for the broader public. How should the private benefits be accounted for?

The inclusion of private benefits is somewhat tricky and, in my observation, often poorly understood.

There are three reasons why the analyst might need to pay attention to private benefits and costs.

(a) The level of private benefits influences the behaviour of the private citizens who the project aims to influence.

(b) The level of private benefits should influence the choice of mechanisms used to try to encourage behaviour change.

(c) It may be appropriate to account for private benefits as one of the benefits of the project, which may affect the ranking of projects.

(a) is pretty obvious, but surprisingly it often gets overlooked. When judging the level of adoption/compliance that is likely (PD240), it’s important to think about the practices being promoted and consider how attractive they are to the people they are being promoted to. If the private benefits of the new practice are much greater than the private costs, adoption will be high, and vice versa. It’s not sufficient to consider the private benefits alone – you have to think about how they stack up against the private costs.

I should clarify that I don’t just mean private financial benefits and costs here. Non-financial benefits and costs should be considered as well, potentially including factors such as riskiness, convenience, complexity, compatibility with existing practices, social pressures, and environmental outcomes that matter to the private person whose behaviour is potentially changing.

tullaroop_reservoir(b) is less obvious, perhaps, but also important. Agri-environmental programs in Australia have tended to over-rely on extension (information provision, awareness raising, etc.) to promote practices for which the private benefits are too low to outweigh the private costs. For those sorts of practices, something stronger, such as payments or regulation, would be needed to generate significant adoption, but of course these are more expensive approaches. For a project where private costs exceed private benefits, we face a trade-off that should be accounted for when the project is being evaluated. Extension is cheaper but generates little adoption, while payments or regulation generate more adoption but are more expensive.

In Europe, there is a tendency to use payments to farmers as the only response, even though extension might be sufficient (and much cheaper) in some cases. My Public: Private Benefits Framework is designed to help people think through these issues (Pannell, 2008).

(c) comes into play in some situations but not others. Including private benefits as a benefit of an environmental project is a can of worms, and they are unlikely to big in most cases anyway, so the brief, pragmatic version of my advice is, leave them out, unless one of the objectives of the program is to generate private benefits. The next few paragraphs explain why I say that.

One reason you might want to leave them out is that they are not the main reason for doing the project. You might consider that the focus should be on maximising environmental benefits and that chasing private benefits when selecting projects might compromise that. Given that your organisation is likely to have a certain focus or set of responsibilities related to the environment, perhaps this is reasonable (provided that you don’t also neglect private benefits when thinking about behaviour or mechanism choice).

However, economists generally prefer to consider all benefits and costs to the community as a whole. Most would consider that, if there are private benefits in excess of private costs resulting from a project, the net private benefits (private benefits minus private costs) should be counted as a benefit of the project.

But this is not as simple as it sounds. If the private benefits do exceed the private costs by a significant amount, one would expect the practice to be adopted by people anyway, even without the practice being promoted by the environmental program. In that case, there are no private benefits generated by the project. Indeed there would be no environmental benefits attributable to the project either, because the benefits would have occurred even without the project (the observed benefits are not “additional”). This gets back to being clear about what would happen with and without the project (PD237).

An exception to this could be if the project makes the adoption occur sooner, by raising awareness of the new practice. The people who adopt sooner would benefit by getting the private benefits sooner than they would have (making them less affected by discounting – PD242). While that is worth something, research I was involved in in agriculture suggests that the difference in adoption timing is likely to be only a year or two, in which case the additional private benefit would be quite small.

Some environmental programs provide payments to people to encourage them to change their behaviour. If the private costs of a new practice outweigh the private benefits, these payments can help by offsetting losses that would have stopped participation. Should these payments be considered a private benefit and included in the project benefits formula? No, for the following reasons.

If the payments are pitched at just the right level to offset the private losses, then they exactly cancel out. There is no private net benefit to include. If the payments are bigger than they need to be, such that the recipients make a net benefit even after allowing for their private costs of participating, then in principle that net benefit could be included in the formula, but the excess payment will also have to be included as a cost to the project (we’ll get to costs in PD245), in a sense cancelling out the benefit. This additional cost is unnecessary and will reduce the funding available for other projects, so rather than worrying about accounting for these private benefits, the best response is to get rid of any private net benefits by making sure that any private payments are not excessive.

So, as I said earlier, the inclusion of private benefits as one of the benefits from an environmental project is complex, and the net private benefits are likely to be small – close to zero in many cases. Unless they are considered particularly important for some reason, or are one of the objectives of the program, it is probably reasonable to round them off to zero and leave them out of the analysis.

That’s as much as I want to say about which variables should be included in the benefits part of the equation. I haven’t discussed factors like whether the project fits within the program scope. I recommend applying that sort of factor as a filter on which projects get considered, rather than as a variable within the benefits formula.

Further reading

Pannell, D.J., Roberts, A.M., Park, G. and Alexander, J. (2013). Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects, Wildlife Research (forthcoming). Journal web page ♦ Pre-publication version at IDEAS