Research and practical experience with agri-environment programs around the world provides many lessons on what leads to success or failure. New programs are often designed without sufficient awareness of these lessons, resulting in lost opportunities to achieve more valuable outcomes. A workshop to identify these lessons was held a couple of years ago, and the results have just been published by ANU Press as a book.
The book is available for sale or free download here. I’ve got a couple of chapters in the book, including one where I try to identify “best-practice in design and implementation” of agri-environmental policies. It is based on my experience over the past 15 years or so, working with federal and state agencies and regional bodies in Australia, and some similar organisations in several other countries. Here’s a longish extract. This is a longer-than-usual PD as I wanted to keep this material together in one post.
Design of programs/institutions
Additionality: Agri-environmental programs should aim to avoid paying farmers for undertaking actions that they would have done in any case. In other words, managers need to evaluate whether the benefits generated by a program investment are ‘additional’.
Continuation after investment ends: Where a program is intended to provide only temporary support to farmers (eg, in all Australian programs, but not in European programs), it is important to ensure that the actions being supported are attractive enough that farmers will continue to undertake them once funding ends. Otherwise the investment has no enduring benefit. These first two principles combine to mean that, where support will be temporary, perhaps the only defensible role for agri-environmental payments is to encourage farmers to get experience in a new practice that they are likely to be keen to continue once funding ends. The practice might be something new of which farmers are currently unaware, or one that becomes more attractive to farmers with experience.
Appropriate institutional delivery: In some agri-environmental programs, responsibility for overseeing some or all on-ground delivery of projects is devolved to regional organisations. This has been the case in all of Australia’s major programs since the late 1990s. In these cases, the program needs to be designed in a way that provides incentives for these regional organisations to respond appropriately. In particular, they should be incentivised to pursue sustained improvements in natural resource outcomes, rather than to support project activities without considering their resulting outcomes. There should be an emphasis on spending program resources well, rather than rapidly. Unfortunately, some of Australia’s major programs have generated incentives that go directly against these recommendations. Short time frames for programs and rules that funding will be withdrawn if not spent rapidly enough increase the difficulty of meeting this best-practice requirement. (Chapter 5 on environmental NGOs discusses how these organisations can help here.)
Balancing small, moderate and large projects: In programs where the availability of funding is small relative to the amount needed to fund all attractive projects (ie, in all Australian programs), there is often a temptation to share the available resources amongst a large number of small projects. Sometimes this results in good leverage of program resources, but often it means that almost all projects have inadequate resources and are unable to achieve worthwhile outcomes. This advice sometimes clashes with political preferences to support many projects rather than few. A compromise strategy could be to use a portion of funding (eg, 25 percent) to support many small projects to satisfy political needs, and use the remaining 75 percent to support larger projects that are more likely to be effective. On the other hand, achieving the most ambitious environmental targets is often disproportionately expensive, with costs increasing dramatically as targets become more ambitious. To maximise outcomes, it may be best to pursue a moderate number of moderate sized projects, rather than many small or few large projects.
Sufficient time for planning: Program performance is often hampered by a tendency for agencies to delay planning new programs until the end of a previous program is imminent or past. Good planning and design of programs and prioritisation of investments requires more time than is usually allowed. Ideally, organisations should commence planning and analysis to develop the next program years before the end of the current program. Even though the scope and parameters of the next program cannot be known in advance, these can be predicted, and sometimes influenced, by the agency to some extent. Having already-analysed investment options ready to put forward can be highly persuasive, and increases the likely environmental benefits generated.
Investment longevity: Finally, funding for agri-environmental programs in Australia tends to be temporary and short-term – typically five years. Environmental problems usually take much longer than this to resolve, so systems for providing long-term funding should be used where possible. If it is not possible to ensure long-term funding, then this should have a strong influence on which projects are selected for funding. In particular, projects that would require significant funding in the long-term to maintain the benefits generated by an initial project should not be supported. For example, most projects for control of feral animals or plants would fall into this category because feral animals reinvade once control ends. Similarly, cases where farmers are likely to disadopt practices once funding ends should be excluded. Typically, programs are much too optimistic about ongoing adoption of practices post-funding.
Design of projects/investments
SMART targets: A number of agri-environmental programs have been criticised for failing to establish appropriate targets (eg, European Court of Auditors, 2011; Auditor General, 2008; Park et al, 2013). Specifically, targets should be SMART (Specific, Measurable, Achievable, Relevant and Time-Bound) in order to facilitate monitoring and evaluation of a program, and to ensure that the funded investments are focused onto suitable activities (see Chapter 4 on setting SMART targets).
Project logic: Many projects funded in agri-environmental programs are not designed in a logically consistent way. They are consistent with a ‘project logic’ but only in a qualitative sense. They fail when assessed against quantitative questions, such as ‘are the funded activities sufficient to achieve the intended land-use changes?’ or ‘are the intended land-use changes sufficient to achieve the desired natural resource outcomes?’ A good project logic is more than a description or diagram of connections between elements of the system being managed or influenced; it quantifies the connections and makes assumptions transparent.
Selecting the right policy tool: There is a tendency for little thought or analysis to be put into the selection of policy mechanisms to be used in a project or program, resulting in inappropriate choices in many cases. In Australia, there is too much reliance on extension in situations where it cannot deliver the desired outcomes. For example, Australia’s national salinity program between 2001 and 2007 relied mainly on extension to encourage farmers to change their practices, but the practices being promoted were not attractive to farmers on the required scale and so were adopted to a very limited extent – much too limited to achieve the program’s goals (Pannell and Roberts 2010). On the other hand, in Europe and the United States, financial payments are almost the sole mechanisms used, often funding activities that are not additional. The framework of Pannell (2008) helps organisations to evaluate the type of mechanism that is most suitable for a particular project. (See also Chapter 18 on the choice of tools depending on public benefits and private benefits arising from an investment.) Sometimes programs specify which policy mechanisms will be used by projects prior to identification of the projects, and then project investments are selected without considering whether they are suitable for the pre-determined policy mechanism. Preferably, policy mechanisms should be selected to match the type of projects that will be necessary to achieve the desired program outcomes. They should be project-specific, to some extent. As noted in Chapter 18, Australian programs tend to rely too much on extension and too little on the development of technology.
Prioritisation: Where funding is limited, prioritisation of investment options is essential. The quality of the prioritisation process can make a major difference to the natural resource outcomes delivered (eg, Barry et al, 2014).
Rank projects: Programs should prioritise projects, not problems, regions or issues. Some programs prioritise regions or issues without defining projects, but this means that it is not possible to properly consider issues of project cost, project risks, project benefits, or time lags, all of which should be factored into the prioritisation process.
Rank according to value for money: Projects should be ranked according to their value for money: a measure of their benefit divided by their cost (see Chapter 15 on designing cost-effective agri-environment schemes). Failure to do this is one of the most serious mistakes that can be made when ranking projects, but unfortunately it is common. Some systems fail to consider costs entirely, some do include costs but fail to divide by them, and many include only some of the costs that should be considered. For example, it is important to factor in long-term maintenance costs, since they vary so much between different projects, but few Australian systems do so. If maintenance costs are needed but are not expected to be provided, then project benefits should be scaled down accordingly in the ranking process.
Measure the gain against a counterfactual: When ranking projects, benefits should be estimated from the predicted difference in natural-resource outcomes with versus without the proposed investment (Chapters 19 on counterfactuals). Although this seems like common sense, Maron et al, (2013) found that 15 out of 16 systems in actual use for ranking biodiversity projects failed to do this correctly.
Incorporate all the benefits and risks: There are many factors that could be considered when estimating the benefits of a proposed project. The essentials are: the potential values generated, the likely level of adoption/compliance with the project by landholders (Pannell et al., 2006; Chapters 12 and 13) various risks that might result in project failure (technical, social, financial and managerial risks) and time lags until benefits are generated.
Use a sound metric: A commonly neglected issue is how to combine the variables that determine benefits and costs into a metric for ranking projects. Most metrics in actual use are theoretically unsound and provide poor rankings, even where project information is accurate. Indeed, as is discussed in chapter 17, the level of benefit delivered is more sensitive to the quality of the metric than the quality of the information fed into the metric. Potential benefits from investment are very sensitive to the use of inferior ranking metrics. Chapter 17 on metrics and Pannell (2015) outline the requirements for a sound ranking metric.
When decisions about project funding are being made, uncertainty about those projects is usually high. Common areas of major uncertainty include the technical feasibility or effectiveness of the proposed actions to be funded by the project, and the valuation of those environmental benefits that are generated. Uncertainty should be accounted for in several ways.
Identify key uncertainties: Project proponents should be required to identify key uncertainties, and to specify what will be done in the project to reduce them.
Carry out feasibility assessment and pilot studies: Projects above a certain scale should be subject to rigorous feasibility assessment before longer term funding is committed. Funding to support information collection, perhaps in a pilot study, should be provided for 6 to 12 months, after which longer-term funding should be conditional on the results obtained.
Learn from early experience: Projects and programs should be managed in an adaptive way, with information collected during early stages of the program or project being used to inform changes in management or even cessation in some cases. In practice, few programs operate with that degree of flexibility, resulting in continuation of poorly designed investments past the time when their faults are apparent.
Managing people’s biases, preconceptions, self-interest
Acknowledge the values that people bring with them: Being human, the people involved with agri-environment programs are subject to biases, preconceptions and self-interest (consider the way restoration vs conservation is valued in different ways in different places Chapter 10), all of which can reduce program performance. If managers are aware of these human traits, they can introduce systems to limit their negative impacts.
Optimism vs realism: A pervasive problem is the tendency for people to be overly optimistic about proposed projects. It is common to see proposals in which the benefits are exaggerated, and the costs, risks and time lags are under-estimated. Several factors contribute to this, including vested interests, wishful thinking, and a failure to recognise all relevant difficulties and risks that are likely to affect a project. The ideal strategy to overcome this problem is serious independent expert review of project proposals, but this is only justifiable where projects are sufficiently large. This is another factor that tends to favour moderately large projects over small.
Self blindness: People involved in allocating program funds commonly perceive that their existing prioritisation process is of high quality. For example, in a survey we found that staff from most regional natural resource management organisations believe that their process is better than average – clearly an impossibility. In reality, the majority of prioritisation processes I have examined have had serious problems. The belief that they are strong is an impediment to the improvements that are needed. Addressing these misperceptions required strong leadership and participation in appropriate training, and it may be assisted by appropriate signals and incentives built into the program.
Equity vs effectiveness: Proposals to target investment in high-priority projects sometimes meet resistance in the form of arguments that this is inequitable – that resources should be distributed widely amongst many projects on the grounds of fairness. If natural resource or environmental outcomes are desired, these arguments should be resisted, as they can have a serious adverse effect on the achievement of those outcomes. A case built on maximising environmental benefits can readily be built.
Managing transaction costs
When considering potential improvements to the design and implementation of agri-environmental programs, there is a balance to be struck between improving natural resource outcomes and increasing transaction costs (Pannell et al., 2013b; and Chapter 16). The most detailed rigorous approaches are only worth the transaction costs involved for relatively large projects. To limit overall transaction costs I have two suggestions.
Beware many small projects: Avoid having the program being dominated by numerous small projects for which an investment in information and analysis cannot be justified. Such programs have little prospect of delivering and demonstrating genuine natural resource benefits.
Start broad, finish deep: Secondly, when evaluating potential investments, adopt a strategy of starting broad/shallow and finishing narrow/deep. In the early stages of the process, you can consider numerous potential projects, but each is evaluated in a relatively simple way that requires low transaction costs. This simple procedure is used to eliminate most of the projects from consideration. In the final stages, consider a relatively small number of project proposals, but require them to be developed in a rigorous way to allow sound decision making about them.
Ansell, D., Gibson, F. and Salt, D. (2016). Learning from Agri-Environment Schemes in Australia: Investing in Biodiversity and Other Ecosystem Services on Farms, Australian National University Press, Canberra. The complete book is available for download (free) or hard copy (purchase) at http://press.anu.edu.au/publications/learning-agri-environment-schemes-australia