290 – Advice for successful agri-environmental policy

Research and practical experience with agri-environment programs around the world provides many lessons on what leads to success or failure. New programs are often designed without sufficient awareness of these lessons, resulting in lost opportunities to achieve more valuable outcomes. A workshop to identify these lessons was held a couple of years ago, and the results have just been published by ANU Press as a book.

The book is available for sale or free download here. I’ve got a couple of chapters in the book, including one where I try to identify “best-practice in design and implementation” of agri-environmental policies. It is based on my experience over the past 15 years or so, working with federal and state agencies and regional bodies in Australia, and some similar organisations in several other countries. Here’s a longish extract. This is a longer-than-usual PD as I wanted to keep this material together in one post.

Design of programs/institutions

Additionality: Agri-environmental programs should aim to avoid paying farmers for undertaking actions that they would have done in any case. In other words, managers need to evaluate whether the benefits generated by a program investment are ‘additional’.

Continuation after investment ends: Where a program is intended to provide only temporary support to farmers (eg, in all Australian programs, but not in European programs), it is important to ensure that the actions being supported are attractive enough that farmers will continue to undertake them once funding ends. Otherwise the investment has no enduring benefit. These first two principles combine to mean that, where support will be temporary, perhaps the only defensible role for agri-environmental payments is to encourage farmers to get experience in a new practice that they are likely to be keen to continue once funding ends. The practice might be something new of which farmers are currently unaware, or one that becomes more attractive to farmers with experience.

aesAppropriate institutional delivery: In some agri-environmental programs, responsibility for overseeing some or all on-ground delivery of projects is devolved to regional organisations. This has been the case in all of Australia’s major programs since the late 1990s. In these cases, the program needs to be designed in a way that provides incentives for these regional organisations to respond appropriately. In particular, they should be incentivised to pursue sustained improvements in natural resource outcomes, rather than to support project activities without considering their resulting outcomes. There should be an emphasis on spending program resources well, rather than rapidly. Unfortunately, some of Australia’s major programs have generated incentives that go directly against these recommendations. Short time frames for programs and rules that funding will be withdrawn if not spent rapidly enough increase the difficulty of meeting this best-practice requirement. (Chapter 5 on environmental NGOs discusses how these organisations can help here.)

Balancing small, moderate and large projects: In programs where the availability of funding is small relative to the amount needed to fund all attractive projects (ie, in all Australian programs), there is often a temptation to share the available resources amongst a large number of small projects. Sometimes this results in good leverage of program resources, but often it means that almost all projects have inadequate resources and are unable to achieve worthwhile outcomes. This advice sometimes clashes with political preferences to support many projects rather than few. A compromise strategy could be to use a portion of funding (eg, 25 percent) to support many small projects to satisfy political needs, and use the remaining 75 percent to support larger projects that are more likely to be effective. On the other hand, achieving the most ambitious environmental targets is often disproportionately expensive, with costs increasing dramatically as targets become more ambitious. To maximise outcomes, it may be best to pursue a moderate number of moderate sized projects, rather than many small or few large projects.

Sufficient time for planning: Program performance is often hampered by a tendency for agencies to delay planning new programs until the end of a previous program is imminent or past. Good planning and design of programs and prioritisation of investments requires more time than is usually allowed. Ideally, organisations should commence planning and analysis to develop the next program years before the end of the current program. Even though the scope and parameters of the next program cannot be known in advance, these can be predicted, and sometimes influenced, by the agency to some extent. Having already-analysed investment options ready to put forward can be highly persuasive, and increases the likely environmental benefits generated.

Investment longevity: Finally, funding for agri-environmental programs in Australia tends to be temporary and short-term – typically five years. Environmental problems usually take much longer than this to resolve, so systems for providing long-term funding should be used where possible. If it is not possible to ensure long-term funding, then this should have a strong influence on which projects are selected for funding. In particular, projects that would require significant funding in the long-term to maintain the benefits generated by an initial project should not be supported. For example, most projects for control of feral animals or plants would fall into this category because feral animals reinvade once control ends. Similarly, cases where farmers are likely to disadopt practices once funding ends should be excluded. Typically, programs are much too optimistic about ongoing adoption of practices post-funding.

Design of projects/investments

SMART targets: A number of agri-environmental programs have been criticised for failing to establish appropriate targets (eg, European Court of Auditors, 2011; Auditor General, 2008; Park et al, 2013). Specifically, targets should be SMART (Specific, Measurable, Achievable, Relevant and Time-Bound) in order to facilitate monitoring and evaluation of a program, and to ensure that the funded investments are focused onto suitable activities (see Chapter 4 on setting SMART targets).

Project logic: Many projects funded in agri-environmental programs are not designed in a logically consistent way. They are consistent with a ‘project logic’ but only in a qualitative sense. They fail when assessed against quantitative questions, such as ‘are the funded activities sufficient to achieve the intended land-use changes?’ or ‘are the intended land-use changes sufficient to achieve the desired natural resource outcomes?’ A good project logic is more than a description or diagram of connections between elements of the system being managed or influenced; it quantifies the connections and makes assumptions transparent.

Selecting the right policy tool: There is a tendency for little thought or analysis to be put into the selection of policy mechanisms to be used in a project or program, resulting in inappropriate choices in many cases. In Australia, there is too much reliance on extension in situations where it cannot deliver the desired outcomes. For example, Australia’s national salinity program between 2001 and 2007 relied mainly on extension to encourage farmers to change their practices, but the practices being promoted were not attractive to farmers on the required scale and so were adopted to a very limited extent – much too limited to achieve the program’s goals (Pannell and Roberts 2010). On the other hand, in Europe and the United States, financial payments are almost the sole mechanisms used, often funding activities that are not additional. The framework of Pannell (2008) helps organisations to evaluate the type of mechanism that is most suitable for a particular project. (See also Chapter 18 on the choice of tools depending on public benefits and private benefits arising from an investment.) Sometimes programs specify which policy mechanisms will be used by projects prior to identification of the projects, and then project investments are selected without considering whether they are suitable for the pre-determined policy mechanism. Preferably, policy mechanisms should be selected to match the type of projects that will be necessary to achieve the desired program outcomes. They should be project-specific, to some extent. As noted in Chapter 18, Australian programs tend to rely too much on extension and too little on the development of technology.

Ranking projects/investments

Prioritisation: Where funding is limited, prioritisation of investment options is essential. The quality of the prioritisation process can make a major difference to the natural resource outcomes delivered (eg, Barry et al, 2014).

Rank projects: Programs should prioritise projects, not problems, regions or issues. Some programs prioritise regions or issues without defining projects, but this means that it is not possible to properly consider issues of project cost, project risks, project benefits, or time lags, all of which should be factored into the prioritisation process.

Rank according to value for money: Projects should be ranked according to their value for money: a measure of their benefit divided by their cost (see Chapter 15 on designing cost-effective agri-environment schemes). Failure to do this is one of the most serious mistakes that can be made when ranking projects, but unfortunately it is common. Some systems fail to consider costs entirely, some do include costs but fail to divide by them, and many include only some of the costs that should be considered. For example, it is important to factor in long-term maintenance costs, since they vary so much between different projects, but few Australian systems do so. If maintenance costs are needed but are not expected to be provided, then project benefits should be scaled down accordingly in the ranking process.

Measure the gain against a counterfactual: When ranking projects, benefits should be estimated from the predicted difference in natural-resource outcomes with versus without the proposed investment (Chapters 19 on counterfactuals). Although this seems like common sense, Maron et al, (2013) found that 15 out of 16 systems in actual use for ranking biodiversity projects failed to do this correctly.

Incorporate all the benefits and risks: There are many factors that could be considered when estimating the benefits of a proposed project. The essentials are: the potential values generated, the likely level of adoption/compliance with the project by landholders (Pannell et al., 2006; Chapters 12 and 13) various risks that might result in project failure (technical, social, financial and managerial risks) and time lags until benefits are generated.

Use a sound metric: A commonly neglected issue is how to combine the variables that determine benefits and costs into a metric for ranking projects. Most metrics in actual use are theoretically unsound and provide poor rankings, even where project information is accurate. Indeed, as is discussed in chapter 17, the level of benefit delivered is more sensitive to the quality of the metric than the quality of the information fed into the metric. Potential benefits from investment are very sensitive to the use of inferior ranking metrics. Chapter 17 on metrics and Pannell (2015) outline the requirements for a sound ranking metric.

Managing uncertainty

When decisions about project funding are being made, uncertainty about those projects is usually high. Common areas of major uncertainty include the technical feasibility or effectiveness of the proposed actions to be funded by the project, and the valuation of those environmental benefits that are generated. Uncertainty should be accounted for in several ways.

Identify key uncertainties: Project proponents should be required to identify key uncertainties, and to specify what will be done in the project to reduce them.

Carry out feasibility assessment and pilot studies: Projects above a certain scale should be subject to rigorous feasibility assessment before longer term funding is committed. Funding to support information collection, perhaps in a pilot study, should be provided for 6 to 12 months, after which longer-term funding should be conditional on the results obtained.

Learn from early experience: Projects and programs should be managed in an adaptive way, with information collected during early stages of the program or project being used to inform changes in management or even cessation in some cases. In practice, few programs operate with that degree of flexibility, resulting in continuation of poorly designed investments past the time when their faults are apparent.

Managing people’s biases, preconceptions, self-interest

Acknowledge the values that people bring with them: Being human, the people involved with agri-environment programs are subject to biases, preconceptions and self-interest (consider the way restoration vs conservation is valued in different ways in different places Chapter 10), all of which can reduce program performance. If managers are aware of these human traits, they can introduce systems to limit their negative impacts.

Optimism vs realism: A pervasive problem is the tendency for people to be overly optimistic about proposed projects. It is common to see proposals in which the benefits are exaggerated, and the costs, risks and time lags are under-estimated. Several factors contribute to this, including vested interests, wishful thinking, and a failure to recognise all relevant difficulties and risks that are likely to affect a project. The ideal strategy to overcome this problem is serious independent expert review of project proposals, but this is only justifiable where projects are sufficiently large. This is another factor that tends to favour moderately large projects over small.

Self blindness: People involved in allocating program funds commonly perceive that their existing prioritisation process is of high quality. For example, in a survey we found that staff from most regional natural resource management organisations believe that their process is better than average – clearly an impossibility. In reality, the majority of prioritisation processes I have examined have had serious problems. The belief that they are strong is an impediment to the improvements that are needed. Addressing these misperceptions required strong leadership and participation in appropriate training, and it may be assisted by appropriate signals and incentives built into the program.

Equity vs effectiveness: Proposals to target investment in high-priority projects sometimes meet resistance in the form of arguments that this is inequitable – that resources should be distributed widely amongst many projects on the grounds of fairness. If natural resource or environmental outcomes are desired, these arguments should be resisted, as they can have a serious adverse effect on the achievement of those outcomes. A case built on maximising environmental benefits can readily be built.

Managing transaction costs

When considering potential improvements to the design and implementation of agri-environmental programs, there is a balance to be struck between improving natural resource outcomes and increasing transaction costs (Pannell et al., 2013b; and Chapter 16). The most detailed rigorous approaches are only worth the transaction costs involved for relatively large projects. To limit overall transaction costs I have two suggestions.

Beware many small projects: Avoid having the program being dominated by numerous small projects for which an investment in information and analysis cannot be justified. Such programs have little prospect of delivering and demonstrating genuine natural resource benefits.

Start broad, finish deep: Secondly, when evaluating potential investments, adopt a strategy of starting broad/shallow and finishing narrow/deep. In the early stages of the process, you can consider numerous potential projects, but each is evaluated in a relatively simple way that requires low transaction costs. This simple procedure is used to eliminate most of the projects from consideration. In the final stages, consider a relatively small number of project proposals, but require them to be developed in a rigorous way to allow sound decision making about them.

Further reading

Ansell, D., Gibson, F. and Salt, D. (2016). Learning from Agri-Environment Schemes in Australia: Investing in Biodiversity and Other Ecosystem Services on Farms, Australian National University Press, Canberra. The complete book is available for download (free) or hard copy (purchase) at http://press.anu.edu.au/publications/learning-agri-environment-schemes-australia

289 – Interpreting evidence through an ideological lens

Humans are prone to “confirmation bias”, meaning that when we see evidence, we tend to interpret it in a way that reinforces our existing beliefs. I felt I was observing confirmation bias in action when I observed a recent article that discussed some research we conducted. 

The tendency for confirmation bias is so strong that, in some cases, people who hold opposing views can both find support for their conflicting positions from exactly the same evidence, resulting in attitude polarisation. I would guess that this phenomenon has probably contributed to the polarization of attitudes to climate change, for example.

Occasionally I see an example of my own research being interpreted in a way that seems to exhibit confirmation bias. A recent example is our research showing that farmers in central Victoria, on average, pay more for land that includes some native woody vegetation, compared with land that is fully cleared (Polyakov et al. 2015 and see PD#287). There was some nice press coverage of the research (here and here), some social media attention, and also a brief article published on “Freedom Watch”, a web site published by the Institute of Public Affairs (IPA).

The IPA describes itself as “the voice of freedom” and on the Freedom Watch site the published articles cover issues such as freedom of speech and legal restrictions on various freedoms, including laws that restrict racial discrimination, live betting ads, and political donations. Of course, different people have different opinions on the sorts of restrictions to freedom that are discussed by the IPA, but the authors of articles on the site would tend to be at one end of the spectrum of public opinion about these issues.

ipaSo it was interesting to see the article by Lorraine Finlay (a Law Lecturer at Murdoch University) titled “We don’t need laws to tell farmers to like trees”. In the article she claims that “the strident environmental lobby … will tell you that farmers basically hate trees” and “This is the reason that we apparently need punitive native vegetation legislation across the country.” She rhetorically asks “if the private market values native vegetation, why does government have to interfere at all?” and suggests that “Rather than stripping away private property rights by imposing punitive native vegetation legislation, perhaps we would achieve better environmental outcomes by actually working cooperatively with our farmers.”

Now I’m an economist, so I’m certainly favourably predisposed towards free markets, and I’m not unsympathetic to the argument that we should avoid having laws that needlessly restrict freedoms, but this article really feels like the evidence has been shoehorned to support a pre-determined position. For a start, the premises of the article are questionable. I’ve worked on issues of agriculture and the environment for many years, in the process dealing with people from many different environmental organisations, and I’ve never heard anybody say that farmers hate trees. Maybe some environmentalists do think that, but the great majority of people who are concerned with environmental issues in rural areas have a much more realistic and nuanced understanding of farmers’ attitudes towards trees.

The claim that farmers’ hatred of trees is put forward as the reason for legal restrictions on land clearing is specious. Those arguing for these legal restrictions don’t need to invoke a non-existent hatred of trees. The real argument is that there are public benefits from preservation of native vegetation that are not sufficiently factored into the farmers’ private decisions about land clearing. The public benefits include conservation of threatened species or ecological communities, reduced salinity in waterways, and aesthetic benefits. Why does government have to interfere at all? Because, with so little native vegetation left in many regions, the marginal value of protecting it is likely to be very high, so that the public benefits of the legislation are likely to far outweigh the private costs. Even though farmers like some trees, when the benefits to the rest of the community are factored in, the optimal area of trees is larger than farmers would select for themselves. Economists describe this situation as “market failure”, since the aggregated decisions of private individuals do not add up to the best outcome for society as a whole.

Lorraine Finlay’s argument that we should work cooperatively with farmers makes me wonder whether she’s aware of existing programs that do just that. Ever since the start of the National Landcare Program in around 1990, there have continuously been major national programs built on working closely with farmers to improve environmental outcomes. That long experience has shown us that the voluntary/cooperative approach can work effectively up to a point, but is less successful for the more challenging environmental issues that are more expensive to deal with, like salinity and biodiversity conservation. It doesn’t necessarily follow that legislation is the best response, or that the system we have in place is as fair and efficient as it should be, but I do think it’s clear that a voluntary/cooperative approach is not sufficient if we seek to maximise the overall public and private net benefits.

We can also observe the results of a recent policy experiment in Queensland, in which a state government led by Premier Campbell Newman made it easier for farmers to clear native vegetation if they wished to. The result was a surge in clearing which is so large that it is likely to wipe out all of the gains in carbon abatement from the Australian Government’s Direct Action program. Apparently, the positive attitude towards native vegetation that we identified amongst farmers in central Victoria does not apply uniformly across the country.

To be fair to the IPA, they aren’t the only ones to make less-than-completely-balanced use of our results. A Tweet from @ARC_CEED (which funded our research) said “Study @MaksymPolyakov @dpannell66 finds trees increase value of rural properties up to 25%”. That’s perfectly true, although the 25% result was an extreme case, and only relevant to very small properties.

Another of our conclusions that hasn’t been picked up in any of the commentary is that increasing the area of woody native vegetation on a property can actually decrease the property value if the area is too large (see the graph in PD#287). It’s interesting that both sides are more enthusiastic about the conclusion that trees can increase property values.

Further reading

Polyakov, M., Pannell, D.J., Pandit, R., Tapsuwan, S. and Park, G. (2015). Capitalized amenity value of native vegetation in a multifunctional rural landscape, American Journal of Agricultural Economics 97(1):299–314.  Journal web page  ♦ Ideas page