There has been a lot of research on the benefits of research, but little of it has addressed environmental research. In some ways, this is understandable, as it’s difficult. But we need to develop better ways to estimate these benefits as researchers are increasingly asked to justify their funding and quantify their impacts.
I organised a small workshop in Brisbane a few weeks ago on estimating the benefits of environmental research. If we could generate this information, it would be useful in several ways. It could be used to make judgements about whether particular research projects are worth doing, to identify priorities from a set of potential projects, and to make the case for continued funding of environmental research. Also, the process of working out the likely benefits could help us understand the ways that research generates benefits, and that might help us to do a better job of generating benefits.
However, as we quickly agreed at the workshop, this is a very difficult thing to do well. For one thing, there are so many different types of environmental research with different possible uses and impacts, and some of them need different thinking and approaches to analysis.
We decided to focus our attention onto the type of research that is least well served by existing tools and frameworks: research that is intended to influence environmental policy. It turns out that this is the most neglected aspect for a reason – it’s the most difficult one to deal with.
You can see why it’s difficult from the following list of stages that one must go through, starting from research and ending up with real-world benefits.
- Funding is allocated to research and research is done
- Something useful is learned – new information is generated (or isn’t)
- The new information influences policy/management (or doesn’t)
- Policy change is implemented by policy makers (or isn’t)
- If the purpose of the policy is to change the behavior of people or businesses, these people respond to the changed policy (or don’t)
- Changes (relative to no research) result in the environment (or not), including unexpected or unintended consequences
To estimate benefits, we need to estimate what happened (or predict what will happen) at each of these stages. If one link in the chain breaks, benefits are not generated. We also need to estimate (or predict) what would have happened in the absence the research – something you can’t actually observe even if the research has been completed and had its impacts.
Research that aims to influence policy is particularly difficult to assess, because the process of policy change is so complex and influenced by numerous factors. It is very difficult to judge what proportion of any particular change may be attributable to the research rather than other factors. This is recognised in the literature as the attribution problem.
Despite all the difficulties, we found that the existing frameworks for research evaluation provided enough of a platform for us to think productively about what we would do for this type of research. A team of us will be working on this challenge over the next while. We aim to work out what would be needed for a comprehensive rigorous framework, and from that produce a set of principles and perhaps rules of thumb that researchers, research funders and policy makers can use when they need to think about the benefits of policy-oriented environmental research.
van der Most, F. (2010). Use and non-use of research evaluation: A literature review, Paper no. 2010/16, Circle, Lund University, Sweden. Here