¶ 1 Leave a comment on paragraph 1 0 I developed an interest in research evaluation as an advocate of open research process. It is clear that researchers are not going to change themselves so someone is going to have to change them and it is funders who wield the biggest stick. The only question, I thought, was how to persuade them to use it
¶ 2 Leave a comment on paragraph 2 0 Of course it’s not that simple. It turns out that funders are highly constrained as well. They can lead from the front but not too far out in front if they want to retain the confidence of their community. And the actual decision making processes remain dominated by senior researchers. Successful senior researchers with little interest in rocking the boat too much.
¶ 3 Leave a comment on paragraph 3 0 The thing you realize as you dig deeper into this as that the key lies in finding motivations that work across the interests of different stakeholders. The challenge lies in finding the shared objectives. What it is that unites both researchers and funders, as well as government and the wider community. So what can we find that is shared?
¶ 4 Leave a comment on paragraph 4 0 I’d like to suggest that one answer to that is Impact. The research community as a whole has stake in convincing government that research funding is well invested. Government also has a stake in understanding how to maximize the return on its investment. Researchers do want to make a difference, even if that difference is a long way off. You need a scattergun approach to get the big results, but that means supporting a diverse range of research in the knowledge that some of it will go nowhere but some of it will pay off.
¶ 5 Leave a comment on paragraph 5 0 Impact has a bad name but if we step aside from the gut reactions and look at what we actually want out of research then we start to see a need to raise some challenging questions. What is research for? What is its role in our society really? What outcomes would we like to see from it, and over what timeframes? What would we want to evaluate those outcomes against? Economic impact yes, as well as social, health, policy, and environmental impact. This is called the ‘triple bottom line’1 in Australia2. But alongside these there is also research impact.
¶ 6 Leave a comment on paragraph 6 0 All these have something in common. Re-use. What we mean by impact is re-use. Re-use in industry, re-use in public health and education, re-use in policy development and enactment, and re-use in research.
¶ 7 Leave a comment on paragraph 7 0 And this frame brings some interesting possibilities. We can measure some types of re-use. Citation, retweets, re-use of data or materials, or methods or software. We can think about gathering evidence of other types of re-use, and of improving the systems that acknowledge re-use. If we can expand the culture of citation and linking to new objects and new forms of re-use, particularly for objects on the web, where there is some good low hanging fruit, then we can gather a much stronger and more comprehensive evidence base to support all sorts of decision making.
¶ 8 Leave a comment on paragraph 8 0 There are also problems and challenges. The same ones that any social metrics bring. Concentration and community effects, the Matthew effect of the rich getting richer. We need to understand these feedback effects much better and I am very glad there are significant projects addressing this.
¶ 9 Leave a comment on paragraph 9 0 But there is also something more compelling for me in this view. It let’s us reframe the debate around basic research. The argument goes we need basic research to support future breakthroughs. We know neither what we will need nor where it will come from. But we know that its very hard to predict – that’s why we support curiosity driven research as an important part of the portfolio of projects. Yet the dissemination of this investment in the future is amongst the weakest in our research portfolio. At best a few papers are released then hidden in journals that most of the world has no access to and in many cases without the data, or other products either being indexed or even made available. And this lack of effective dissemination is often because the work is perceived as low, or perhaps better, slow impact.
¶ 10 Leave a comment on paragraph 10 0 We may not be able to demonstrate or to measure significant re-use of the outputs of this research for many years. But what we can do is focus on optimizing the capacity, the potential, for future exploitation. Where we can’t demonstrate re-use and impact we should demand that researchers demonstrate that they have optimized their outputs to enable future re-use and impact.
¶ 11 Leave a comment on paragraph 11 0 And this brings me full circle. My belief is that the way to ensure the best opportunities for downstream re-use, over all timeframes, is that the research outputs are open, in the Budapest Declaration3 sense4. But we don’t have to take my word for it, we can gather evidence. Making everything naively open will not always be the best answer, but we need to understand where that is and how best to deal with it. We need to gather evidence of re-use over time to understand how to optimize our outputs to maximize their impact.
¶ 12 Leave a comment on paragraph 12 0 But if we choose to value re-use, to value the downstream impact that our research or have, or could have, then we can make this debate not about politics or ideology but how about how best to take the public investment in research and to invest it for the outcomes that we need as a society.
¶ 13 Leave a comment on paragraph 13 0 Notes: This post was originally published in July 2011 as “(S)low impact research and the importance of open in maximising re-use“. It is an edited version of the text that I spoke from at the Altmetrics Workshop 5 in Koblenz in June 2011 and amongst the first times I linked “impact” with re-use, something that has been a theme of my talks ever since. There is also an audio recording of the talk6 I gave available as well as the submitted abstract 7 for the workshop.
- ¶ 14 Leave a comment on paragraph 14 0
- Triple Bottom Line, Wikipedia, retrieved from https://en.wikipedia.org/wiki/Triple_bottom_line ↩
- I’d come across this in the Australian context but as the Wikipedia article referenced above makes clear this wasn’t a peculiarly Australian invention. The term was coined by John Elkinghorne ↩
- Budapest Open Access Initiative (2003), retrieved from http://www.budapestopenaccessinitiative.org/read ↩
- By 2011 it is the Budapest Declaration that I reflexively use to define “open” ↩
- The first Altmetrics Workshop was organised by Jason Priem and others and was the start of many conversations. It would be interesting to look at the attendance list for that meeting today. ↩
- Neylon, C (2011) Re-use as Impact (audio recording), retrieved from https://archive.org/details/Re-useAsImpactTalkAtaltmetricsWorkshop ↩
- Neylon, C (2011) Re-use as Impact: How re-assessing what we mean by “impact” can support improving the return on public investment, develop open research practice, and widen engagement(v0), retrieved from http://altmetrics.org/workshop2011/neylon-v0/ ↩