"What drives societal and economic progress is innovation ... not the same as excellence." @MelindaKenneway #alpsp16 pic.twitter.com/XRXAVnFATC— Kudos (@GrowKudos) September 15, 2016
Change is coming, but it's not just metric wars, it's much bigger than that. Ben Johnson, Policy Advisor for HEFCE stressed the need for responsible metrics. They are everywehere - a layer on top of peer review. And for researchers, quality is wrapped up in publishing. The Research Excellence Framework (REF) costs £250 million. While it's only 2.5% of the overall cost of research, it is always questioned in terms of value for money.
If peer review is already happening in academia, should research assessment be more metrics driven? Asks @ersatzben #alpsp16— Phill Jones (@phillbjones) September 15, 2016
Whatever the concerns around peer review, he observed, it is still considered the gold standard. Inappropriate indicators can create perverse incentives so these indicators need to be underpinned by an open and interoperable data infrastructure. This means that ORCID ids and DOIs become even more important moving forwards. Johnson then announced the UK Forum for Responsible Metrics. Further details are available on the HEFCE website.
Jennifer Lin, Director of Product Management and Crossref, talked about the foundations needed to support the development of metrics. There has been an explosion of new, more diverse metrics that fit under the heading of 'altmetrics'. They hold power, constituting power, values, and livelihoods. But the infrastructure is mostly invisible, and you only feel it when it breaks. We need to understand the infrastructure behind metrics, understand metrics better, and the effect they have on research and researchers. Would a single non-proprietary body be ideal to coordinate our efforts on metrics?
Publisher call to action! Stop building the data desert, deposit & share full metadata, consider metrics you promote #alpsp16— Clare Hooper (@ClareHooperLUP) September 15, 2016
Liz Allen, Director of Strategic Initiatives at F1000, talked about the opportunities to share science and its impact. There are suggestions that the concept of the journal is outdated, and open access is merely tinkering at the edges.
Publishing processes can get in the way. They don't have editors, but do pre-pub checks for ethics, quality and readability. Their open approach allows them to give full credit to reviewers. Open Science feels like the democratisation of research - it's very exciting. They are working with funders on these types of initiatives, including the Wellcome Trust on Wellcome Open Research.Liz Allen: for me, the point of open access is not about free-to-read. It’s about the ability to repurpose and accelerate. #alpsp16— Charlie Rapple (@charlierapple) September 15, 2016
Dr Claire Donovan FRSA, Reader at Brunel University London, was the final panellist. She talked about the broader impact of research but warned about 'metric fatigue' as the practice of measuring impact ran ahead of the theory. In the UK the 2014 REF had 20% impact, but may increase to 25% for 2020. This had led to concerns that impact may be more time consuming and potentially gamed in the next assessment. Donovan observed that research is a craft industry, with lots of bespoke outputs, but we're trying to assess it as if it were mass produced. (You can read her slides here.)Beyond article-level metrics? Think instead beyond journals, beyond articles, beyond authors - contributors and content #alpsp16— Laura Snuffles (@LauraSnuffles) September 15, 2016
The Beyond Article-Level Metrics panel was held at the ALPSP Conference 2016. View all the sessions on the ALPSP YouTube channel.“Metrics should be about how to do things better” @melindakenneway sums up “beyond metrics” session at #alpsp16— Charlie Rapple (@charlierapple) September 15, 2016
No comments:
Post a Comment