In this series, we hear from the finalists for the 2020 ALPSP Awards for Innovation in Publishing. The winners will be announced at the ALPSP Virtual Conference on 16 September.
In this post, we hear from Anita Bandrowski and Martijn Roelandse about the RTI, Rigor and Transparency Index, powered by SciScore.
Tell us a bit about your organization.
We are a group of inventors that come from a long tradition of breaking technology and social barriers. We have a technology toolbox, but always ask “is the answer technology or culture?”
What is the project/product that you submitted for the Awards?
What is included? The rigor and transparency criteria (sex/gender as a biological variable, randomization of subjects into groups, investigator bias reduction through blinding, group size determinant via power calculation, identification of statistical tools, as well as the identification and authentication of key biological resources such as antibodies, cell lines, and organisms). Authors, funded by several funding agencies such as NIH or submitting to various journals such as Nature, are expected to address these criteria in their work, but some may not. The RTI is the average number of these criteria found in journal articles in a given year.
We think that if journals know where there are particular problems or where they stand compared to their peers, it may inspire them to work with authors to improve the quality of manuscripts.
Tell us a little about how it works and the team behind it
The RTI is based on our tool, SciScore, which has now been integrated into eJournal Press and will be available in Editorial Manager in the fall. This tool acts as an automated reviewer, which can check whether rigor criteria are addressed. It looks for sentences that discuss the particular criteria, for example “experiments were performed on both male and female mice, with subjects balanced for sex in each group” fulfills the “sex/gender as a biological variable” criteria. The authors or reviewers can then determine if the criteria are met in the context of the study, something that automated tools really can’t do at this point. While SciScore acts as a reviewer, the RTI index is the average SciScore for papers published in a given journal and given year.
The team behind this endeavor includes Anita Bandrowski who heads the Antibody Registry and the RRID initiative; Maryann Martone, one of the architects of FAIR; Martijn Roelandse, a former head of content innovation at a major publisher; and Burak Ozyurt, a computer scientist that has just taken the Google BERT AI, and made it run ...on a laptop ….in his spare time. We bring varied skills to find interesting solutions to very difficult problems.
In what ways do you think it demonstrates innovation?
Innovation is a tough thing to assess. Certainly there are plenty of ways that someone can score journals. However, as far as I am aware, all ways of scoring journals are based on the data around the article itself, i.e, how many times it is cited or tweeted about. The RTI delves deep into the science described in the paper and attempts to assess how good that is. It seems to me that this is something quite different, therefore potentially innovative.
What are your plans for the future?
“Pinky we are going to take over the world!!!” - The Brain (...or maybe not)
We are passionate about making science a little better, and a little more efficient. We want every study to be done in a rigorous way, we want every reagent to be explicitly specified so people reading scientific papers can quickly replicate key findings. If successful, this should accelerate the pace of scientific discovery, not just the rate of publishing.
Web links
RTI data
SciScore Press Releases (see who is incorporating SciScore)
Twitter
@sciscore
COVID-19 Project Twitter bot
You can hear from all of this year's finalists at the ALPSP Awards for Innovation Lightning Session on Tuesday 8 September. Visit the ALPSP website to register and for full details of the ALPSP Virtual Conference and Awards 2020.
The 2020 ALPSP Awards for Innovation in Publishing are sponsored by PLS.
No comments:
Post a Comment