James Hardcastle on Citation Analysis: Some common issues and problems.

James Hardcastle is a Senior Research Executive at Taylor & Francis where he has worked for the past five years specialising in citation analysis and bibliometrics.

He has responsibility for leading the citation analysis within the company and training staff worldwide in this area. Here, he reflects on common issues and problems with citation analysis in a guest post.

Powerful tools for publishers

Citation analysis and broader bibliometric study are powerful tools for academic publishers to use in developing and supporting their journals. This analysis allows us to understand both who is writing in our journals as well as who is citing them: the strength of bibliometrics from a publishers perspective is that it allows us to look beyond our own journals and data to find information regarding broader trends in the market and subject area.

Further citation-based metrics such as the Impact Factor are still dominate measures of journal value. Using tools such as Thomson Reuters Web of Science allows us to analysis these numbers in more detail. However, there are some major issues around citation analysis and metrics that have to be taken into consideration.

Not all subjects are the same

Subjects behave very differently in terms of citation and authorship patterns, even across related subject areas.  Therefore articles and journals from different subject areas should not be compared against each other. 

Coverage counts

The main databases have very different coverage; Web of Science covers around 11,000 journals, compared to more than 18,000 journals in Scopus and an unknown corpus in Google Scholar. This means that content will receive different citation counts in different databases, which should be considered when using generic metrics such as the h-index that can yield different results between databases.

ALPSP training course delegates debate...

Not all numbers are the same

Different types of published content behave in different ways, for example review articles tend to receive more citations than research articles and short communications tend to receive citations more quickly. Therefore Impact Factors and citation counts for different types of content should not be directly compared.

Distribution, time and gaming

Other important issues that should be born in mind include the distribution of citations; the time it takes for citations to appear and are not instantly metric; the fact that citation metrics can be gamed; and citations to editorials, book reviews and meeting abstracts as distinct from articles. These issues can lead to mistakes in using citation analysis tools by not considering the effect they have on the data and the metrics.

If you want to find out more, join me and Iain Craig from Wiley-Blackwell on the ALPSP Citation Analysis for Publishers course next month in Oxford.


Note
James is co-tutor on Citation Analysis for Publishers alongside Iain Craig from Wiley-Blackwell. Book now for Citation Analysis for Publishers, 2 May 2013, Oxford. Follow James on Twitter @JamesHTweets