Altmetrics: New Ways to Measure the Impact of Research Products

By Yan Fu, PSC Librarian, April 2014

If you went to Christopher Monk’s PSC brown bag in February, “Neuroimaging and Population Science,” you may remember he mentioned the article in PNAS where he and his colleagues published their findings.

Viewing this article on the PNAS website, you find the expected abstract, full text, and supplemental materials. But you may be new to the metrics component, which aims to capture the article’s online impact and usage statistics.

These current article-level metrics, termed altmetrics, go beyond traditional citation counts, demonstrating an article’s impact and reach by collecting data on page views and downloads, bookmarks, blog postings, and mainstream/social media dissemination. Publishers such as Springer, Elsevier, Wiley, Cambridge Journals, BMJ Journals, Nature Publishing Group and PLoS have all started to experiment with the use of alternative metrics. Although no standard practice has emerged so far, all use an altmetrics score – some along with traditional citation data. Currently, about 35% of biomedical papers and about 10% of social science papers have altmetrics scores.

How does the leading provider, Altmetric, derive scores? The “Cancer Immunotherapy” article in Science shown below has a score of 350, which reflects three main factors: volume, sources, and authors.

The score rises as more people mention the article, but the algorithm also accounts for the sources, giving descending weight to newspaper articles, blog posts, and tweets, and factoring in the authoritativeness of the bloggers/tweeters and their intended audiences. So a subject specialist/entity sharing a link with other specialists gets a higher score than a Facebook mention by a lay person.

The visual representation — the Altmetric donut — is color coded by source type, blue for Twitter, yellow for blogs, and red for mainstream media sources. While traditional citations often take years to accrue, altmetrics provide an overview of an article’s online attention from the moment it is posted.

Several major players provide alternative metrics data including Altmetric, Plum Analytics, and ImpactStory. They overlap in the data sources they collect, but each has its own focus area.

Altmetric, owned by Macmillan Publishers, provides article-level metrics for researchers and publishers. Its website provides a list of data sources it tracks.

In addition to journal articles, Plum Analytics covers books, conference papers, datasets, web pages, and more. It provides a dashboard that summarizes online activities of researchers, groups, and institutions, categorizing metrics into five types: usage, captures, mentions, social media, and citations.

The University of Pittsburgh dashboard, for example, captures the impact of U of Pittsburgh’s work and outreach by listing researchers and their products with metrics data. You can easily browse to the group or individual level of data from the institution dashboard.

Funded by NSF and Sloan Foundation, ImpactStory is an open-source tool that researchers can use to discover how their research articles, datasets, software, slides, etc. are being used. It collects and displays data on viewing, discussion, saving, and citations. You can create a collection by importing your Google Scholar profile, for example. The web tool sorts metrics by engagement type and audience.

How are researchers reacting to the rise of altmetrics? The Chronicle of Higher Education featured a story last year about how a researcher used altmetrics data for tenure promotion. It points out that the rise of altmetrics revives questions about how to measure the impact of research.

Altmetrics expand the view of what research impact looks like by tapping new sources and new dissemination routes. Because altmetrics encompass both nontraditional data and the algorithms that interpret the data, many have questioned the meaningfulness of these scores.

But even without the scores, gauging how research findings spread throughout layers of the popular and scholarly dissemination systems is of interest in itself.

Regardless of the shortcomings associated with altmetrics, their use is a natural outgrowth of the wide online availability of scholarly literature, the expansion of research sharing and collaborative tools such as Mendeley, Figshare, SlideShare, Dryad and ResearchGate and the increasing use of social media for research dissemination.

It is also reasonable to predict that the increasing use of alternative measures will prompt more researchers to use social media and collaborative tools to maximize the visibility and impact of their research.

Contact PSC Library to find out more about managing your online presence and increasing visibility and dissemination of your research results.