How should scholarly communications be evaluated? Because scholars’ work varies so widely, it’s impossible to find quantitative measurements that work equally well for everyone, but more and more tools are being developed to provide a broader picture of a work’s relevance and importance in its field and beyond.
Scholars have long been evaluated based on how many publications they have, and which journals or presses they publish with. Sometimes these journals are ranked or scored with systems like Impact Factors, Eigenfactors, and other journal ranking systems.
- UC Irvine Libraries guide on Journal Impacts
- UCLA guide on Journal Impacts
- UC Merced Library guide on Impact Factor
The problem with journal ranking schemas is that, while they represent the perceived value of any given journal, they don’t give an accurate picture of the perceived value of any single article within that journal. Increasingly, metrics are shifting to account for the impact made by articles, regardless of their place of publication.
Article Level Metrics (ALM)
The most straightforward implementation of ALM is the calculation of the number of views and downloads of an article for a given period. These metrics might also list the number of citations for a particular article. For a particular individual, this might be used to calculate things like an h-index, which is based on an author’s most cited papers and the number of times these papers are cited in other publications.
AltMetrics and ALM are often confused because they both represent the effort to find new and more precise ways of identifying a single publication’s impact. AltMetrics, however, is a widespread initiative to look beyond download and formal citation rates and discover where, how and by whom a given article is being used. Currently, those efforts are focused on social media, such as Facebook and Twitter, but the general philosophy transcends any particular medium. Examples include:
- See Atlmetric’s data applied to PLOS articles in the PLOS Impact Explorer
All of these measurements are only as useful as the accuracy of the data on which they’re based. One way to ensure reliable data is through the use of permanent, unique identifiers. Identifiers for books and journals have been around for a long time, but increasingly, identifiers are being attached to individual articles and datasets. They’re also being used to identify scholars, since authors’ names may change, have many permutations (first name or initial? include middle initial or not?), or be commonly shared among multiple individuals.
Examples of Identifiers
Benefits of identifiers
As mentioned above, the basic function of identifiers is to identify a unique item and distinguish it from other items. All the identifiers listed above have the additional advantage of being persistent – that is, unlike a URL which may change or disappear, they are intended to remain permanently linked to the identified item, or to information describing that item. And as discussed above, once a publication or an author has a unique identifier, indexes can find and collect information related to that publication or author.
Profiles for authors
Many services offer free profile services to academic authors intended to increase their visibility, help them connect with other scholars, or track the use of their publications. Some of these services are affiliated with a particular author identifier, others are not. Your campus may also have its own profile system, such as UCSF Profiles or the UC Irvine Faculty Profile System, which may interact with certain identifiers or other profile services.
In fact, many of these systems can connect with each other – for instance, ORCID can gather information from Scopus or Europe PubMedCentral if an author authorizes it. The Utrecht University Library describes the basic steps required to get started with most of them.
|ORCID||Non-profit open source effort to tie all researcher identifier systems together. Connects to ResearcherID, Scopus, CrossRef, and more. Authors can add information to their profiles about their website, education, employment, grants, publications, and more.|
|Google Scholar Citations||Allows an author to set up a profile and link it to publications indexed in Google Scholar. Authors can add a photo, affiliation, and interests. Automatically tracks citations to those articles in Google Scholar, then uses that data for things like graphs and calculating h-index.|
|ResearcherID||Allows an author to set up a profile and link it to publications indexed in Web of Science. Authors can add websites, affiliations, and a description of their work.|
|Mendeley||Mendeley is a reference manager that people use to track, annotate, and share publications they read. Authors can set up a profile in Mendeley to track Mendeley’s millions of users’ interest in their publications. They can then add CV information, upload their papers, and join groups based on their research interests.|
|Academia.edu and ResearchGate||Academia.edu and ResearchGate are social networking sites for academic authors. Authors can set up a profile, add their publications, and follow other authors.|
|ImpactStory||ImpactStory attempts to track a wide variety of scholarly outputs – articles, posters, presentations, software, and more. It tracks not just traditional citations to these outputs, but also whether they’re being publicly discussed in places like blogs and Twitter. It connects with ORCID, Scopus, Crossref, Github, and more.|
Services for creating identifiers for publications and other items
Book and journal publishers typically handle the process of obtaining ISBNs and ISSNs. Depending on the publisher, journals may also provide identifiers to articles. If you are an institution or publisher who needs to issue identifiers, you may want to learn more about EZID.
If you are an individual who has a publication – a dataset, a poster, a presentation, or something else – that needs an identifier and your publisher has not assigned one to it, you can see whether your institution is already using EZID. If it is, please contact EZID for help getting connected. If it isn’t, you can check out some of EZID’s clients that create DOIs for any author, regardless of affiliation, so long as they agree to share it under a broad license:
- Dryad is a nonprofit repository that welcomes data files associated with any published article in the sciences or medicine, as well as software scripts and other files important to the article.
- Figshare, started by a PhD student and now owned by MacMillan Publishers, supports any research output, associated with an article or not, including posters, figures, and media.
To learn more about publishing data generally, see our page on open data.