Bibliometrics is the application of mathematical and statistical methods to measure, evaluate and study the production and dissemination of books, articles and other publications. Like scientometrics, bibliometrics is a sub-discipline of the information sciences.  Bibliometrics has produced indicators that are now used for the evaluation of research, at different scales: researcher, team, laboratory, institution, country and continent. It would be difficult to make an exhaustive list of bibliometrics indicators. Moreover, the list has increased over the years. The most well-known indicators are presented here.

Doctoral students will hear talk of bibliometrics and even, in the STM (Sciences, Technology, Medicine) fields, use bibliometric indicators:

• when choosing a thesis subject, to assess the thematic environment
• when establishing the state of the art on a subject, to identify and locate the journals and other publications in the field, or to identify research units specialized in certain areas
• when publishing their own articles, to choose a publisher,
• when looking for collaborations for their research unit to initiate a project...

photo credit: bcostin via photopin cc

The analysis of the scientific literature published from 1850 to 1860 in the field of anatomy might be considered the first bibliometric study: Cole, F. J., Eales, N. B. 1917. The history of comparative anatomy. Part I: A Statistical Analysis of the Literature. Science Progress, 11, 578-596.

In 1927, the references cited in The Journal of American Chemistry, were used to help librarians choose their journals: Gross, P. L. K. & Gross, E. M. 1927. College libraries and chemical education. Science, 66, 385-389.

In 1934, the Bradford Law, named after the British librarian Samuel C. Bradford, was used to model the distribution of journals by scientific field, based on the number of articles and the themes for each journal. It is still a resource for librarians.

In the 1960’s, the American researcher, Eugene Garfield made his mark on the information sciences. He created the Institute for Scientific Information (ISI) and developed the Science Citation Index (SCI) bibliographic and bibliometric database, followed by the Social Sciences Citation Index (SSCI) and the Arts & Humanities Citation Index (AHCI) databases. These databases, which are essential for institutional bibliometrics, are now commercialized by Thomson Reuters as the Web of Science.

A research community is gradually being built up around quantitative usages, and new journals have appeared: Journal of The American Society for Information Science (JASIS, 1950), Scientometrics (1978), Research Evaluation (1991)…

The last few years have seen the creation of a multitude of bibliometric indicators, and the development of "webometrics" based on the "PageRank" algorithms (1998). From a domain restricted to library science, bibliometrics has now spread to the research sector and its administration.

There is no single database that includes all the publications of all researchers, and access to the different bibliometric databases will depend notably on your institution of affiliation. Some tools are open-access: Google Scholar, CiteseerX, Citebase, Publish or Perish, SCImago Journal & Country Rank, and others.  However, the tools most widelyused for research evaluation are restricted-access (by subscription or authorization): Web of Science (Thomson Reuters), Journal Citation Report (Thomson Reuters), Scopus (Elsevier), SIGAPS.

Regardless of the indicators used, it is important to always mention the database from which they have been calculated. The sources identified in the database, the types of publications, the citation period, and the corrections to affiliations made by the producer are all elements that will affect the results.

Yves Gingras wrote, in the conclusion of his book "Les dérives de l'évaluation de la recherche" (Raisons d'agir, 2014), "bibliometrics is essential to the global mapping of the research status at a given time and a given place, and thus to surpass local and anecdotal perceptions. It also allows the identification of trends at different levels: regional, national and global, that would be impossible to establish otherwise. Bibliometrics also highlights the fact that publication, citation and collaborative practices are different depending on the discipline and specialty [...]. Although very useful, bibliometrics should be handled with caution and rigor”.

Like this book, most publications on bibliometrics refer to the limitations of the indicators used and warn users in particular about:

1. Invisibility: in certain disciplines, particularly in the humanities and social sciences, scientific journals are not indexed in the bibliometric databases. Among those that are, Anglo-Saxon journals represent the large majority, since the databases will necessarily cover more Anglo-Saxon research than French, Spanish or Swedish. Similarly, books and conference proceedings, which are important vectors for the dissemination of research, are almost always absent. Some research domains therefore go completely unnoticed in the bibliometric analyses.
2. Different results depending on the tool used: Although the Thomson Reuters tool has the advantage of precedent, different bibliometric results for the same corpus of journals obtained with competing tools (notably Scopus, but also Google Scholar), raise suspicion on this type of evaluation (Maya Bacache-Beauvallet, 2010).
3. The alleged value of the article derives from that of the journal: the impact factor is used as an indicator of the journal quality. Everybody’s goal is therefore to publish in high impact factor journals, so that the article itself is cited often. The value assigned to the articles in this case is no longer intrinsic, but depends on the value given to the journal based on its impact factor.
4. Use factor: articles are not only intended to be cited in other publications. In some fields, reading the article permits a practical use (in the field of medicine and diagnosis for example). An article covering the “state of the art" for a topic of interest to many researchers, or the description of an analysis method that can be used in many laboratories will be more cited than an innovative article in a very specialized research area. Calculating the value of an article based on the number of citations obtained by the journal in which it is published only provides a partial picture of its value.
5. Rhythms of dissemination of a journal: the limitation to the past two years has been criticized: at the end of this time, an article in Nature will have a peak of citations, but an article in another journal with less rapid dissemination will peak only after 3 or 4 years. There are also similar disparities between disciplines: articles can be cited as many times, although over a longer period, in a social sciences journal as in a life sciences journal.
6. Snowball effect: bibliometric tools encourage a "snowball effect": rapidly, a small number of journals attract a considerable number of citations, while high-quality journals, with fewer citations, fall into oblivion. For editors, this means that it is vital to maintain their journal among the 20% that receive 80% of the citations. Some journals even resort to disreputable practices, such as inflation of self-citations or switching of citations.
7. Number of hits on social networks: Today, it is an oversimplification to calculate the value of an item based only on its impact factor. The influence of an article can also be measured by the number of times it is commented or quoted on social networks.
8. Impact is money: The impact factor has become an important commercial argument in the hands of publishers who use them to justify the very high prices of their subscriptions. The producers of major bibliographic databases also exploit the addition of bibliometric indicators and rankings to sell services to research institutions at a higher price.

Bertignac, Catherine, Fouquet, Josiane, Marie, Emilie. "Bibliometrics". In UBL (European University of Brittany). Formadoct. Rennes: UBL, 2010 (last updated 11 December 2013). Available at: http://guides-formadoct.u-bretagneloire.fr/bibliometrics