Aller au contenu principal

Bibliometrics: Other indicators

What is bibliometrics ? What are indicators ? Where to find them ? Why are they important for doctoral students ?

The number of publications

The evaluation reports for research units often require that publications be counted and ranked. Depending on the discipline and type of evaluation, these rankings may differ. For example, for 2010-2013, the AERES defined a count and ranking according to the following headings:

ACL: Articles in international or national, peer-reviewed or refereed, journals listed in international databases (ISI Web of Knowledge, Google Scholar, Harzing Publish or Perish, Pub Med...). The journal impact factor may be indicated, specifying the data source used.

ACLN: Articles in refereed journals not listed in the international databases.

ASCL: Articles in non peer-reviewed journals.
INV: Presentation at the invitation of the Organizing Committee in a national or international conference.

ACT: Communications with acts in an international or national conference

COM: Oral communications without acts in an international or national conference

AFF: Communications by poster in a national or international conference

OS: Scientific books (or chapters of these books)

OV: Books for the general public (or chapters of these books)

DO: Book editor

AP: Other publications

TH: Submitted theses

Exploitation and patents: Specify research contracts and industrial partnerships, and complete the "Exploitation" table of the form.

Other modes of presentation may be requested:

  • Number of publications in journals with impact factor greater than X
  • Number of publications in journals included in recognized lists (e.g. AERES list of  SHS journals)
  • Number of publications as first author...

The number of citations

The number of citations is a widely used indicator in the "hard sciences". A publication, once published, will receive x citations, i.e. it will be integrated into the final bibliography of x other publications. The count of these citations can give rise to various indicators:

  • the total number of citations of articles: the citations received by all your publications is summed.
  • the average number of citations per article (citation rate): the total number of citations received by your publications is divided by the total number of your publications.

How do you know when a publication has been cited x times? 
Generally, the institutional evaluations are based on data from the Web of Science or Scopus because the database is restricted to a set of controlled publications and there are no duplicates.

The data may also be collected from an advanced search via Google Scholar, but in this case it is necessary to ensure that there are no duplicates or triplets among the citing references, and that these references are really scientific publications. In any case, the source used to calculate your citations should be specified.

Specific indices for institutions or countries

The evaluation of research in an institution or in a country requires specific indicators, including:

  • The fraction of publications for the entity with respect to a larger set: for example, the fraction of French publications in the EU countries measures the number of publications for France divided by the number of publications for Europe, times 100.
  • The scientific specialization index, which measures the fraction of publications for the entity in a field, compared to its overall contribution: for example the fraction of French publications in mathematics in the EU, compared to the fraction of France publications in the EU.
  • The fraction of citations over two years, for example for an institution in France, measures for a given year N, the number of citations received in the years N and N+1, divided by the total number of citations for French institutions in year N and N+1, times 100.
  • The two-year relative impact index, for example for an institution in France, measures over 2 years, the fraction of citations received by this institution divided by the fraction of the publications for this institution at the French level

Source: http://urfist.enc.sorbonne.fr/sites/default/files/mdb/OSTsynthIndices.pdf

The h-index

The h-index  (or Hirsch index) aims to quantify the scientific productivity and impact of researchers, or research units, depending on the citation level of their publications.

The index was proposed in 2005 by Jorge Hirsch as a tool for determining a relative quality of theoretical physicists, although the measure can be extended to all researchers publishing in journals with peer review.

To determine the h-index, the publications are ranked in descending order of citations. An h-index of n means that the author has at least n publications cited n times.

 

Rang des publications

Chercheur 1

Chercheur 2

Chercheur 3

1

12

30

59

2

11

14

1

3

9

5

0

4

8

4

0

5

8

3

0

6

7

2

0

7

5

2

0

h index

6

4

1

For example, the researcher 1 has at least 6 publications cited 6 times, but the 7th publication is not cited 7 times. Therefore, the h-index is 6. It will be 7 when the seventh publication is cited at least 7 times.

The researcher 3 who has published a single highly cited article (which possibly revolutionizes the discipline...) will have a low index.

The h-index is calculated by the Web of Science, Scopus, Google Scholar...

The g-index

The g-index is a variant of the h-index. It is always higher. It corresponds to the highest rank, at which the first g publications accumulate a number of citations of at least g².

Example: 
With the 18 most cited publications, a researcher has accumulated 369 citations. The number of citations is thus at least 18² (= 324) for this rank
With the 19 most cited publications, a researcher has accumulated 381 citations. The number of citations is thus at least 19²  (= 381) for this rank
With the 20 most cited publications, a researcher has accumulated 392 citations. The number of citations is no longer at least 20² (=400) for this rank
The g-index for this researcher is therefore 19.

The g factor was developed in 2006 by Prof. Leo Egghe, chief librarian at the University of Hasselt in Belgium and editor-in-chief of Journal of Informetrics.

The n-index

The h-index of an author is obviously lower for a young researcher who has published few articles. The n-index was created to take into account the number of years of experience. It is calculated by dividing the h-index by the number of years since the first publication.

A researcher who has an h-index of 12 and who published his/her first article 10 years ago, will have a n-index of 1.2.

A young researcher who has an h-index of 4 and who published his/her first article 2 years ago, will have a n-index of 2

Network analysis

Network analysis and data exploration go beyond ad hoc and quantitative indicators, and represent a growing portion of current bibliometric studies. It involves mapping the occurrences and the relationships existing between the various elements of the publications (words, words in the title, keywords, authors, research units, citations...). 

A few examples of studies:

  • A mapping of the positioning of research units by analysis of words extracted from the metadata for publications, including patents
  • A mapping of co-publications in a domain to better understand the collaborations between research units
  • An analysis of innovative actors or emerging topics in a field by studying the appearance of keywords
  • An identification of experts (authors, inventors) by fields, combined with geographical information...

The tools: Intellixir, Orbit, Thomson Innovation, Minolta.

The rise of altmetrics

The 4 ways to measure the impact of an article, from the Altmetrics manifesto:

Its usage, peer-review, citation rate in bibliometric tools... and altmetrics.

 

Traditional bibliometric indicators only measure the number of citations of an article or a journal in the scientific press. This limitation means that the indicators cannot take into account the impact that the article has in the blogosphere, the twittosphere or bibliographic management software. A new wave of indicators (Altmetrics) tries to measure the influence of each article on the social web.

The altmetrics count

  1. the number of views or downloads in the warehouse where the item is deposited (PLOS One for example)
  2. the number of captures by users in bookmarking tools (Diigo) or reference managers (Mendeley)
  3. The number of references/citations on the social web:
  • Twitter
  • Specialized social networks like ResearchGate or Vivo
  • The blogosphere (for example, blogs hosted by Google)

and, of course, the number of citations in Google Scholar and Wikipedia.

The image below shows figures concerning the usage of an article published in PLOS ONE but also mentions that two users of Mendeley have referenced it in their collection.

below: measure of the impact of an article on the social web by Altmetrics.org

Altmetrics schema