Big fish and small ponds: why the departmental h-index should not be used to rank universities

Olesya Mryglod; Yurij Holovatch; Ralph Kenna

doi:10.1007/s11192-022-04373-y

Big fish and small ponds: why the departmental h-index should not be used to rank universities

Olesya Mryglod, Yurij Holovatch, Ralph Kenna

Research output: Contribution to journal › Article › peer-review

5 Citations (Scopus)

57 Downloads (Pure)

Abstract

The size-dependent nature of the so-called group or departmental h-index is reconsidered in this paper. While the influence of unit size on such collective measures was already demonstrated a decade ago, institutional ratings based on this metric can still be found and still impact on the reputations and funding of many research institutions. The aim of this paper is to demonstrate the fallacy of this approach to collective research-quality assessment in a simple way, focusing on the h-index in its original form. We show that randomly reshuffling real scientometric data (varying numbers of citations) amongst institutions of varying size, while maintaining the volume of their research outputs, has little effect on their departmental h-index. This suggests that the relative position in ratings based on the collective h-index is determined not only by quality (impact) of particular research outputs but by their volume. Therefore, the application of the collective h-index in its original form is disputable as a basement for comparison at aggregated levels such as to research groups, institutions or journals. We suggest a possible remedy for this failing which is implementable in a manner that is as simple and understandable as the h-index itself.

Original language	English
Pages (from-to)	3279-3292
Number of pages	14
Journal	Scientometrics
Volume	127
Issue number	6
Early online date	27 Apr 2022
DOIs	https://doi.org/10.1007/s11192-022-04373-y
Publication status	Published - Jun 2022

Bibliographical note

Copyright © and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders.

This document is the author’s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it.

Funder

This work was supported in part by the National Academy of Sciences of Ukraine, Project KPKBK 6541230.

Keywords

Group h-index
Hirsch index
Size effects

ASJC Scopus subject areas

General Social Sciences
Computer Science Applications
Library and Information Sciences

Access to Document

10.1007/s11192-022-04373-y

Post-PrintAccepted author manuscript, 1.62 MB

Cite this

@article{fdcfdd6e30254835848fc78458b5b9d7,

title = "Big fish and small ponds: why the departmental h-index should not be used to rank universities",

abstract = "The size-dependent nature of the so-called group or departmental h-index is reconsidered in this paper. While the influence of unit size on such collective measures was already demonstrated a decade ago, institutional ratings based on this metric can still be found and still impact on the reputations and funding of many research institutions. The aim of this paper is to demonstrate the fallacy of this approach to collective research-quality assessment in a simple way, focusing on the h-index in its original form. We show that randomly reshuffling real scientometric data (varying numbers of citations) amongst institutions of varying size, while maintaining the volume of their research outputs, has little effect on their departmental h-index. This suggests that the relative position in ratings based on the collective h-index is determined not only by quality (impact) of particular research outputs but by their volume. Therefore, the application of the collective h-index in its original form is disputable as a basement for comparison at aggregated levels such as to research groups, institutions or journals. We suggest a possible remedy for this failing which is implementable in a manner that is as simple and understandable as the h-index itself.",

keywords = "Group h-index, Hirsch index, Size effects",

author = "Olesya Mryglod and Yurij Holovatch and Ralph Kenna",

note = "Copyright {\textcopyright} and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. This document is the author{\textquoteright}s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it. ",

year = "2022",

month = jun,

doi = "10.1007/s11192-022-04373-y",

language = "English",

volume = "127",

pages = "3279--3292",

journal = "Scientometrics",

issn = "0138-9130",

publisher = "Springer Verlag",

number = "6",

}

TY - JOUR

T1 - Big fish and small ponds

T2 - why the departmental h-index should not be used to rank universities

AU - Mryglod, Olesya

AU - Holovatch, Yurij

AU - Kenna, Ralph

N1 - Copyright © and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. This document is the author’s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it.

PY - 2022/6

Y1 - 2022/6

N2 - The size-dependent nature of the so-called group or departmental h-index is reconsidered in this paper. While the influence of unit size on such collective measures was already demonstrated a decade ago, institutional ratings based on this metric can still be found and still impact on the reputations and funding of many research institutions. The aim of this paper is to demonstrate the fallacy of this approach to collective research-quality assessment in a simple way, focusing on the h-index in its original form. We show that randomly reshuffling real scientometric data (varying numbers of citations) amongst institutions of varying size, while maintaining the volume of their research outputs, has little effect on their departmental h-index. This suggests that the relative position in ratings based on the collective h-index is determined not only by quality (impact) of particular research outputs but by their volume. Therefore, the application of the collective h-index in its original form is disputable as a basement for comparison at aggregated levels such as to research groups, institutions or journals. We suggest a possible remedy for this failing which is implementable in a manner that is as simple and understandable as the h-index itself.

AB - The size-dependent nature of the so-called group or departmental h-index is reconsidered in this paper. While the influence of unit size on such collective measures was already demonstrated a decade ago, institutional ratings based on this metric can still be found and still impact on the reputations and funding of many research institutions. The aim of this paper is to demonstrate the fallacy of this approach to collective research-quality assessment in a simple way, focusing on the h-index in its original form. We show that randomly reshuffling real scientometric data (varying numbers of citations) amongst institutions of varying size, while maintaining the volume of their research outputs, has little effect on their departmental h-index. This suggests that the relative position in ratings based on the collective h-index is determined not only by quality (impact) of particular research outputs but by their volume. Therefore, the application of the collective h-index in its original form is disputable as a basement for comparison at aggregated levels such as to research groups, institutions or journals. We suggest a possible remedy for this failing which is implementable in a manner that is as simple and understandable as the h-index itself.

KW - Group h-index

KW - Hirsch index

KW - Size effects

UR - http://www.scopus.com/inward/record.url?scp=85128857887&partnerID=8YFLogxK

U2 - 10.1007/s11192-022-04373-y

DO - 10.1007/s11192-022-04373-y

M3 - Article

SN - 0138-9130

VL - 127

SP - 3279

EP - 3292

JO - Scientometrics

JF - Scientometrics

IS - 6

ER -

Big fish and small ponds: why the departmental h-index should not be used to rank universities

Abstract

Bibliographical note

Funder

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Big fish and small ponds: why the departmental h-index should not be used to rank universities

Cite this