Absolute and specific measures of research group excellence

O. Mryglod, Ralph Kenna, Y. Holovatch, B. Berche

    Research output: Contribution to journalArticle

    13 Citations (Scopus)

    Abstract

    A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly funded research institutions and universities to serve as a basis for their ranking and financing. While citation-based indicators and metrics are easily accessible, they are far from being universally accepted as way to automate or inform evaluation processes or to replace evaluations based on peer review. Here we consider absolute measurements of research excellence at an amalgamated, institutional level and specific measures of research excellence as performance per head. Using biology research institutions in the UK as a test case, we examine the correlations between peer review-based and citation-based measures of research excellence on these two scales. We find that citation-based indicators are very highly correlated with peer-evaluated measures of group strength, but are poorly correlated with group quality. Thus, and almost paradoxically, our analysis indicates that citation counts could possibly form a basis for deciding on, how to fund research institutions, but they should not be used as a basis for ranking them in terms of quality.
    Original languageEnglish
    Pages (from-to)115-127
    JournalScientometrics
    Volume95
    Issue number1
    DOIs
    Publication statusPublished - 2013

    Fingerprint

    Group
    peer review
    ranking
    evaluation
    biology
    university
    management
    performance

    Bibliographical note

    The full text is available free from the link given. The published version can be found at http://dx.doi.org/10.1007/s11192-012-0874-7.
    The final publication is available at www.springerlink.com.

    Keywords

    • higher education
    • scientific evaluation
    • scientometrics

    Cite this

    Absolute and specific measures of research group excellence. / Mryglod, O.; Kenna, Ralph; Holovatch, Y.; Berche, B.

    In: Scientometrics, Vol. 95, No. 1, 2013, p. 115-127.

    Research output: Contribution to journalArticle

    Mryglod, O. ; Kenna, Ralph ; Holovatch, Y. ; Berche, B. / Absolute and specific measures of research group excellence. In: Scientometrics. 2013 ; Vol. 95, No. 1. pp. 115-127.
    @article{f45fbfa170ab45479b90d6973d5fb69a,
    title = "Absolute and specific measures of research group excellence",
    abstract = "A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly funded research institutions and universities to serve as a basis for their ranking and financing. While citation-based indicators and metrics are easily accessible, they are far from being universally accepted as way to automate or inform evaluation processes or to replace evaluations based on peer review. Here we consider absolute measurements of research excellence at an amalgamated, institutional level and specific measures of research excellence as performance per head. Using biology research institutions in the UK as a test case, we examine the correlations between peer review-based and citation-based measures of research excellence on these two scales. We find that citation-based indicators are very highly correlated with peer-evaluated measures of group strength, but are poorly correlated with group quality. Thus, and almost paradoxically, our analysis indicates that citation counts could possibly form a basis for deciding on, how to fund research institutions, but they should not be used as a basis for ranking them in terms of quality.",
    keywords = "higher education, scientific evaluation, scientometrics",
    author = "O. Mryglod and Ralph Kenna and Y. Holovatch and B. Berche",
    note = "The full text is available free from the link given. The published version can be found at http://dx.doi.org/10.1007/s11192-012-0874-7. The final publication is available at www.springerlink.com.",
    year = "2013",
    doi = "10.1007/s11192-012-0874-7",
    language = "English",
    volume = "95",
    pages = "115--127",
    journal = "Scientometrics",
    issn = "0138-9130",
    publisher = "Springer Verlag",
    number = "1",

    }

    TY - JOUR

    T1 - Absolute and specific measures of research group excellence

    AU - Mryglod, O.

    AU - Kenna, Ralph

    AU - Holovatch, Y.

    AU - Berche, B.

    N1 - The full text is available free from the link given. The published version can be found at http://dx.doi.org/10.1007/s11192-012-0874-7. The final publication is available at www.springerlink.com.

    PY - 2013

    Y1 - 2013

    N2 - A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly funded research institutions and universities to serve as a basis for their ranking and financing. While citation-based indicators and metrics are easily accessible, they are far from being universally accepted as way to automate or inform evaluation processes or to replace evaluations based on peer review. Here we consider absolute measurements of research excellence at an amalgamated, institutional level and specific measures of research excellence as performance per head. Using biology research institutions in the UK as a test case, we examine the correlations between peer review-based and citation-based measures of research excellence on these two scales. We find that citation-based indicators are very highly correlated with peer-evaluated measures of group strength, but are poorly correlated with group quality. Thus, and almost paradoxically, our analysis indicates that citation counts could possibly form a basis for deciding on, how to fund research institutions, but they should not be used as a basis for ranking them in terms of quality.

    AB - A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly funded research institutions and universities to serve as a basis for their ranking and financing. While citation-based indicators and metrics are easily accessible, they are far from being universally accepted as way to automate or inform evaluation processes or to replace evaluations based on peer review. Here we consider absolute measurements of research excellence at an amalgamated, institutional level and specific measures of research excellence as performance per head. Using biology research institutions in the UK as a test case, we examine the correlations between peer review-based and citation-based measures of research excellence on these two scales. We find that citation-based indicators are very highly correlated with peer-evaluated measures of group strength, but are poorly correlated with group quality. Thus, and almost paradoxically, our analysis indicates that citation counts could possibly form a basis for deciding on, how to fund research institutions, but they should not be used as a basis for ranking them in terms of quality.

    KW - higher education

    KW - scientific evaluation

    KW - scientometrics

    U2 - 10.1007/s11192-012-0874-7

    DO - 10.1007/s11192-012-0874-7

    M3 - Article

    VL - 95

    SP - 115

    EP - 127

    JO - Scientometrics

    JF - Scientometrics

    SN - 0138-9130

    IS - 1

    ER -