Performance Indicators
A Multidimensional Framework for Research Performance
The AHSC project develops a comprehensive, multidimensional framework to assess the research performance of Italian Academic Health Science Centres. By combining data from OpenAlex, Dimensions, PubMed, and Altmetric, we define and compute a rich set of indicators across five key dimensions: bibliometrics, social impact, industrial innovation, clinical research, and funding capacity.
All scientometric measures are normalized considering various aspects such as AHSC size, research output types, and subject sub-categories, enabling fair and meaningful comparisons across institutions of different scale and disciplinary focus.
The analytical backbone of the project is the bibliometrix R-package — a comprehensive, open-source tool for quantitative research in bibliometrics and scientometrics. For this project, bibliometrix has been upgraded to accommodate all types of scientometric data, creating a new analytical platform called Scientometrix that allows analyzing research outputs in a combined, multidimensional fashion.
Bibliometric Dimension
Productivity
Research productivity captures the volume and intensity of scientific output. We calculate it as the ratio between AHSC publications and the total number of affiliated authors — a necessary normalization step for comparing AHSCs of different sizes.
Key indicators include:
- Scientific productivity: Publications per internal author (fractional counting)
- Publication growth rate: Annual percentage of growth or decrease in publications over the considered timespan
- Authorship patterns: Share of internal authors in first, last, and corresponding positions
- Open Access share: Proportion of publications available in Open Access
- Single-authored articles: Number of articles produced independently
These indicators reflect both institutional research engagement and the level of independent scholarship.
Impact
Impact indicators assess the influence and quality of research output through citations and journal prestige.
- Research Impact (RI): Total citations (TC) associated with the i-th AHSC, normalized by the total number of published papers (M): RI = TC / M
- Field-weighted citation scores: Normalized citation metrics accounting for disciplinary differences
- Top journal share: Percentage of publications in the top 10% journals
- Citation trends: Temporal evolution of citation patterns over the observation period
The impact factor remains dominant in research evaluation, but we complement it with field-normalized measures to account for disciplinary citation practices.
Disruption and Novelty
These recent metrics capture the innovative character of research output:
- Disruption index: Measures whether a publication introduces a new direction (disruptive) or consolidates existing knowledge (consolidating), based on atypical combinations of journals, references, keywords, and topics
- Novelty/Conventionality index: Assesses the degree to which a publication combines established and novel ideas
- Interdisciplinarity indexes: Quantify the diversity of disciplines or journals reflected in a publication’s references and citation relationships, using the Rao-Stirling diversity index, the Gini coefficient, and entropy measures
These indicators identify AHSCs that are at the frontier of scientific innovation versus those focused on incremental research.
Collaboration
Collaboration indicators reveal the extent of networking, interdisciplinary effort, and participation in multi-center research:
- International co-authorship rate: Percentage of articles with authors from foreign institutions
- Cross-institutional affiliations: Partnerships across different AHSCs and external research centers
- Average number of authors per article: Proxy for team-based research culture
Thematic Orientation
Thematic analysis characterizes the scientific specialization of each AHSC:
- MeSH terms and primary research topics: Standardized classification of clinical and biomedical research areas
- Concept mapping: Hierarchical topic classification from OpenAlex
- Bigrams from titles and abstracts: Text-mining of publication content
- Network similarity: Comparison of research profiles across AHSCs using co-occurrence and coupling techniques
These approaches support science mapping — identifying thematic convergence, divergence, and emerging research fronts across institutions.
Talent Distribution
Talent indicators assess the internal distribution and concentration of research activity:
- Gini coefficient: Computed on both equal and fractional publication counts, measuring inequality in research productivity among affiliated researchers
- Top contributors: Identification of high-performing researchers based on volume, impact, and leadership roles
These metrics shed light on whether scientific output is concentrated among few prolific researchers or distributed more broadly within the institution.
Industrial Impact and Research Innovation
Patent metadata measure the capacity for innovation and technology transfer. Patent scientometrics is analogous to literature bibliometrics — it applies similar analytical techniques to identify emerging technologies and disruptive innovations.
Key indicators include:
- Number of patents registered per institution
- Technological domain classification: Distribution across medical devices, pharmaceuticals, biotechnology, and other fields
- Top inventors: Identification based on volume and relevance of patents filed
- Citation analysis: Citations associated with clinical trials and patent documents, revealing the industrial impact of academic research
Clinical Research Activity
Clinical trials represent a critical interface between scientific discovery and patient care. In the life sciences, they are research studies performed on people aimed at evaluating medical, surgical, or behavioral interventions — often used to determine if a new treatment is more effective and/or has fewer harmful side effects than the standard.
Key indicators include:
- Number of newly initiated clinical trials per AHSC
- Average funding amount per trial, as a proxy for resource allocation
- Distribution by trial phase (Phase I–IV), distinguishing early-stage discovery from late-stage translational research
- Collaboration metrics: Partnerships between AHSCs, healthcare providers, industry, and external research centers
- Citation impact: Citations received by clinical trial publications, measuring their influence on subsequent research
Funding and Grants
Grant metadata indicate an AHSC’s ability to attract funding and establish stable national and international collaborative networks. Research funding plays a central role in sustaining scientific activity and expanding institutional capabilities.
Key indicators include:
- Number of competitive research grants awarded to each AHSC
- Total funding amount obtained, from both national and international sources
- Funding source diversity: Distribution across national agencies, EU programmes, and private foundations
- Grant success rate: Proportion of submitted proposals that received funding
These metrics reflect both research capacity and competitiveness in an increasingly selective funding landscape.
Social and Digital Impact
Altmetrics provide a complementary perspective to traditional citation-based indicators, capturing the public and policy-level visibility of research. Altmetrics should not be seen as alternatives but as complementary to traditional measurements — they assess the immediate social impact of a research paper by measuring its digital circulation in real time.
Key indicators include:
Research shows a meaningful relationship between the altmetric data of an article and the citations it subsequently receives, reinforcing the value of social impact metrics as early signals of academic influence.