Science Performance and Research Funding

The Minister of Industry, on behalf of the Natural Sciences and Engineering Research Council (NSERC), asked the Council of Canadian Academies to examine the international practices and supporting evidence used to assess performance of research in the natural sciences and engineering disciplines.

Progress Report

This assessment recently underwent the formal peer review process. The Expert Panel on Science Performance and Research Funding held its fourth and final meeting from December 1-2, 2011 in Toronto to address reviewers’ comments and to finalize the content. The report is expected to be released in 2012.

Background

There is no straightforward approach for measuring outputs of the research generated from the natural sciences and engineering disciplines. Policy makers and researchers alike are interested in developing a better understanding of the different approaches used to assess science performance, and to look beyond basic economic outcomes. 

Lack of a “one-size-fits all” solution to monitoring and measuring research performance is driving a growing international need to develop and implement new methods and, share best practices and lessons learned.  The Council’s assessment will draw from both international and national approaches for evaluating research performance.  Overall, the assessment presents an important opportunity to address science performance from a uniquely Canadian perspective.

Question

What do the scientific evidence and the approaches used by other funding agencies globally have to offer, in terms of performance indicators and related best practices in the context of research in the natural sciences and engineering, carried out at universities, colleges, and polytechnics?

More precisely:

1. What existing qualitative and quantitative indicators and metrics are relevant to budget allocation in the context of support for research in the natural sciences and engineering, and how can they be categorized (e.g. shelf life; cross-disciplinary and international comparability; relevance to interdisciplinary vs. focused disciplinary areas; and applicability to emerging vs. established research areas)?

In this context, quantitative indicators include researcher population dynamics and cost-based indices, while qualitative indicators include collective quality/excellence of research and areas of strength indices.

2. What are international best practices in the construction, methodological review, and use of quantitative and qualitative indicators for research evaluation and budget allocation in support of research in the natural sciences and engineering?

3. Considering the foregoing, and in light of the Government of Canada’s Science and Technology Strategy and NSERC’s objectives for the support of research, what key considerations (e.g., risks, advantages/disadvantages, behavioural and institutional consequences) and principles emerge in determining defensible use and balance/ weighting of performance indicators/metrics for budget allocation?

Expert Panel

The Expert Panel on Science Performance and Research Funding is chaired by Dr. Rita Colwell, Distinguished University Professor at the University of Maryland. For a complete listing of panel members, please visit the Expert Panel on Science Performance and Research Funding Membership page.

For further information, please contact:

Christina Stachulak, Program Director, at 613-567-5000 ext. 259 or christina.stachulak@scienceadvice.ca.