Citation analysis is used for disciplines in which research findings are predominantly disseminated through academic journals and there are sufficient outputs in indexed peer-reviewed journals to allow robust citation analysis. For a range of disciplines, such as humanities, social sciences, information sciences and disciplines at the applied end of the spectrum, citation analysis may not be appropriate—either because these disciplines do not predominantly disseminate their research findings through academic journals, or the citations information for the journals for these disciplines is not available. Many of the disciplines disseminate their research findings through other types of outlets, such as books, conferences, reports, creative works, exhibitions and performances. Therefore, in these disciplines, peer review of a 30% sample of outputs across all output types is the indicator used. In ERA, a sample of research outputs is evaluated by committees of internationally recognised experts, and additional peer reviewers. For ERA, the ARC identified disciplines suitable for citation analysis through consultation with discipline peak bodies. There are also four additional categories of contextual indicators which assist evaluators to understand each unit of evaluation:
volume and activity
•
publishing profile
•
research income
•
applied measures
•
For more information on the application of specific indicators to individual disciplines, refer to the ERA 2018 Discipline Matrix. Further details regarding the citation and peer review methodologies are provided in the following sections.
Issues to be explored
Q3.7
The current methodology meets the objectives of ERA. Strongly agree; Agree; Neither agree or disagree; Disagree; Strongly disagree. Please explain your answer.
Q3.8
What are the strengths of the overall methodology? Please describe.
Q3.9
What are the weaknesses of the overall methodology? Please describe.
Q3.10
Does the discipline-specific approach for evaluating research quality (citation analysis or peer review for specific disciplines) continue to enable robust and comparable evaluation across all disciplines?
3.3.3 Citation analysis methodology The most basic and common measure of research activity is the number of peer-reviewed journal publications. Tracking the number of citations to these publications can reveal trends in the impact and influence of the research. While analysis of citation metrics is a key indicator for some disciplines in ERA, expert review of the indicators by the research evaluation committees is fundamental to the methodology. The analysis of citation metrics is considered by the Research Evaluation Committees and it is the committees that decide the ratings.
12
Made with FlippingBook Proposal Creator