3.3.4 Peer review methodology For a range of disciplines, such as humanities, social sciences, and disciplines at the applied end of the spectrum, citation analysis may not be appropriate—either because these disciplines do not predominantly disseminate their research findings through academic journals, or because the citation data for the journals for these disciplines is not available. If the research output of the discipline is not predominantly made up of journal articles, then citation analysis would only give a partial view of the research activity and would not support an accurate evaluation of the research quality. The research outputs available for peer review through ERA evaluations include the traditional range of academic outputs such as journal articles, books, book chapters, and conference publications. ERA evaluations also include a range of non-traditional research outputs for some disciplines such as original creative works, live performance of creative works, recorded/rendered creative works, curated or produced substantial public exhibitions and events, and research reports for an external body. In ERA, a peer review sample of 30 per cent of research outputs is evaluated by committees of internationally recognised experts, and additional peer reviewers. The sample is nominated by the university. As with disciplines that use the citation analysis methodology, there must be a sufficient volume of research outputs within a unit of evaluation to ensure that the evaluation is robust. A detailed explanation of the peer review methodology located in Section 5.6 of the ERA 2018 Evaluation Handbook .
Issues to be explored
Q3.15
The peer review methodology for evaluating the quality of research is appropriate. Strongly agree; Agree; Neither agree nor disagree; Disagree; Strongly disagree. Please explain your answer.
Q3.16
What are the strengths of the peer review methodology? Please describe.
Q3.17
What are the weaknesses of the peer review methodology? Please describe.
Q3.18
Can the peer review methodology be modified to improve the evaluation process while still adhering to the ERA Indicator Principles? Yes/No. a. If you answered ‘Yes’, please describe how the peer review methodology could be improved.
3.3.5 Contextual indicators
Apart from the key quality indicators, ERA also includes a suite of contextual, or supporting, indicators. These are:
volume and activity
•
publishing profile
•
research income
•
applied measures.
•
For the most part, the contextual indicators are designed to provide expert evaluators with a deeper level of understanding about the unit of evaluation they are assessing, and their presence or
14
Made with FlippingBook Proposal Creator