Krzysztof Cipora
When a measure becomes a goal — measuring excellence without gaming it
Cipora starts from the idea that a measure which becomes a goal stops being a proper measure: every time evaluators introduce a new metric, people start hacking or gaming it to their advantage, and the measure loses its grounding. Measuring academic excellence is therefore intrinsically difficult and must be considerate and thorough, covering the several missions academia is meant to fulfil. A crucial distinction is between evaluating institutions and evaluating individual researchers — expecting every single academic to be well-rounded across all missions is far less reasonable than expecting it of the institution, so evaluation has to balance specialization against the institution’s universality. Metrics should be chosen to be as robust as possible against exploitation; rigidly sticking to one metric such as the impact factor — now maximally gamed by journals inflating it — defeats the measure’s original purpose.
A measure which becomes a goal stops being a proper measure.