What CoARA Action Plans Commit To

Hartley Belmar, Ricardo and Schöpf, Veronika (2026) What CoARA Action Plans Commit To. Working Paper. fteval Preprints. (Submitted)

[thumbnail of PDF file]
Preview
Text (PDF file)
5903_preprint.pdf

Download (885kB) | Preview
[thumbnail of Word document file] Text (Word document file)
5903_preprint.docx

Download (3MB)
[thumbnail of Figure 1]
Preview
Image (Figure 1)
figures-Figure 1 - procedure.png

Download (354kB) | Preview
[thumbnail of Figure 2]
Preview
Image (Figure 2)
figures-Figure 2 - uptake by commitment.png

Download (285kB) | Preview
[thumbnail of Data spreadsheet] Spreadsheet (Data spreadsheet)
S1-actions.csv - Supplemental Material

Download (38kB)

Abstract (english)

The Agreement on Reforming Research Assessment asks each signatory to share an action plan. We analysed the 335 CoARA action plans that a search of Zenodo returned up to 6 August 2026. The reading is descriptive: it records which actions each plan commits to and whether each is in place or planned. Language-model agents followed a documented procedure: one extracted the actions a plan commits to, each with a verbatim quote; a second matched them to the 62-action catalogue of an open planning tool; a third tried to refute every record; a fourth settled the disputed matches; a script checked every quote against the plan text. The coding rules were set after 50 records of an earlier reading of the same plans were coded by hand. In all, 8,870 records passed the checks of quote, commitment and wording, and are counted. The most common action is to redesign evaluation rubrics (64.8% of the plans). Within the commitment to abandon inappropriate uses of journal- and publication-based metrics, the median action is taken by 9.0% of the plans, the lowest of the ten commitments. Of these records, 67% are planned rather than in place, most of all under the commitment to communicate progress. Three items of DORA, the Leiden Manifesto and SCOPE are linked only to actions that fewer than one plan in ten takes, and where keyword counts and reading can be compared, they disagree in both directions: the keywords find more plans for seven of 22 actions and fewer for 15. No sample coded by people measures this reading’s accuracy yet; we report its internal checks and a second run on 30 plans. The catalogue and the counts are built into the Research Assessment Reform Planner, an open tool in which an institution assesses itself against the ten commitments and drafts an editable plan from the actions it proposes, each shown with the share of plans that take it.

Subjects: fteval Preprints
Number of Pages: 23
Identification Number: 10.22163/fteval.2026.771
Date Deposited: 06 Nov 2026 13:31
URI: https://repository.fteval.at/id/eprint/853

Actions (login required)

View Item View Item