Hartley Belmar, Ricardo and Schöpf, Veronika (2026) What CoARA Action Plans Commit To. Working Paper. fteval Preprints. (Submitted)
Preview |
Text (PDF file)
5903_preprint.pdf Download (885kB) | Preview |
|
Text (Word document file)
5903_preprint.docx Download (3MB) |
|
Preview |
Image (Figure 1)
figures-Figure 1 - procedure.png Download (354kB) | Preview |
Preview |
Image (Figure 2)
figures-Figure 2 - uptake by commitment.png Download (285kB) | Preview |
|
Spreadsheet (Data spreadsheet)
S1-actions.csv - Supplemental Material Download (38kB) |
Abstract (english)
The Agreement on Reforming Research Assessment asks each signatory to share an action plan. We analysed the 335 CoARA action plans that a search of Zenodo returned up to 6 August 2026. The reading is descriptive: it records which actions each plan commits to and whether each is in place or planned. Language-model agents followed a documented procedure: one extracted the actions a plan commits to, each with a verbatim quote; a second matched them to the 62-action catalogue of an open planning tool; a third tried to refute every record; a fourth settled the disputed matches; a script checked every quote against the plan text. The coding rules were set after 50 records of an earlier reading of the same plans were coded by hand. In all, 8,870 records passed the checks of quote, commitment and wording, and are counted. The most common action is to redesign evaluation rubrics (64.8% of the plans). Within the commitment to abandon inappropriate uses of journal- and publication-based metrics, the median action is taken by 9.0% of the plans, the lowest of the ten commitments. Of these records, 67% are planned rather than in place, most of all under the commitment to communicate progress. Three items of DORA, the Leiden Manifesto and SCOPE are linked only to actions that fewer than one plan in ten takes, and where keyword counts and reading can be compared, they disagree in both directions: the keywords find more plans for seven of 22 actions and fewer for 15. No sample coded by people measures this reading’s accuracy yet; we report its internal checks and a second run on 30 plans. The catalogue and the counts are built into the Research Assessment Reform Planner, an open tool in which an institution assesses itself against the ten commitments and drafts an editable plan from the actions it proposes, each shown with the share of plans that take it.
| Subjects: | fteval Preprints |
|---|---|
| Number of Pages: | 23 |
| Identification Number: | 10.22163/fteval.2026.771 |
| Date Deposited: | 06 Nov 2026 13:31 |
| URI: | https://repository.fteval.at/id/eprint/853 |
Actions (login required)
![]() |
View Item |


