Palfinger, Thomas and Gaisbauer, Felix and Wagner, Isabella and Beck, Susanne (2024) Working Paper Draft: How does artificial intelligence impact the evaluation system? Discussion points for designing tomorrow's evaluation system. Technical Report. Austrian Platform for Research and Technology Policy Evaluation. Vienna. (Submitted)
Preview |
Text (Working Paper Draft)
AI_Paper_EN.pdf Download (506kB) | Preview |
Abstract (english)
***DISCLAIMER This is a translated draft of a working paper. The original German version was submitted as a contribution to the fteval Journal issue #55 and is currently under review. The translation was done with the support of Chat GPT 4.o and has not received a final editorial check. If the article is accepted, it will be published in its final carefully edited English and German versions in the fteval repository here: https://repository.fteval.at/id/eprint/707/ *** In this discussion paper, initial considerations are made on possible changes through artificial intelligence (AI) in an evaluation system using the example of research and innovation policy evaluation. Previous discourse on artificial intelligence (AI) has focussed primarily on data protection, ethics and scientific and methodological reliability. This discussion paper aims to introduce an additional, systemic perspective: It examines how the relationships between actors in an evaluation system change when this system is confronted with generative AI. A working group of the Austrian Platform for Research and Technology Policy Evaluation (fteval) addressed this question from July 2023 to March 2024. Based on internal discussions and accompanying literature research, a model was developed that serves as an instrument for reflecting on and jointly discussing the practical actions of the actors in the RTI evaluation system. The starting points are the assumptions that generative AI is entering the existing evaluation system as a disruptive element and that these tools can change the relationships between the actors. Although the focus is on the Austrian evaluation system, it is assumed that the model could also be useful in other evaluation systems or industries with similar structures.
| Subjects: | Studies on Evaluation |
|---|---|
| Divisions: | AIT - Austrian Institute of Technology GmbH AQ - Agentur für Qualitätssicherung und Akkreditierung Austria Ludwig Boltzmann Gesellschaft Platform non-member WWTF - Wiener Wissenschafts-, Forschungs- und Technologiefonds fteval - Austrian Platform for Research and Technology Policy Evaluation |
| Number of Pages: | 18 |
| Identification Number: | forthcoming |
| Date Deposited: | 01 Aug 2024 06:44 |
| URI: | https://repository.fteval.at/id/eprint/707 |
Actions (login required)
 |
View Item |
