The intersection of AI and RTI (Policy) evaluation: principles and considerations

This article explores the integration of Artificial Intelligence (AI) into Research and Technology Innovation (RTI) policy evaluation. It synthesizes findings from a working group established by the Austrian Platform for Research and Technology Policy Evaluation (fteval) to examine the ethical, technical, and practical implications of AI use in this context. The document focuses on defining AI's role, setting standards for its application, and addressing challenges related to transparency, fairness, accountability, and data privacy. It presents principles and considerations to guide evaluation practitioners and introduces potential use cases for AI systems in various evaluation phases. The article also includes a survey analysis of AI adoption within the Austrian RTI community, revealing insights into current practices, challenges, and the need for robust guidelines and training.