The GenAI4IDN Benchmark 2.0 - AI-Assisted Assessment of Generative AI Tools for the Design of Interactive Digital Narratives
2026 (English)In: Interactive Storytelling: 18th International Conference on Interactive Digital Storytelling, ICIDS 2025, Saint Julian, Malta, December 1–5, 2025, Proceedings, Part I / [ed] Reyes, M.C.; Nack, F., Cham: Springer, 2026, p. 254-279Conference paper, Published paper (Refereed)
Abstract [en]
Where do generative AI (GenAI) tools stand in 2025 when it comes to the design of Interactive Digital Narratives (IDNs)? In 2024, we introduced a first effort in benchmarking GenAI tools for IDN design [14]. This paper updates the findings for the current generation of previously evaluated tools (ChatGPT, Claude, Gemini, Copilot), and also adds several open source entries such as DeepSeek, Llama, Mistral, and Qwen. We introduce two key additions to the GenAI4IDN benchmarking framework: methods to benchmark iterative prompting and the effects of contextualized prompting. Furthermore, we introduce and apply a method for AI-assisted evaluation, facilitating rapid evaluation of large amounts of IDN design artifacts with a level of quality similar to human raters.
Place, publisher, year, edition, pages
Cham: Springer, 2026. p. 254-279
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 16374
Keywords [en]
AI-assisted evaluation, contextualized prompting, GenAI tools, IDN benchmarking, interactive digital narrative design, interactive storytelling theory and design, iterative prompting, method, Benchmarking, Design, Human computer interaction, Generative AI tool, Interactive digital narrative benchmarking, Interactive digital narratives, Interactive storytelling, Narrative designs, Theory and designs, Iterative methods
National Category
Information Systems
Identifiers
URN: urn:nbn:se:sh:diva-58808DOI: 10.1007/978-3-032-12408-1_14Scopus ID: 2-s2.0-105023835721ISBN: 978-3-032-12407-4 (print)ISBN: 978-3-032-12408-1 (electronic)OAI: oai:DiVA.org:sh-58808DiVA, id: diva2:2027006
Conference
ICIDS 2025, Saint Julian, Malta, December 1–5, 2025.
2026-01-122026-01-122026-01-12Bibliographically approved