Why this matters Interactive narrative and in-game storymasters are among the first widespread use-cases that ask LLMs to both generate fluent text and maintain a persistent, rule-bound fictional world across many turns. This paper shows that high linguistic quality does not imply logical trustworthiness: models that write well still frequently break their own world rules when pushed by adversarial player inputs, which undermines usability for games, simulations, and persistent virtual characters.
Key Findings
- Concrete benchmark and evaluation protocol: defines Narrative Commitment Preservation (NCP) and releases NCP-Bench, 100 narrative environments derived from movie synopses. Each environment encodes a structured narrative specification (trajectory, commitments, initial facts) that can be automatically checked during play.
- Large consistency gap: across state-of-the-art LLMs, fluency and narrative quality fail to guarantee commitment preservation. The best-performing model in the study (GPT-5.2) achieved roughly a 42% survival rate after 20 turns; fact conflict rates across models ranged from about 40% to 68%.
- Long-horizon brittleness: even models with strong single-turn generation produce logically conflicting content under adversarial interventions, and only isolated runs met all achievement commitments within a 100-turn limit.
Who it's for + Tradeoffs
Great fit if you are a researcher or developer building narrative agents, interactive game GMs, or evaluation suites: the paper provides a repeatable benchmark, automated checks for narrative violations, and empirical baselines showing where current models fail. Look elsewhere if your primary interest is multimodal storytelling, open-ended creativity without explicit commitments, or user-facing usability studies — NCP-Bench focuses on logical commitment preservation under adversarial, rule-focused scenarios and does not by itself measure empathy, aesthetic quality, or multimodal grounding.
Where it fits
This work sits between long-form story-consistency research and applied agent evaluation: it complements benchmarks that measure fluency or user preference by targeting logical integrity and rule-following in interactive, multi-turn settings. The dataset and evaluation protocol are directly usable for stress-testing LLM agents intended to run persistent narratives in games and simulations.