Release status. The benchmark and the curation and evaluation code will be released here upon publication. Access to the video portion will be governed by a dataset Terms of Use; the original annotations will be released under CC BY-NC-SA 4.0.
Overview
Recent Large Multimodal Models handle video understanding well, yet reasoning about human-centered social situations remains hard to measure. Existing benchmarks rely on videos with a single observed trajectory, so it is difficult to tell whether a model understands social dynamics or is exploiting recurring narrative patterns.
SocialReasonBench is a video multiple-choice QA benchmark for socially grounded reasoning in scenarios derived from interactive narratives. It is built from gameplay videos of Detroit: Become Human, whose branching storylines make alternative social outcomes observable: because the alternative branch was actually played and recorded, counterfactual and causal questions can be graded against the game's own script, flowchart, and recorded branches rather than against annotator speculation.
A multi-agent curation pipeline localizes socially meaningful clips, grounds answer labels in game-state signals, and generates theory-guided questions with five types of diagnostic distractor. Every instance is verified by human auditors.
Citation
@inproceedings{huang2026socialreasonbench,
title = {SocialReasonBench: A Video-QA Benchmark for Social Reasoning
with Counterfactual Narrative Videos},
author = {Huang, Zheyu and Shi, Zijing and Luo, Haozhe and Tang, Huadong
and Liu, Mingyu and Fang, Meng and Chen, Ling},
booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
year = {2026}
}
Data use and attribution
The benchmark is built from publicly available gameplay videos of Detroit: Become Human, developed by Quantic Dream and published by Sony Interactive Entertainment. We do not modify the game, reverse engineer its software, or extract proprietary assets from the game files. All game-related audiovisual assets remain the property of their respective rights holders, and users are responsible for complying with applicable copyright law, platform terms, and licensing conditions.