Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions
Abstract
Researchers created the first benchmark dataset for egocentric screen-view movie emotion understanding and developed a multimodal long-context emotion reasoning framework to address ___domain gaps between cinematic footage and real-world viewing scenarios.
Embodied robotic agents often perceive movies through an egocentric screen-view interface rather than native cinematic footage, introducing ___domain shifts such as viewpoint distortion, scale variation, illumination changes, and environmental interference. However, existing research on movie emotion understanding is almost exclusively conducted on cinematic footage, limiting cross-___domain generalization to real-world viewing scenarios. To bridge this gap, we introduce EgoScreen-Emotion (ESE), the first benchmark dataset for egocentric screen-view movie emotion understanding. ESE contains 224 movie trailers captured under controlled egocentric screen-view conditions, producing 28,667 temporally aligned key-frames annotated by multiple raters with a confidence-aware multi-label protocol to address emotional ambiguity. We further build a multimodal long-context emotion reasoning framework that models temporal visual evidence, narrative summaries, compressed historical context, and audio cues. Cross-___domain experiments reveal a severe ___domain gap: models trained on cinematic footage drop from 27.99 to 16.69 Macro-F1 when evaluated on realistic egocentric screen-view observations. Training on ESE substantially improves robustness under realistic viewing conditions. Our approach achieves competitive performance compared with strong closed-source multimodal models, highlighting the importance of ___domain-specific data and long-context multimodal reasoning.
Get this paper in your agent:
hf papers read 2604.15823 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper