Papers
arxiv:2604.15823

Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions

Published on Apr 17
Authors:
,
,
,
,
,

Abstract

Researchers created the first benchmark dataset for egocentric screen-view movie emotion understanding and developed a multimodal long-context emotion reasoning framework to address ___domain gaps between cinematic footage and real-world viewing scenarios.

Embodied robotic agents often perceive movies through an egocentric screen-view interface rather than native cinematic footage, introducing ___domain shifts such as viewpoint distortion, scale variation, illumination changes, and environmental interference. However, existing research on movie emotion understanding is almost exclusively conducted on cinematic footage, limiting cross-___domain generalization to real-world viewing scenarios. To bridge this gap, we introduce EgoScreen-Emotion (ESE), the first benchmark dataset for egocentric screen-view movie emotion understanding. ESE contains 224 movie trailers captured under controlled egocentric screen-view conditions, producing 28,667 temporally aligned key-frames annotated by multiple raters with a confidence-aware multi-label protocol to address emotional ambiguity. We further build a multimodal long-context emotion reasoning framework that models temporal visual evidence, narrative summaries, compressed historical context, and audio cues. Cross-___domain experiments reveal a severe ___domain gap: models trained on cinematic footage drop from 27.99 to 16.69 Macro-F1 when evaluated on realistic egocentric screen-view observations. Training on ESE substantially improves robustness under realistic viewing conditions. Our approach achieves competitive performance compared with strong closed-source multimodal models, highlighting the importance of ___domain-specific data and long-context multimodal reasoning.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2604.15823
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2604.15823 in a model README.md to link it from this page.

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2604.15823 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.