SE-Mem: Semantic-Episodic Memory for Scene-Consistent Generative View Synthesis

概要

Generative view synthesis aims to synthesize novel-view videos from reference observations along user-defined camera trajectories. Maintaining global scene consistency throughout this process remains challenging for camera-conditioned interactive video generators. Existing approaches typically select reference frames based solely on camera poses, overlooking whether the content remains semantically relevant or even visible. To address these limitations, we draw inspiration from semantic and episodic memory in human cognition, in which semantic memory captures generalized knowledge across experiences, whereas episodic memory preserves information tied to specific past events and contexts. Motivated by the complementary roles of semantic and episodic memory, we introduce SE-Mem, a Semantic-Episodic Memory framework that organizes historical observations in a structured 3D representation and augments its elements with semantic importance scores. A two-stage memory retrieval strategy is designed to identify the most informative frames to provide contextual guidance. Taking into account both semantic significance and geometric visibility, SE-Mem preserves semantic coherence and fine-grained visual details during novel view synthesis. Extensive evaluations show that our approach improves visual quality and achieves competitive geometric consistency across multiple benchmarks.

参考リンク

• Paper ↗

リンク先のライセンス・再現性・データ条件をご確認ください。自動分類や実装の目安は保証ではありません。

実装を始めるプロンプト

参考リンクを確認したうえで、次の内容をコピーして利用できます。

# 論文・記事
SE-Mem: Semantic-Episodic Memory for Scene-Consistent Generative View Synthesis

# 概要
Generative view synthesis aims to synthesize novel-view videos from reference observations along user-defined camera trajectories. Maintaining global scene consistency throughout this process remains…

# 原典
https://arxiv.org/abs/2611.00658

上記の手法を実装してください。実行前に依存関係とライセンスを確認してください。