iconLogo
Published:2025/12/25 5:25:36

最強ギャルAI、参上~!😎✨

SIFThinker: 空間認識で画像理解爆上げ!💖

超要約: 空間把握と注意力を合体させた画像理解AI、爆誕!

✨ ギャル的キラキラポイント ✨

● 3D空間(スリーディーくうかん)の情報も読み取れるから、立体的な画像も余裕~!😎 ● 気になる場所に自動で注目してくれるから、見逃しなし!👀💕 ● 難しいお絵かき(推論、すいろん)も、SIFThinkerが代わりにやってくれるってこと!✨

続きは「らくらく論文」アプリで

SIFThinker: Spatially-Aware Image Focus for Visual Reasoning

Zhangquan Chen / Ruihui Zhao / Chuwei Luo / Mingze Sun / Xinlei Yu / Yangyang Kang / Ruqi Huang

Current multimodal large language models (MLLMs) still face significant challenges in complex visual tasks (e.g., spatial understanding, fine-grained perception). Prior methods have tried to incorporate visual reasoning, however, they fail to leverage attention correction with spatial cues to iteratively refine their focus on prompt-relevant regions. In this paper, we introduce SIFThinker, a spatially-aware "think-with-images" framework that mimics human visual perception. Specifically, SIFThinker enables attention correcting and image region focusing by interleaving depth-enhanced bounding boxes and natural language. Our contributions are twofold: First, we introduce a reverse-expansion-forward-inference strategy that facilitates the generation of interleaved image-text chains of thought for process-level supervision, which in turn leads to the construction of the SIF-50K dataset. Besides, we propose GRPO-SIF, a reinforced training paradigm that integrates depth-informed visual grounding into a unified reasoning pipeline, teaching the model to dynamically correct and focus on prompt-relevant regions. Extensive experiments demonstrate that SIFThinker outperforms state-of-the-art methods in spatial understanding and fine-grained visual perception, while maintaining strong general capabilities, highlighting the effectiveness of our method. Code: https://github.com/zhangquanchen/SIFThinker.

cs / cs.CV / cs.AI