Papers
arxiv:2611.19200

LogT: Logically Think with Images for Visual Search

Published on Nov 14
· Submitted by
Yanjun Fu
on Nov 19
Authors:
,
,
,
,
,
,

Abstract

Thinking with images empowers vision-language models (VLMs) to use an external Zoom-in tool for fine-grained visual search. Recent works employ supervised fine-tuning (SFT) and reinforcement learning (RL) with tool-specific designs to enable such capabilities. However, achieving logical reasoning alongside accurate and efficient tool use remains a significant challenge. To address this, we shift the focus away from increasing RL complexity and instead identify SFT data quality as a key bottleneck. We then propose LogT (short for Logically Think with Images), a fully automated SFT data engine that explicitly enforces logic-chain-guided reasoning. LogT generates an SFT dataset featuring challenging, visually grounded questions and structured tool-use traces with human-like logic. We validate our approach by fine-tuning two base models, Qwen2.5-VL-7B and InternVL3.5-8B, on the dataset and evaluating them across seven benchmark test sets. We show that LogT-SFT based on Qwen2.5-VL-7B outperforms the state-of-the-art trained with both SFT and RL, despite using 3.1times less data and >120times fewer GPU hours. Furthermore, applying a simple RL recipe without tool-specific design on LogT-SFT yields a model that surpasses prior methods with complex RL setups by a substantial 9.59-point margin. In addition to strong performance gains, our model exhibits higher tool-use accuracy and efficiency, as well as more structured and logically consistent reasoning traces, directly validating the effectiveness of our approach.

Community

Paper author Paper submitter
•
edited 1 day ago

When answering questions about images, vision-language models (VLMs) can benefit from zooming in on small but important details, much like humans do. However, teaching VLMs when and how to zoom in typically requires complex post-training methods. In this work, we take a different approach: instead of increasing training complexity, we focus on improving the quality of supervised fine-tuning (SFT) data. We introduce LogT, an agentic data engine that generates high-quality SFT examples. Remarkably, models trained solely on our SFT data outperform those trained with both SFT and reinforcement learning (RL). With standard, off-the-shelf RL, our model surpasses the state-of-the-art open-source model by an average of 9.59 points across seven test sets. These results demonstrate that high-quality SFT data can be more effective than sophisticated training methods, offering a much more efficient yet powerful approach to improving visual reasoning in VLMs.

This work was accepted at NeurIPS 2026.

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.19200 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.19200 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.19200 in a Space README.md to link it from this page.

Collections including this paper 1