Papers
arxiv:2607.23782

N_0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

Published on Jul 26
· Submitted by
Zhou Heng
on Aug 3
#3 Paper of the day
Authors:
,

Abstract

We present N_0-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, N_0-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, N_0-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. N_0-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.

Community

Paper submitter

Hi all, author here đź‘‹

We're excited to share N0-VTLA, a vision–tactile–language–action (VTLA) foundation model built for two things current VLA backbones struggle with: fine-grained contact-rich manipulation with real tactile feedback control, and offline policy improvement from data you've already collected during deployment.

The recipe has three parts:

  • Visuo-tactile pre-training on NeoData, our large-scale visuo-tactile robot dataset. To our knowledge this makes N0-VTLA the first VTLA model pre-trained on tactile data at scale.
  • Staged tactile-pathway integration in post-training, a predictive tactile pathway that distills the contact priors learned at scale into fine motion adjustments for downstream tactile-centric tasks.
  • [ALTER], an advantage-conditioned offline RL method that turns relative progress and trajectory-event comparisons into binary advantage labels, so a fixed deployment corpus can keep improving the policy.

Results: N0-VTLA wins all nine real-robot NeoReal tasks, and reaches 63.8% mean success on our 20-task simulation suite vs. 44.0% for the strongest baseline. With [ALTER], policies hit 75–95% success on three long-horizon real-robot tasks, including deformable object manipulation.

Happy to answer questions here — feedback very welcome!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2607.23782
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2607.23782 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2607.23782 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2607.23782 in a Space README.md to link it from this page.

Collections including this paper 2