Qwen3.5-4B-MMFlash

MMFlash draft model for speculative decoding of Qwen/Qwen3.5-4B on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang DFLASH).

This repository contains the model introduced in MMFlash: Faster Speculative Decoding for Vision-Language Models.

Project page: https://fscdc.github.io/MMFlash/ — Code: https://github.com/fscdc/MMFlash

MMFlash builds on DFlash block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data with a loss weighted by per-token visual-grounding (visual-KL) scores.

Qwen3.5-9B on a MathVista question, recorded in real time on one NVIDIA H200 with SGLang (greedy decoding).

Need a drafter for another model? Email us at fscnkucs@gmail.com and we can help train one. We will keep updating these releases, and a Qwen4 checkpoint is planned.

Model details

Target model Qwen/Qwen3.5-4B
Parameters 537M (bf16)
Block size 16
Target hidden states layers 1, 8, 15, 22, 29 (of 32)

Usage

SGLang

SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1 python -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-4B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path FSCCS/Qwen3.5-4B-MMFlash \
  --speculative-dflash-block-size 16 \
  --attention-backend fa3 \
  --mm-attention-backend sdpa \
  --disable-radix-cache \
  --reasoning-parser qwen3 \
  --trust-remote-code

vLLM

vllm serve Qwen/Qwen3.5-4B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --attention-backend FLASH_ATTN \
  --reasoning-parser qwen3 \
  --speculative-config '{"method": "dflash", "model": "FSCCS/Qwen3.5-4B-MMFlash", "num_speculative_tokens": 15}'

Tested with vLLM 0.32.0, which serves DFlash drafts natively. num_speculative_tokens is the block size minus one.

Results

Each cell is τ / speedup: τ is the average acceptance length (tokens accepted per verification step), and the speedup is the decoding throughput over Qwen3.5-4B without speculative decoding.

Temperature 0

Method ChartQA CharXiv MMStar MMBench DynaMath MathVista MathVerse Avg.
MTP (3) 3.18 / 1.69× 3.27 / 1.76× 2.95 / 1.61× 2.43 / 1.40× 3.27 / 1.69× 3.18 / 1.68× 3.17 / 1.69× 3.06 / 1.64×
MTP (7) 4.26 / 1.68× 4.64 / 1.87× 3.75 / 1.57× 2.58 / 1.24× 4.42 / 1.69× 4.29 / 1.67× 4.27 / 1.69× 4.03 / 1.63×
MTP (15) 4.67 / 1.26× 5.37 / 1.43× 3.99 / 1.14× 2.60 / 0.87× 4.76 / 1.23× 4.65 / 1.21× 4.61 / 1.22× 4.38 / 1.19×
EAGLE-3 (16) 4.31 / 1.74× 4.38 / 1.86× 3.49 / 1.49× 2.61 / 1.08× 4.29 / 1.67× 4.02 / 1.56× 3.95 / 1.68× 3.86 / 1.58×
DFlash-z (16) 4.92 / 2.76× 5.63 / 3.10× 3.81 / 2.35× 5.01 / 2.62× 4.82 / 2.56× 4.43 / 2.45× 4.34 / 2.50× 4.71 / 2.62×
DFlash (16) 5.43 / 2.97× 5.60 / 3.13× 4.05 / 2.31× 5.17 / 2.93× 5.57 / 2.57× 5.16 / 2.53× 4.84 / 2.62× 5.12 / 2.72×
MMFlash (16) 5.97 / 3.28× 6.17 / 3.46× 4.49 / 2.55× 5.68 / 3.19× 6.26 / 2.87× 6.00 / 2.78× 5.35 / 2.88× 5.70 / 3.00×

Values in parentheses are the number of draft steps for MTP, the draft tree size for EAGLE-3, and the block size for DFlash and MMFlash. MTP is Qwen3.5-4B's built-in multi-token-prediction head. DFlash-z is the public z-lab/Qwen3.5-4B-DFlash draft. DFlash is our DFlash draft trained on the same data without the visual-KL weighting.

Concurrency (temperature 0)

MMFlash at 1, 8 and 32 requests in flight, on a single NVIDIA H200. Each cell is the decode throughput in tokens/s, with the speedup over Qwen3.5-4B alone at the same concurrency in parentheses.

Concurrency ChartQA CharXiv MMStar MMBench DynaMath MathVista MathVerse Avg.
1 783 (3.28×) 826 (3.46×) 607 (2.55×) 772 (3.19×) 685 (2.87×) 664 (2.78×) 686 (2.88×) 718 (3.00×)
8 2,937 (2.23×) 3,102 (2.37×) 2,347 (1.73×) 1,451 (2.16×) 3,081 (2.12×) 1,963 (1.60×) 2,956 (2.02×) 2,548 (2.03×)
32 3,799 (1.59×) 4,395 (1.61×) 3,296 (1.19×) 2,364 (1.35×) 4,643 (1.35×) 2,654 (1.16×) 4,237 (1.30×) 3,627 (1.37×)

Citation

@article{feng2026mmflash,
  title   = {MMFlash: Faster Speculative Decoding for Vision-Language Models},
  author  = {Feng, Sicheng and Chen, Zigeng and Wang, Song and Fang, Gongfan and Wang, Xinchao},
  journal = {arXiv preprint arXiv:2611.18482},
  year    = {2026}
}
Downloads last month
88
Safetensors
Model size
0.5B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for FSCCS/Qwen3.5-4B-MMFlash

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,457)
this model

Collection including FSCCS/Qwen3.5-4B-MMFlash

Paper for FSCCS/Qwen3.5-4B-MMFlash