linyq/kiwi-edit-5b-instruct-reference-diffusers

Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance

Kiwi-Edit is a versatile video editing framework built on an MLLM encoder and a video Diffusion Transformer (DiT). It supports both instruction-based video editing and reference-guided editing (using a reference image and instruction).

• Paper: Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance
• Project Page: https://showlab.github.io/Kiwi-Edit/
• Repository: https://github.com/showlab/Kiwi-Edit

Model Description

Kiwi-Edit introduces a unified editing architecture that synergizes learnable queries and latent visual features for reference semantic guidance. It addresses the challenge of precise visual control in instruction-based editing by allowing users to provide a reference image to guide the transformation. The framework achieves significant performance improvements in instruction following and reference fidelity through a scalable data generation pipeline and a multi-stage training curriculum.

Usage

This model is compatible with the diffusers library. To run inference, follow the installation instructions in the official repository.

Quick Test with Diffusers

You can run a quick test on a demo video using the following command provided in the repository:

python diffusers_demo.py \
    --video_path ./demo_data/video/source/0005e4ad9f49814db1d3f2296b911abf.mp4 \
    --prompt "Remove the monkey." \
    --save_path output.mp4 \
    --model_path linyq/kiwi-edit-5b-instruct-only-diffusers

Citation

If you find this work useful, please cite:

@misc{kiwiedit,
      title={Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance}, 
      author={Yiqi Lin and Guoqiang Liang and Ziyun Zeng and Zechen Bai and Yanzhe Chen and Mike Zheng Shou},
      year={2026},
      eprint={2603.02175},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2603.02175}, 
}