drbaph/Viggle-Animate-ComfyUI

Viggle-Animate — ComfyUI conversions

ComfyUI-native conversions of Viggle/Viggle-Animate — a 33.1 B full finetune of MiniMax-H3's ref2va transformer for character replacement in video: a driving clip provides motion, camera, background and lighting; one reference still provides the character. No text encoder — conditioning is a frozen 362-token embedding. The sampler is DMD2-distilled: 4 steps (3 forward passes).

ComfyUI node pack (required for the conditioning layout): github.com/Saganaki22/ComfyUI-Viggle-Animate-H3

Example	Video
Example 1
Example 2
Example 3
Example 4

Model Links

diffusion_models (pick one — pruned is the VRAM-friendly option)

• minimax_h3_ref2va_viggle_pruned_int8_convrot.safetensors (21 GB)
• minimax_h3_ref2va_viggle_int8_convrot.safetensors (47 GB)
• minimax_h3_ref2va_viggle_bf16.safetensors (66.3 GB, max quality)

loras (DMD 4-step accelerator — pick one)

• viggle_animate_dmd_lora_r64.safetensors (0.94 GB, recommended)
• viggle_animate_dmd_lora.safetensors (3.8 GB, full rank)

text_cond

• fixed_embed_fwd_anyframe.safetensors — precomputed text conditioning, load with Load Text Conditioning (Viggle) (no text encoder needed)

vae

• minimax_h3_video_vae_int8_convrot.safetensors (3.17 GB, low VRAM)
• or minimax_h3_video_vae_fp16.safetensors (5.21 GB)

Model Storage Location

📂 ComfyUI/
├── 📂 models/
│   ├── 📂 diffusion_models/
│   │   └── minimax_h3_ref2va_viggle_pruned_int8_convrot.safetensors
│   ├── 📂 loras/
│   │   └── viggle_animate_dmd_lora_r64.safetensors
│   ├── 📂 text_cond/
│   │   └── fixed_embed_fwd_anyframe.safetensors
│   └── 📂 vae/
│       └── minimax_h3_video_vae_int8_convrot.safetensors

Workflow Notes

• Custom nodes required: ComfyUI-Viggle-Animate-H3 (Viggle Animate Conditioning + Load Text Conditioning) and ComfyUI-KJNodes (fast preview).
• Load Video: set frame_load_cap equal to the conditioning node's length (e.g. 124) and force_rate to 24.
• Output resolution follows the driving video by default; set the conditioning node's width/height to override (each axis rounds to 32), or pre-scale the clip with Scale Image to Total Pixels. Tested canvas range: 0.4–0.98 MP.
• Sampler: euler, er_sde, exp_heun_2_x0, lcm, simple, normal, beta, bong_tangent, 4-8 steps, cfg 1.0, ModelSamplingMiniMaxH3 shift 3.0.
• Stacks with Comfy Kitchen and block sparse attention patches.

Limitations

• Identity drift on re-entry: when the subject leaves the camera view and re-enters, the re-entry settles toward the driving video's original appearance rather than the reference image. The same applies when the subject moves far from the reference pose or makes abrupt large motions (e.g. a backflip) — the further from the still, the weaker the identity hold.

Base model (VAEs, audio VAE): MiniMaxAI/MiniMax-H3 · Original model & inference code: Viggle/Viggle-Animate

Citation

@misc{viggle2026animate,
  title  = {Viggle-Animate: Character Replacement in Video from a Single Repainted Frame},
  author = {Viggle Research},
  year   = {2026},
  url    = {https://huggingface.co/Viggle/Viggle-Animate}
}

License

The weights are a Model Derivative of MiniMax H3 — the MiniMax H3 Community License Agreement applies to them, including these converted/quantized variants. Read it before redistributing or shipping a product on them. Do not run the model on people who have not consented, and label generated output as AI-generated.

Report Issue

• Weights/conversion issues: ComfyUI-Viggle-Animate-H3/issues