Instructions to use stabilityai/stable-video-diffusion-img2vid-xt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use stabilityai/stable-video-diffusion-img2vid-xt with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-video-diffusion-img2vid-xt", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Had to hack it to use fp16 to work on my 24GB 4090 with OOM
Had to hack it to use fp16 to work on my 24GB 4090 with OOM
and I'd like to know what motion_bucket_id does.
Had to hack it to use fp16 to work on my 24GB 4090 with OOM
and I'd like to know what motion_bucket_id does.
Hey I also have a 4090 what did you hack so I can get this running on my machine as well many thanks!
Had to hack it to use fp16 to work on my 24GB 4090 with OOM
and I'd like to know what motion_bucket_id does.Hey I also have a 4090 what did you hack so I can get this running on my machine as well many thanks!
Set this line to True
https://github.com/Stability-AI/generative-models/blob/059d8e9cd9c55aea1ef2ece39abf605efb8b7cc9/scripts/demo/streamlit_helpers.py#L61
Also set the "decode t frames" to lower like 7, would fit 24GB VRAM easily
Will this work with 16gb cards ?
14297MiB with low vram mode and "Decode t frames at a time (set small if you are low on VRAM)" set to 1
Had to hack it to use fp16 to work on my 24GB 4090 with OOM
and I'd like to know what motion_bucket_id does.Hey I also have a 4090 what did you hack so I can get this running on my machine as well many thanks!
Set this line to True
https://github.com/Stability-AI/generative-models/blob/059d8e9cd9c55aea1ef2ece39abf605efb8b7cc9/scripts/demo/streamlit_helpers.py#L61Also set the "decode t frames" to lower like 7, would fit 24GB VRAM easily
what's mean "set the 'decode t frames' to lower like 7"?
how to set 'decode t frames' ?
It's the last option when you launch the web interface
I dont get it, how come i can create nice animations with only 6gb of VRAM RTX 2060 and 16 GB RAM, using COMFY UI, it takes 15 minutes for 5 second animation, but at least DOES WORK, look here