wizdroid/video-generation-workflow

batch-clips

batch-clips turns a folder of clip prompts into a finished film. It sends each prompt to ComfyUI, keeps the shots in order, stitches the clips, and writes the picture and the sound as separate files.

One MiniMax H3 job runs at a time. When you give it several projects, it finishes one film before it starts the next. You do not start the next project by hand.

The program is quickies/comfyui/batch-clips. This guide sits in the same folder, next to minimax-fl2v.json. The separate plan for writing those prompts with Ollama is quickies/comfyui/story-pipeline.md. The usage steps for the Ollama prompt test are quickies/comfyui/experiments/README.md. From ~/Projects/short-films you can run it as ./quickies/comfyui/batch-clips, or as batch-clips after reload.

What you need

1. ComfyUI is running at http://127.0.0.1:9000.
2. ffmpeg and ffprobe are on your PATH.
3. The graph is ~/Projects/short-films/quickies/comfyui/minimax-fl2v.json. It is a MiniMax H3 image-to-video graph. The prompt is node 105:104. The first frame is LoadImage node 144.
4. The GPU is free. Do not run IndexTTS, or a second H3 job, at the same time.

Open a new terminal, or run reload, so the batch-clips command is on your PATH.

A project folder

A project is one directory of prompts. The folder name becomes the film name.

quickies/2020s/reva-bengaluru/
  arc.md                 ignored
  clip-01-15s.txt        required name shape: clip-NN-Ds.txt
  clip-02-15s.txt
  clip-03-15s.png        optional still for a clip that does not continue
  renders/               created by the tool

Clips run in numeric order: clip-01, then clip-02, and so on. arc.md and any other file are ignored.

The number before s in the filename is the duration in seconds. clip-04-15s.txt is sent as 15 seconds.

Step 1. Write the handoff header

The first lines of each prompt tell the tool whether the shot continues:

handoff-from: none
end-frame: a description of the held picture
next: new scene, do not reuse this frame

integrated_multimodal_description: ...

handoff-from: none starts a new picture. The tool does not reuse the previous clip.

handoff-from: clip-03 last frame
end-frame: the picture this clip holds at the end
next: clip-05

handoff-from: clip-03 last frame means this clip opens on the last frame of clip 03. After clip 03 has rendered, the tool extracts that frame and loads it as the first frame of this job.

Step 2. A still is optional

first_frame on MiniMax H3 Image to Video is optional.

• handoff-from: clip-03 last frame uses the last frame of clip 03. You do not add a still.
• A file beside the prompt (clip-02.png, .jpg, .jpeg, or .webp) is loaded as that clip's first frame.
• --first-image still.png is the first frame of the first fresh clip in the project.
• A fresh clip with none of those is queued with the image input disconnected. It generates from the text alone.

Step 3. Dry-run one project

From ~/Projects/short-films:

batch-clips --dry-run --first-image stills/reva.png quickies/2020s/reva-bengaluru

Read the plan before you spend GPU time. A dry-run does not queue ComfyUI. It prints, for each clip:

• where the first frame comes from
• the output prefix
• the duration
• the stitch that will run at the end

A fresh clip with no still is listed as first frame bypassed. That is a text-to-video shot, not an error.

Step 4. Generate

batch-clips --first-image stills/reva.png quickies/2020s/reva-bengaluru

For each clip the tool:

1. Copies a still or an extracted last frame into the ComfyUI input folder when this clip has one. A clip with no reference is sent with first_frame disconnected.
2. Puts the whole prompt file into the graph.
3. Sets the save prefix to videos/quickies/<film>/<clip>.
4. Queues one job and waits until ComfyUI finishes it.
5. Downloads the video to renders/<clip>.mp4.

Then it stitches and splits. You wait. Do not start another H3 job while this is running.

Step 5. Collect the files

Everything is under the project’s renders/ folder.

File	What it is
clip-01-15s.mp4	One generated clip
clip-01-15s_last.png	Last frame taken from a clip that a later shot continues
reva-bengaluru.mp4	All clips joined in order, picture and sound
reva-bengaluru_video.mp4	The joined picture, with no audio stream
reva-bengaluru_audio.m4a	The joined sound, with no picture

The join is a stream copy in clip order. If the clips contain no audio, the sequence and the picture file are still written, and the tool says that no audio file was made.

Several projects, one after another

Pass the folders in the order you want them made. The second project starts only after the first has been generated, stitched, and split.

batch-clips \
  --first-image stills/open.png \
  quickies/2020s/reva-bengaluru \
  quickies/2020s/dewi-mariam \
  quickies/2020s/meher-mumbai

--first-image is used for each project’s first fresh clip. A project that already has clip-01.png beside the prompt uses that file instead.

For a long unattended run, put one project per line in a queue file. A line is the folder, then an optional starting still. Blank lines are skipped. Anything after # is a comment. A relative path is resolved from the queue file’s own directory.

overnight.txt:

# Save this file in ~/Projects/short-films.
# Paths below are relative to that file.

quickies/2020s/reva-bengaluru  stills/reva.png
quickies/2020s/dewi-mariam     stills/dewi.png
quickies/2020s/meher-mumbai    stills/meher.png
quickies/40s/kamla-grant-road  stills/kamla.png
quickies/80s/leela-bombay      stills/leela.png

Start it and leave it:

batch-clips --queue overnight.txt

You can mix a queue file with extra folders. The file runs first, then the folders on the command line, still one project at a time.

batch-clips --queue overnight.txt quickies/2020s/extra-film

If one project fails, the tool prints the error and goes on to the next project. At the end it exits with status 1 and tells you how many projects failed. The ones that finished stay in their renders/ folders. Fix the failed folder and run it again with --skip-existing so finished clips are not generated twice.

Resume

batch-clips --skip-existing --first-image stills/reva.png quickies/2020s/reva-bengaluru

--skip-existing keeps any renders/clip-NN-Ds.mp4 that is already there and generates only the missing clips. Clips that continue from an existing file still extract that file’s last frame.

Stitch clips you already have

This does not call ComfyUI. Every clip-*.txt must already have a matching renders/*.mp4.

batch-clips --assemble-only quickies/2020s/reva-bengaluru

A queue of finished projects can be stitched the same way:

batch-clips --assemble-only --queue overnight.txt

--no-assemble generates the clips and does not join them.

Other options

Option	Meaning
--workflow PATH	API graph to queue. The default is minimax-fl2v.json.
--host URL	ComfyUI. The default is http://127.0.0.1:9000.
--input-dir PATH	ComfyUI input folder. The default is ~/Applications/comfyui/input.
--timeout SECONDS	How long to wait for one clip. The default is 2700 (45 minutes).
--dry-run	Print the plan for every project. Do not generate.
--self-test	Check parsing, the graph, last-frame extract, stitch, and a two-project queue.

A complete overnight example

1. Start ComfyUI and leave the GPU idle.
2. Save a starting still for each film that begins with handoff-from: none.
3. Write overnight.txt as shown above.
4. Check the plan:

batch-clips --dry-run --queue overnight.txt

5. Start the run and leave the terminal open:

batch-clips --queue overnight.txt

6. In the morning, each project’s renders/ folder holds the clips, the joined film, the picture-only file, and the audio-only file.