ComfyUI, AnimateDiff, and the Open Local Stack

Running ComfyUI and AnimateDiff on a laptop. The stack, the recipe, and the work it makes.

Owning the model loop on the machine in front of you.

There is a version of generative AI that lives behind a paywall, and a version that runs on the laptop you already paid for. I work in the second one.

ComfyUI on a laptop

ComfyUI is the graph editor. Every step of the pipeline is a node, the graph is the document. On Apple Silicon:

git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
python main.py

That gets the canvas up at localhost:8188. Drop ComfyUI Manager into custom_nodes/ so installs become a search box instead of a scavenger hunt. For motion work, add three packs: AnimateDiff-Evolved, VideoHelperSuite, Advanced-ControlNet. Pull weights from Hugging Face into models/checkpoints, models/loras, models/animatediff_models, models/animatediff_motion_lora. Keep a manifest so a client can ask what produced an image and the answer is a row in a file, not a guess.

AnimateDiff in practice

The recipe: SD 1.5 with the AnyLoRA checkpoint for the line-driven anime look. AnimateDiff (v3_sd15_mm.ckpt) as the motion module. Uniform context options, 16-frame windows, pyramid scheduling. Optional motion LoRAs for pan, zoom, tilt, rotate. A Video Combine node writes the mp4.

Same graph, different scene cards. Late Night TV, eight channels:

That is the whole point of doing this locally. The graph is editable, the weights are local, the JSON is a document I can hand to a collaborator. Hosted text-to-video can’t do any of that.

DiffusionBee and AUTOMATIC1111

When the question is “can we see this idea in the next ten seconds,” DiffusionBee wins. One installer, no Python. When the question is “lock the seed and sweep a parameter,” AUTOMATIC1111’s WebUI wins, browser interface, every knob exposed. ComfyUI is for when the answer is a workflow.

Why local

A designer who only knows the hosted tools is a designer renting their pipeline. A designer who can stand up the open stack on a laptop is a designer who can answer questions about the work: what model produced this, under what license, with what controls, and how do we reproduce it next quarter.

Same reason I work on the open web. I want to own the system that makes the work, not lease it.