ComfyUI, Hugging Face, Ollama
running AI locally

Three tools, three roles

"Running AI locally" means nothing until you have separated the trades. With us, three building blocks share the work:

Why a Mac Studio, and not a server

A Mac Studio M4 Max has 36 GB of unified memory: the CPU and GPU share the same space. A multi-gigabyte model does not need to fit in separate VRAM, it fits in the machine's memory, and the GPU accesses it directly.

Add silence, the power consumption of a set-top box and instant startup: for a visual production workshop, it is the right machine. For a service that must respond to a thousand simultaneous users, it is not, and we do not ask it to do that.

What we run on it

• FLUX.2-klein 4B for production visuals: a recent diffusion model, with its dedicated text encoder, mixing them produces blank images or dimension errors.
• Z-Image Turbo for fast iterations: fewer steps, coarser renders, but enough to validate a framing in two minutes instead of ten.
• bge-m3 under Ollama for embeddings: it is what turns a document into queryable vectors in our Qdrant database.

Pitfalls that cost an evening

• A "downloaded" model that weighs 15 bytes licensed repositories return an HTML error page instead of the file. `curl` says "success", the error only appears at generation. Always check the size.
• The 4 steps that produce blank output a setting tuned for a portrait does not suffice for a detailed scene. Twelve steps, not four.
• Two model families in the same queue loading a video model alongside an image model crashes the server. We clear the queue and restart between two families.
• The fp8 format on Apple Silicon some quantized variants are not supported by the Metal backend: you need the bf16 version, heavier but functional.

And virtual machines, in all this

The Mac produces images; it does not orchestrate. Orchestration, API calls, queues, conversions, backups, live in virtual machines, with a dedicated system for each task: one VM for the API, one for the vector database, one for batch processing.

This split has a cost (memory, administration) and a benefit that far exceeds it: a failed update only breaks one component, a snapshot lets you roll back in thirty seconds, and each service keeps its own library versions. On a single machine, everyone shares the same, and a crash is guaranteed.

The rule we impose on ourselves

No model enters production without knowing its size, license and origin. A model is code that executes: it is treated as a dependency, with the same requirement for version and provenance.