Claude Code and a local LLM on a Mac mini: What runs where in our setup

🖥️ The setup in one paragraph

🖥️ This guide is part of our Mac mini series. If you are still deciding on the hardware, start with is a Mac mini good for local LLMs and how much RAM and storage you need.

Most articles about local AI describe a setup someone could build. This one describes the one we use every day.

A Mac mini M4 Pro with 24GB of memory sits on a shelf with no monitor, no keyboard and no mouse. It never sleeps. An HDMI dummy plug convinces it a screen is attached, and an external 2TB SSD holds the projects, the photo library and the model files, because the internal drive filled up long ago. We reach it from a Windows Surface in the living room and spend most of our time in a terminal.

In that terminal runs Claude Code, Anthropic’s coding agent. Next to it runs Ollama, the local model server, with two vision models. The interesting part is not that both exist. It is how the work is split between them, and which of our assumptions turned out to be wrong.

Why a cloud agent and a local model, not one or the other

The local-versus-cloud debate is usually framed as a choice. For us it never was, because the two are good at different things.

A cloud coding agent is good at work that needs judgment across many files: read a codebase, work out why something broke, write a script, check it, explain the trade-off. A 12B model running on a Mac mini is not in the same league for that kind of work, and pretending otherwise costs more time than it saves.

A local model is good at a different job: the same small task, thousands of times, on material that should stay at home. Describe this photo. Give it ten tags. Say whether a hand is in the frame. It is free per run, it is private by construction, and nobody minds if it takes all night.

So the rule we work by is simple. Anything private stays on the Mac and only ever meets the local model. Code, articles and public content go to the cloud agent. Claude Code writes and runs the scripts that call the local model, but the photos themselves never go to the cloud.

What Claude Code does here

Claude Code runs in the terminal on the Mac mini and works inside our project folders. In practice it does four kinds of work.

It writes and maintains the scripts. The photo catalog, the import pipeline, the video helpers and the reports are all scripts it wrote and keeps working.

It runs them and reads the results. A script that fails at two in the morning leaves a log. Claude Code reads the log, finds the cause and proposes the fix.

It does the reviews. Before anything changes on the website, it runs the quality checks and the tests, and it reports what failed rather than what it hoped would pass.

It drafts. Articles like this one start as a draft in the same terminal, built from our notes and measurements, and only go live after a human reads them.

None of that needs to be clever about hardware. Claude Code runs on Anthropic’s servers, so the Mac mini’s chip barely matters for it. What matters is that the Mac is always on, always reachable, and holds all the files in one place.

There are ways to point coding agents at local models. We have not done it, because the reasoning quality is the whole point of using the agent, and our local models are chosen for a different job.

What the local model does here

The local side is Ollama running as a background service, with its model files on the external SSD. Two models are installed: gemma3 at 12B parameters and qwen3-vl at 8B parameters, both of which can look at images.

Their job is the photo library. When new photos arrive, a script measures each one (sharpness, brightness, how much of the frame is blown out or crushed), then asks the local model for a subject, a category, ten tags and a one-line description. The result goes into a small database, which is what makes “find every photo of the LEGO Razor Crest outdoors” a one-line question instead of an afternoon. The same approach describes video clips frame by frame, so the rough-cut helper can suggest which seconds are usable.

That is exactly the kind of work a local model is for. It is repetitive, it involves private family material, and speed per item matters less than the total cost of running it thousands of times.

The numbers we measured

Before choosing a default, we ran both models over the same 20 photos on the M4 Pro with 24GB.

Model	Average per photo	Valid structured output
gemma3:12b	about 12 seconds	20 of 20
qwen3-vl:8b	about 58 seconds	19 of 20

The smaller model was almost five times slower. That surprised us, and it is a good reminder that parameter count is not a speed rating. The descriptions were close in quality, with qwen3-vl occasionally more precise on hard subjects.

So gemma3:12b is the default and qwen3-vl is the second opinion for the photos where the first answer looks wrong. At 12 seconds a photo, a batch of a thousand runs in under three and a half hours, which is an evening, not a week.

On 24GB, the 12B model runs while everything else on the Mac stays usable. That matches what we wrote in our local LLM buying guide: 24GB is the honest floor for real work, and 16GB is fine for experimenting.

What the local model got wrong

This is the section we would have wanted to read before we started.

Quality scores are useless. We asked both models to rate each photo from 1 to 10. Both gave almost every photo the same score, sharp or soft, good or bad. We stopped asking and measure sharpness and exposure with plain image analysis instead. The model describes, the code measures.

Location guesses are invented. Asked where a landscape was taken, both models answered confidently and freely. A guess that sounds right is worse than no answer, because it ends up in the database as a fact. Location now comes only from GPS data or from us.

Single frames hallucinate. When the model describes a video one frame at a time, it sometimes sees things that are not there. A foggy LEGO Star Wars clip produced a cat and a character who was never in it. A hand in the frame was caught about two times in three. For cut decisions we check the contact sheet and the timeline instead of trusting the description.

None of this makes the local model bad. It makes it a tool with a narrow job: describe what is plainly visible, in a fixed format, many times. It is good at that.

The macOS traps nobody mentions

Three things cost us real time, and none of them show up in the usual setup guides.

Scheduled jobs cannot read the external drive. macOS treats an external SSD as removable media, and background jobs started by the system scheduler (launchd) are not allowed to read it by default. A job that works perfectly in the terminal silently fails at night. Our fix is to keep the copies that the scheduler runs in a folder on the internal drive, with the source still on the SSD. Do not hand the system shell full disk access to work around it.

The model server has to wait for the drive. If Ollama starts before the external SSD is mounted after a restart, it does not find its models. Ours waits for the drive before it starts.

The Mac must never sleep. Power settings are set so the Mac never sleeps, never spins down its disks and restarts by itself after a power cut. Without that, every scheduled job and every remote session depends on someone walking over and pressing a key. The headless setup guide covers the settings in detail.

Reaching it from a Windows PC

The Mac mini has no screen, and the computer on the couch is a Windows Surface, so the built-in macOS Screen Sharing is not an option from there. We use Sunshine, a free streaming host, on the Mac, and Moonlight, its client, on the Surface. It runs inside the home network.

It is not perfect. The biggest annoyance is that the mouse pointer disappears as soon as you type. Moving it to the bottom edge of the screen brings it back, but it gets old quickly. But it is stable and reliable, and because we spend most of the time in a terminal, it fits the job well. Of everything we tried, it is the best option we found.

Two details help. The HDMI dummy plug gives the Mac a display to render to, so the stream has a proper resolution. And a display utility (we use BetterDisplay) lets us lower the resolution of the streamed screen, which makes the text bigger and easier to read on the Surface. Some black bars remain, because the Mac and the Surface do not share an aspect ratio.

What we would buy today

Our machine is the M4 Pro with 24GB, which has since been replaced in the line-up. If we were buying now:

• For this exact workload, the Mac mini with the M5 Pro. The Pro chip’s faster memory bus is what makes a local model feel responsive, and it leaves headroom for a bigger model later.
• If price decides, the M6 Mac mini with 24GB. It is the cheapest way to the memory tier that matters. It is slower per photo than a Pro chip, but for overnight batches that hardly matters.
• An external SSD from day one, not when the internal drive is full. A 2TB Samsung T7 Shield is what we would pick.
• An HDMI dummy plug if the Mac will live without a monitor. It costs a few dollars and fixes the low-resolution remote session for good.

For the full comparison of chips, memory and storage, see our Mac mini RAM and storage guide and the accessories guide.

Can Claude Code use a local LLM?

In our setup Claude Code itself runs on Anthropic’s cloud models, and the local model is a separate tool that Claude Code’s scripts call through Ollama. That split is deliberate: the reasoning-heavy work stays with the stronger cloud model, and the private bulk work stays on the Mac.

Is a Mac mini with 24GB enough for a local LLM?

For our workload, yes. A 12B vision model (gemma3:12b) runs at about 12 seconds per photo on an M4 Pro with 24GB, with the rest of the system still usable. 24GB is the tier we would call the floor for real work. 16GB is fine for experimenting. More in our local LLM guide.

Which local model is best for tagging photos?

In our test of 20 photos, gemma3:12b took about 12 seconds per image and returned valid structured output 20 times out of 20. qwen3-vl:8b took about 58 seconds and returned valid output 19 times out of 20. We use gemma3 as the default and qwen3-vl only as a second opinion.

Can a local model judge photo quality?

Not reliably in our experience. Both models we tested gave almost every photo the same score. We measure sharpness and exposure with plain image analysis instead and use the model only for subject, tags and a description.

Why not do everything in the cloud?

Because some of the material is private. Family photos and personal data stay on the Mac and only ever see the local model. Code, articles and public content go to the cloud agent, which is simply better at that work.

Does the Mac mini need to stay on all the time?

For this setup, yes. Ours never sleeps and restarts on its own after a power cut. Scheduled jobs, the local model server and remote access all depend on it being awake, and the power draw of a Mac mini at idle is small. The headless guide has the settings.

How do you control a headless Mac mini from a Windows PC?

We use Sunshine on the Mac and Moonlight on a Windows Surface, inside the home network. It is stable and reliable for terminal work. The one annoyance is that the mouse pointer disappears when you type; moving it to the bottom edge of the screen brings it back.