Which local model still answers on a 16 GB Mac after the browser has taken half the memory?

A 3B-class instruct model at 4-bit. On a 16 GB Mac, after the browser is sitting on about 8 GB, Llama 3.2 3B Instruct, Qwen2.5 3B Instruct, or Gemma 2 2B still answers in Ollama or MLX. Phi-4-mini, around 4B at 4-bit, is the largest I still trust with the browser left open and the context kept short. An 8B Q4 might load. It does not keep answering. It swaps, the fans come up, and the reply arrives after you have already switched back to the tab.

Unified memory is one pool. The model does not get a private half. Chrome, Safari, the OS, and the KV cache drink together. “16 GB” on the laptop sticker is not 16 GB for weights.

What is actually left after the browser takes half?

Half of 16 GB is 8 GB, and that number is already the optimistic reading of Activity Monitor. A dozen tabs, a docs site, Slack in a tab, and a video paused in the background land in that range fast. macOS wants a few more gigabytes so the machine does not immediately compress everything. What remains for weights plus cache is often 3 to 5 GB, not 8.

A 4-bit 3B model is about 2 GB of weights. A short context, a few thousand tokens, adds a cache you can afford. That is why it still answers while you leave the research tabs up. A 4-bit 8B is about 5 GB of weights before the cache. It fits on an idle 16 GB Mac. It does not fit in the leftover slice. The loader will still say the model is ready, because “ready” means the process started, not that the pages are resident.

Which names still answer, and which only load?

These still answer, in Ollama as a Q4 GGUF or in MLX at 4-bit, with the browser left open:

• Llama 3.2 3B Instruct. The default small one people already have. Good enough for a rewrite, a summary of the tab you pasted, a regex. Not a coding partner.
• Qwen2.5 3B Instruct. The one I reach for when the 3B has to follow a slightly fussier instruction. Similar weight. Often a bit more coherent on short chores.
• Gemma 2 2B. Smaller, snappier, thinner answers. Use it when you want the fan to stay quiet.
• Phi-4-mini (the roughly 4B one, not a 14B Phi). The top of the range. Keep the context window at 4k, not 32k. The extra billion parameters are fine. A huge cache is not.

These load and then punish you:

• Llama 3.1 8B, Qwen2.5 7B, Mistral 7B, and the 7B DeepSeek distillations at Q4. On an empty 16 GB Mac they are the sensible local default. Next to an 8 GB browser they are a swap storm. You get a first sentence, then a stall, then a reply you could have typed yourself.
• Anything 14B or above, including the reasoning models people download because a screenshot looked smart. They do not answer under this memory split. They page. Quitting the browser is the product decision, not a quant trick.
• An 8B at Q8 or Q5 “for quality.” You already did not have room for Q4. A heavier quant is a larger file in a smaller hole.

Ollama and MLX can both run the 3B. The stack is not what saves you once the weights do not fit. It will not shrink an 8B into the leftover 4 GB.

Why does a short context matter as much as the parameter count?

People set the context to 32k or 128k because the model card allows it, then paste one paragraph. The cache is allocated for the window you configured, not for the paragraph you typed. On a 3B that was fine at 4k, a 32k window can cost as much RAM as jumping to a bigger model. The browser did not grow. Your setting did.

In Ollama, set the context on that run to 4096 unless you have measured otherwise. In MLX, do not pass a max sequence that you copied from a 64 GB studio config. If the answer you need is longer than a page, you still do not need a 32k cache to draft it. You need the model to stay in RAM for the next sentence.

Pasting the whole browser is the other way this fails. If the reason the browser is at 8 GB is twenty tabs of documentation, do not also paste all twenty into the 3B. Paste the one section. The model that “stopped answering” is often a 3B that was fine until the prompt became a book.

How do you tell an answer from a swap?

Activity Monitor, memory pressure, not the token counter. Yellow is the 3B living next to the browser. Red, with compressed memory climbing and the beachball between tokens, is the 8B pretending. Quit the model, not the test. If the same prompt is fast on Llama 3.2 3B and sticky on the 8B, you have the result. Do not “give the 8B a minute to warm up.” Warm-up is a few seconds. A minute is the disk.

Also watch wired memory and the model process. If the process is huge and the system is compressing, tokens will keep arriving and the Mac will feel broken everywhere else, including the browser you were trying to keep. A local model that costs you the tabs is not the model that answers after the browser took half. It is the model that evicted the browser.

What should you stop asking the 3B to do?

Stop asking it to be the 8B you wanted. A 3B will summarize a section, draft a reply, explain an error message you pasted, and write a small function you can read. It will invent APIs, lose the plot on a long diff, and sound confident about a codebase it cannot see. That is the trade for leaving Chrome open. If the task is the hard diff, close the browser or use a hosted model. The 16 GB Mac with half its memory in tabs is a 3B machine. Treating it as a 14B machine just makes both the model and the browser worse.

Do not keep a second model loaded “for when you need it.” Ollama will hold a runner. Two runners on this split is how the 3B that was fine becomes the stall. One model. Unload it when you are back to only browsing.

Apple’s on-device system features are not this test. They are small models with a budget Apple already reserved. They do not answer the open-ended question you were about to ask Ollama. Do not count them as the local model that replaced the 8B.

Does an M1 differ from an M4 on this split?

Not in the way the ads suggest. An M4 moves tokens faster when the weights are resident. It does not grow the 16 GB. Memory bandwidth helps a 3B feel instant. It does not stop an 8B from being paged out by Safari. I have had the same red memory graph on an M1 Air and on a newer 16 GB Pro, with the same browser and the same Q4 8B. The Pro recovered quicker after I unloaded the model. It did not answer during the stall.

An 8 GB Mac is a different machine. Half a browser there leaves almost nothing, and even the 3B starts to hurt if the OS is already compressed. This piece is the 16 GB case. If you are on 8 GB, the model that still answers is the 2B, or no local model until the browser is quit. Do not interpolate the 16 GB advice downward and then blame Qwen.

Quantization experiments will not save the afternoon. Q3 on an 8B shaves some weight and wrecks the answers enough that you will rerun the prompt, which costs the time you saved. Q4 on a 3B is the boring pair that matches the hardware you actually have while the tabs are open. Download the 8B later, for the hour you close them.

A setup that matches the half-eaten Mac

1. Look at Activity Monitor. If the browser is not actually near half, you may still have room for an 8B. This article is for when it is.
2. Run one of Llama 3.2 3B Instruct, Qwen2.5 3B Instruct, or Gemma 2 2B, 4-bit, context 4096.
3. Ask for the short job. If you need Phi-4-mini, use it for that chat and unload it after.
4. If memory pressure goes red, you picked the 7B or a giant context. Go back to the 3B. Do not add swap and call it a tuning.
5. When the work is a large coding change, quit the browser or leave the machine. The 3B will still answer. It will not become the model you closed the tabs to avoid downloading.

The local model that still answers on a 16 GB Mac, with the browser holding half the memory, is a 3B at 4-bit, or a 4B mini with a short context. The 8B is the model you run when the browser is closed. Both can be installed. Only one of them is telling the truth while the tabs stay up.