New Macs for Local AI: What Each Tier Can Run

Apple's new desktop Macs make local AI look simple: choose a faster chip, load a larger model and stop paying for cloud tokens. The useful buying decision is less glamorous. First buy enough unified memory for the model, context and runtime to coexist. Then ask whether the memory bandwidth and software stack deliver a speed you can live with.

That is the central insight in Yume_X's detailed local-AI buying map. It is also the right correction to Apple's large launch multipliers. Apple measured up to 4.8× faster LM Studio prompt processing on M6 versus M4 and up to 4× on M5 Ultra versus M3 Ultra. Those are important prefill results; they are not a promise that every model will generate tokens four times faster.

This guide is current to 27 August 2026, before retail units ship and before credible Mac benchmarks exist for the newly released Qwen3.8-Flash-Next and GLM-5.3-Flash. The tier recommendations below are therefore model-fit assessments. Any speed range attributed to the source post remains a forecast to test, not an AIEngine benchmark.

Separate capacity, prefill and decode

Three hardware properties answer three different questions.

Apple's Mac mini announcement says M6 reaches 170GB/s and M5 Pro reaches 307GB/s. Its headline LM Studio comparisons are prompt-processing tests: up to 4.8× M4 for M6 and up to 4× M4 Pro for M5 Pro. The Mac Studio announcement similarly gives up to 614GB/s for M5 Max and 1.2TB/s for M5 Ultra, while its 3.9× and 4× LLM figures are also LM Studio prompt processing.

This distinction matters for agents. A long system prompt, retrieved evidence and tool definitions can make prefill the visible delay before anything happens. Neural Accelerators in each GPU core directly target that wait. A short interactive chat with a long answer may care more about decode. A batch service with several simultaneous users adds a third requirement: aggregate throughput. One launch number cannot represent all three.

Bandwidth is not a guaranteed token-rate equation. Quantisation layout, cache format, model architecture, expert routing, runtime version and kernel quality all change how efficiently a model uses the pipe. A 47% bandwidth increase is a useful first estimate for a memory-bound decode comparison on the same model and mature stack. It is not a substitute for a timed run.

Work out model fit before comparing chips

The capacity test is not “model parameters × bits”. A safe working set includes:

• the packaged model weights, including quantisation metadata;
• the key-value cache at the context length actually used;
• runtime buffers, vision encoders, draft or MTP state where applicable;
• memory reserved by macOS, the display and ordinary applications; and
• enough free headroom to avoid sustained memory pressure and swap.

Treat an 8–12GB operating reserve as a planning allowance, not a fixed macOS law. Close apps during a benchmark, but size a production machine for the way it will really run. Context is especially easy to omit: a model advertising 262K or one million tokens does not make that window free. Our context working-set guide explains why maximum context and useful context are different numbers.

Quantisation trades memory for fidelity. Four-bit weights often provide a strong practical balance, but “Q4” is a family of formats rather than one identical file size or quality level. Two-bit builds can make an impressive model load on smaller hardware while losing capability that justified the model in the first place. Compare the intended quant against a higher-precision reference on real tasks before treating “it opens” as “it works”.

Mixture-of-experts models need another caution. A model may activate only a small fraction of its parameters for each token, reducing compute, while the full expert pool still needs to remain addressable in memory or be fetched from a slower tier. Active-parameter count predicts neither package size nor Mac decode speed by itself.

What the new Mac tiers are actually good for

Apple's live Mac mini specifications and Mac Studio specifications expose an important detail hidden by family-level headlines: the base and top GPU configurations can have different bandwidth and memory choices. M5 Max begins with a 32-core GPU and 460GB/s; the 40-core configuration reaches 614GB/s and unlocks 48GB, 64GB or 128GB. M5 Ultra starts at 96GB, while 256GB and 512GB require the top 36-core CPU and 80-core GPU.

The table is deliberately conservative. A 32GB M6 can load useful models, and a 64GB M5 Pro can load some 70B quants. The question is whether enough room remains for the desired context, vision input, runtime, other applications and a stable session. A machine that survives a one-line prompt at zero swap is not automatically a comfortable daily host.

The 96GB Ultra illustrates the memory-first rule. It offers much more bandwidth than a 128GB Max, so a model that fits can decode faster. It cannot use that bandwidth to load a quant that needs more than its available working set. Choose it only when measurements show that speed on a known fitting model is worth more than the Max's extra 32GB.

Apple opened pre-orders on 25 August. Mac mini starts at $899 for M6 and $1,699 for M5 Pro in the US; Mac Studio starts at $2,499 for M5 Max and $5,499 for M5 Ultra. Those are base prices, not the cost of the memory configuration in the table. Retail availability begins 22 September, while Apple says the 512GB Studio will arrive in late October. Check the live local configurator before approving a budget.

The two new models sharpen the memory question

Qwen3.8-Flash-Next is a multimodal MoE and an early preview of Qwen4's architecture. Qwen reports a 125B-parameter main model, another 51B parameters in n-gram embeddings and 6B active parameters per token. It supports 262,144 tokens natively and can extend to one million with YaRN. The embeddings are designed so they can be kept in host memory and prefetched, which makes backend implementation part of the fit calculation.

A 128GB M5 Max is therefore an interesting candidate for a compressed Qwen3.8-Flash-Next build, but not a confirmed recommendation. File format, embedding placement, cache, multimodal components and MLX or llama.cpp support decide whether a useful four-bit-style package fits with room. The 256GB Ultra removes most of that capacity anxiety, but spending more before a stable Mac recipe and benchmark exist reverses the evidence-first process.

GLM-5.3-Flash's official model card reports 320B total parameters and 18B active, with an MIT licence and a hybrid sparse-and-linear attention design. Simple four-bit arithmetic is already about 160GB before scales, metadata, cache and runtime buffers. That puts a high-quality Q4 deployment naturally in the 256GB class, subject to an actual compatible quant and Metal implementation. A 128GB machine would need a materially more aggressive build or offload.

GLM-5.2 shows what the 512GB option is for. Unsloth's published GLM-5.2 GGUF set lists a 754B model, with its UD-Q4_K_M package at 466GB and several other four-bit variants from 365GB to 467GB. A 512GB Studio can hold some of those files, but the largest leaves only a narrow margin for macOS, cache and runtime. This is a loadable specialist configuration, not evidence of a pleasant one-million-token workstation.

Model licences, supplied formats and redistribution rights also matter after fit. Use the open-weight licence field guide before turning a successful local test into a customer-facing service.

Forecasts are hypotheses until shipping Macs are timed

The source post does useful work by separating its estimates. It forecasts roughly 55–65 tokens per second with MTP for a 27B model on a 48GB M5 Pro, 58–70 on a 64GB 40-core M5 Max, 40–50 base and 60–80 with MTP for DeepSeek V4-Flash Q4 on a 256GB M5 Ultra, and 18–28 for GLM-5.2 Q4 on the 512GB Ultra.

Those ranges are plausible hypotheses, not launch facts. They scale measured older machines by official bandwidth, then treat Apple's prompt-processing multiplier separately. The method is more defensible than applying a 4× prefill result to decode, but it still assumes mature Metal kernels, comparable memory utilisation and similar speculative-decoding acceptance on new hardware.

MTP and other speculative methods predict several tokens and verify them together. They can produce large speed-ups when the workload is predictable and the runtime supports the model well. They can also consume more memory, reduce context headroom and deliver a smaller gain on open-ended prose than on code or structured output. Every MTP number should therefore name the model build, runtime, prompt type, context, acceptance rate and whether the reported rate excludes prefill.

For M5 Ultra in particular, test the quad-die implementation rather than assuming perfect scaling. More memory controllers may help scattered MoE expert reads, but expert-routing kernels can leave theoretical bandwidth unused. A first-week result may also improve after MLX or llama.cpp updates. Do not purchase against the best predicted line and then benchmark against a different quant, context or backend.

Understand the CUDA trade-off before choosing a Mac

Apple silicon's advantage is not simply speed. Unified memory lets one compact, quiet machine expose far more model memory than a 24GB consumer GPU. MLX is open source and designed for Apple silicon, and Metal support continues to improve. The Mac is attractive for private deskside inference, development and always-on agents where power, noise and a large shared memory pool matter.

NVIDIA retains a powerful software advantage. Many inference engines, quantisation formats and new-model optimisations are implemented for CUDA first. The RTX 4090 architecture specification lists 24GB of GDDR6X at 1,008GB/s: far less capacity than a high-memory Mac, but enormous bandwidth and mature kernels for a 27B model that fits. A pair of used 24GB GPUs can shard a 70B quant, at the cost of a larger, louder and more operationally complex system.

DGX Spark occupies the middle. NVIDIA's hardware guide specifies 128GB of coherent unified memory at 273GB/s, with the CUDA software stack and support for linking systems. One Spark offers the capacity class of a 128GB Max but much lower raw memory bandwidth; application kernels, low-precision formats and multi-device recipes can still make the end result competitive for supported models.

The practical comparison is workload-shaped:

• choose NVIDIA when CUDA-only support, established low-precision kernels, training or maximum speed on a model that fits in VRAM dominates;
• choose a high-memory Mac when one quiet box must hold a larger quant locally and Metal support is verified;
• choose Spark when CUDA compatibility and a 128GB coherent pool matter more than peak bandwidth; and
• use cloud capacity when demand is intermittent, hardware would sit idle or the target model will change before the purchase pays back.

Local processing can improve control over prompts and files, but hardware ownership is not a complete privacy design. The private AI operating guide-uk) covers access, logging, retention and supplier boundaries that still apply on premises.

Benchmark the decision, not the box

Do not preorder a configuration solely because its memory can load a headline model. Write a one-page acceptance test first.

• Pin the exact model revision, quant file, runtime and commit.
• Use representative prompts, context lengths, modalities and output types.
• Record cold model-load time, prefill rate, time to first token and decode rate separately.
• Measure useful context before memory pressure, swap or instability appears.
• Compare the chosen quant with a higher-precision reference on task outcomes, not vibes.
• Test one user and the expected concurrency; report both per-stream and aggregate throughput.
• Run long enough to expose thermal, memory and session-length failures.
• Repeat after the first relevant runtime update, while retaining the original receipt.

The purchase gate should name a minimum accepted outcome, maximum time to first token, minimum sustained decode rate, required context, concurrency, power and total configured price. Reject a machine that misses any non-negotiable condition even if one benchmark chart looks exceptional.

The practical preorder verdict

For an 8B-class workload, buy the Mac you otherwise need and treat local AI as a bonus. For a dedicated 27B host, start serious consideration at 48GB M5 Pro and prefer 64GB when context or other services will grow. For 70B, 64GB is the tight floor; 128GB M5 Max is the safer balanced choice. For the new 125B-to-320B sparse models, wait for stable Mac quants and measured receipts before committing to the 256GB Ultra. The 512GB Ultra is justified only by a named model and workload that cannot fit lower down the range.

The chip name tells you how quickly a compatible stack may move data. Memory tells you whether the work exists on the machine at all. Buy for a model you have tested, at a quant you accept, with a context and speed you can actually use—not for the resale value of this week's largest local-AI demo.