Mac mini clusters over Thunderbolt 4

Distributed inference is bandwidth-bound. Every token has to cross the link between nodes, so the interconnect decides whether a cluster is fast or merely large.

Thunderbolt 4 gives each link 40 Gb/s in both directions, forty times gigabit Ethernet, and enough that the machines stop waiting on each other. That is the whole reason to cable a cluster instead of putting the Macs on a switch and hoping.

The second reason is memory. Apple Silicon shares memory between CPU and GPU, so a node's full RAM is available to a model. Across a cluster, that pools:

Exact totals depend on the per-node configuration you choose across M4 and M6 machines, each up to 32 GB.

Models too large for one machine, sharded across nodes with MLX distributed, EXO or llama.cpp RPC. This is the case where the Thunderbolt link earns its keep, and where you would otherwise be renting datacentre GPUs by the hour.

Four or eight machines in one rack, one account, builds distributed across them. Same hardware, different job.

Frame or shot distribution across nodes, with the finished output pulled back over the same link.

Several AI agents running unattended, each with its own machine, sharing a private network and one point of contact.

Pooled memory is the constraint that decides node count. Tell us the model and quantisation you intend to run and we will tell you the configuration that fits, rather than selling you a node you do not need.

A cluster is the right answer when a model does not fit on one machine. If it does fit, one bigger machine will beat two smaller ones every time, and we will say so.

How nodes are cabled changes throughput. We build ring topologies for sharded inference and hub arrangements for job distribution, and set it up for the workload you describe.

Tell us what you want to run and how much memory it needs. We size the cluster, build it, test it, and hand it over working.