saved
Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices
Hraness wrote this summary from a saved copy of the source. Quotations are taken word for word from the source.
gist
Apple engineers describe Minions, a system that pools idle Apple Silicon GPUs on employees' Macs into a virtual GPU cloud for embedding and LLM inference. Executors pull subtasks from a server-backed PostgreSQL queue, a watchdog reclaims work from sleeping or offline devices, and GPU queue times drop from hours to minutes.
ideas
- Idle Macs are untapped GPUs. Apple Silicon devices sit mostly idle, so pooling them avoids new GPU procurement when central clusters are oversubscribed.
- Pull, don't schedule. Executors fetch work through REST APIs and claim subtasks with a unique-constraint insert in PostgreSQL, avoiding a central scheduler.
- Expect churn. Heartbeats every five minutes let a watchdog return subtasks from devices that sleep, lose network, or get reclaimed by their owners.
- Owners steer their compute. Dynamic task binding lets device owners prioritize a team or task before falling back to the global FIFO queue.
- Trust limits adoption. Data-confidentiality worries led to team- or organization-scoped deployments with dedicated databases and executor pools.
quotes
“modern Apple Silicon–based macOS devices are equipped with powerful GPUs that remain largely idle during normal usage.”
“Minions adopts a pull-based execution model and intentionally avoids a centralized scheduler.”
“With Minions, tasks can often begin execution within minutes by leveraging idle macOS devices.”