Introducing the Third Generation of Apple’s Foundation Models

gist

Apple’s third-generation Foundation Models include a 20B on-device model that keeps its full weights in flash and loads experts per prompt. AFM 3 Core is a 3B dense model; AFM 3 Core Advanced, built with Google like the rest of the family, is natively multimodal and activates 1 to 4 billion parameters depending on the request, alongside three server models on Private Cloud Compute.

ideas

• Flash holds the model, DRAM holds the working set. Instead of keeping all weights in DRAM, AFM 3 Core Advanced stores the full model in NAND and assembles a dense model in memory from selected experts plus shared weights.
• Routing is per prompt, not per token. NAND-to-DRAM bandwidth is too slow for token-by-token expert swaps, so a small dense block picks a fixed set of experts during initial processing and periodically reselects them during generation.
• Shared experts keep data movement low. A high share of always-active shared experts sits beside input-dependent routed experts that are swapped into DRAM only when needed.
• Model size becomes elastic. Each use case gets a set number of active parameters, so weights load incrementally across requests of different difficulty, scaling beyond DRAM limits; the technique builds on Apple’s Instruction-Following Pruning research.

quotes