Apple releases 3B on-device and larger server foundation models with quantization and adapter-based specialization, claims performance parity with 7B-70B competitors on human evaluation

Apple disclosed its foundational language models powering Apple Intelligence at WWDC 2024: a 3-billion-parameter on-device model optimized for iPhone 15 Pro (0.6 millisecond first-token latency, 30 tokens per second) via 3.7 bits-per-weight quantization, and a larger server-based model on private cloud compute. On human-evaluated tasks (brainstorming, QA, coding, math, safety), the on-device model outperformed larger open-source models (Mistral-7B, Gemma-7B, Llama-3-8B) and compared favorably to GPT-3.5 and Llama-3-70B. The design uses task-specific LoRA adapters (10s of MB per adapter) dynamically loaded on device to specialize the base model. This signals that aggressive quantization and adapter-based fine-tuning can achieve frontier-grade single-task performance at substantially smaller model sizes, and that on-device inference at real-time latencies is now viable for consumer hardware without cloud dependency for most features.