This talk focuses on practical deployment experience of local large language models on high-performance chips such as the Apple M5 Max. It covers key hardware-selection factors — like how unified memory capacity and bandwidth affect 27B models — compares quantization schemes under the MLX framework, and shares real benchmark data on tuning techniques such as ANE acceleration and MTP speculative decoding. Through real benchmarks across different configurations, it shows the differences in prefill and generation speed, helping developers quickly find an efficient local inference setup that balances performance and stability for their own devices.
Practical Local LLM Deployment
LightningAbstract
Real-world deployment experience on high-performance chips like the Apple M5 Max, covering hardware selection, quantization trade-offs, and tuning benchmarks.