低端设备需要小内存、低功耗的本地AI推理模型

来源证据

全球超过210亿的物联网连接设备中，约15亿是高端设备；新兴市场大多数手机低于200美元、无NPU、使用廉价GPU，这些设备包括树莓派、微控制器、可穿戴设备、小型机器人和智能家居。

just 1.5 billion of over 21 billion connected IoT devices in the world today, and in emerging markets most phones ship under $200, no NPU, cheap GPUs. These include budget phones, Raspberry Pis, microcontrollers, wearables, small robots like Reachy Mini, and connected home devices. A conventional transformer of Needle's width and depth spends 164 MFLOPs per token, and even one squeezed down to Needle's parameter count spends 87, Needle spends 70. Even on a high-end phone, an always-on assistant