BitNet

BitNet is an official inference framework developed by Microsoft and designed for 1-bit large language models (LLMs). It provides a set of optimized cores that support fast and lossless 1.58-bit model inference on the CPU (NPU and GPU support coming soon). BitNet achieved a speed increase of 1.37 times to 5.07 times on ARM CPU, and the energy efficiency ratio increased by 55.4% to 70.0%. On x86 CPUs, the speed improvement ranges from 2.37 times to 6.17 times, and the energy efficiency ratio increases by 71.9% to 82.2%. In addition, BitNet is able to run the 100B parameter BitNet b1.58 model on a single CPU, achieving inference speeds close to human reading speed, broadening the possibility of running large language models on local devices.

The target audience is developers, data scientists, and machine learning engineers, especially those who have high requirements for large-scale language model inference performance. BitNet promotes the development and application of natural language processing technology by providing an optimized inference framework that enables these professionals to efficiently run and test large-scale language models in resource-constrained environments, such as personal computers or mobile devices.