AI language model

This product is a self-rewarding language model that uses LLM as a referee and uses the reward signal provided by the model itself for training. Through iterative DPO training, the model not only improves its ability to follow instructions, but also provides high-quality self-rewards. After three iterations of fine-tuning, the product outperformed many existing systems on the AlpacaEval 2.0 rankings, including Claude 2, Gemini Pro, and GPT-4 0613. This work, although preliminary, opens the door to the possibility of continued improvement of the model in two aspects.

Beagle14-7B is a powerful Chinese language model that can be used for various natural language processing tasks. It is based on the merging of multiple pre-trained models and contains rich language knowledge and expression capabilities. Beagle14-7B has efficient text generation capabilities and accurate semantic understanding capabilities, and can be widely used in chat robots, text generation, summary extraction and other tasks. For pricing information on Beagle14-7B, please visit the official website for details.

Lemur is an open language model designed to provide language agents with optimized natural language and coding capabilities. It balances natural language and coding skills to enable agents to follow instructions, reason about tasks and take practical action. Lemur combines the advantages of natural language and coding to produce state-of-the-art performance on different language and coding benchmarks through two-stage training, surpassing other available open source models and closing the gap in agent capabilities between open source and commercial models.

StreamingLLM is an efficient language model capable of handling infinite length inputs without sacrificing efficiency and performance. It works by retaining recent tokens and attention pooling, discarding intermediate tokens, thereby enabling the model to generate coherent text from recent tokens without the need for cache resets. The advantage of StreamingLLM is the ability to generate responses from recent conversations without relying on past data without flushing the cache.

LongLLaMA is a large language model capable of processing long texts. It is based on OpenLLaMA and fine-tuned using the Focused Transformer (FoT) method. It is capable of handling text up to 256k marks and more. We provide a smaller 3B base model (without instruction tuning) and inference code supporting longer contexts on Hugging Face. Our model weights can serve as a replacement for LLaMA in existing implementations (for short contexts up to 2048 tokens). Additionally, we provide evaluation results and comparisons with the original OpenLLaMA model.