AI speech synthesis

LLaMA-Omni is a low-latency, high-quality end-to-end voice interaction model built on Llama-3.1-8B-Instruct, aiming to achieve GPT-4o level voice capabilities. The model supports low-latency voice interaction and is able to generate text and speech responses simultaneously. It completed training in less than 3 days using only 4 GPUs, demonstrating its efficient training capabilities.

EVI 2 is a new basic speech-to-speech model launched by Hume AI, which can have smooth conversations with users in a natural way close to humans. It has the ability to respond quickly, understand user intonation, generate different intonations, and perform specific requests. EVI 2 has enhanced emotional intelligence through special training to predict and adapt to user preferences, maintaining a fun and engaging character and personality. In addition, EVI 2 also has multi-language capabilities and can adapt to different application scenarios and user needs.

The Xinchen Lingo speech model is an advanced artificial intelligence speech model that focuses on providing efficient and accurate speech recognition and processing services. It can understand and process natural language, making human-computer interaction smoother and more natural. The model relies on Xihu Xinchen’s powerful AI technology and is committed to providing high-quality voice interaction experience in various scenarios.

SpeechGPT2 is an end-to-end speech conversation language model developed by the School of Computer Science at Fudan University, capable of perceiving and expressing emotions and providing appropriate speech responses in multiple styles based on context and human instructions. The model uses an ultra-low bitrate speech codec (750bps), simulates semantic and acoustic information, and is initialized with a multiple-input multiple-output language model (MIMO-LM). Currently, SpeechGPT2 is still a turn-based dialogue system, a full-duplex real-time version is being developed, and some promising progress has been made. Although limited by computing and data resources, SpeechGPT2 still has shortcomings in noise robustness for speech understanding and sound quality stability for speech generation. It plans to open source technical reports, code and model weights in the future.

Hume AI’s Empathic Voice Interface (EVI) is an API driven by the Empathic Large Language Model (eLLM), which can understand and simulate speech pitch, word accent, etc. to optimize human-computer interaction. It is based on more than 10 years of research, millions of patent data points and more than 30 papers published in top journals. EVI aims to provide a more natural and compassionate voice interface for any application, making people's interactions with AI more humane. This technology can be widely used in sales/meeting analysis, health and wellness, AI research services, social networks and other fields.