AI digital human

JoyHallo is a digital human model designed for Mandarin video generation. It created the jdh-Hallo dataset by collecting 29 hours of Mandarin videos from employees of JD Health International Co., Ltd. The dataset covers different ages and speaking styles, including conversational and professional medical topics. The JoyHallo model uses the Chinese wav2vec2 model for audio feature embedding, and proposes a semi-decoupled structure to capture the interrelationships between lips, expressions and gesture features, improving information utilization efficiency and speeding up inference by 14.3%. In addition, JoyHallo also performs well in generating English videos, demonstrating excellent cross-language generation capabilities.

metahuman-stream is an open source real-time interactive digital human model project. It uses advanced technology to realize synchronous audio and video dialogue between digital humans and users, and has commercial application potential. The project supports a variety of digital human models, including ernerf, musetalk, wav2lip, etc., and has functions such as voice cloning, digital human speech interruption, and full-body video splicing.

HeyGen Interactive Avatar is an online AI video generator that focuses on creating and optimizing avatar videos, supporting real-time interaction. It allows users to create an avatar optimized for continuous streaming, while reminding users to maintain minimal head and hand movements. HeyGen's background includes collaborations with big names like Baron David and Ryan Hoover, and the product is currently in beta testing with a free trial available.