AI text to video

text2video is a tool that can automatically convert text to video. It uses technologies such as stable-diffusion and edge-tts, which can segment text to generate images and voices, and synthesize them into videos with subtitles and dubbing through ffmpeg. The original intention of this tool is to realize the visual reading function of novels, which can help users read text content more vividly. The tool is free to use and can be customized according to personal needs.

CustomVideo is a novel framework designed to generate videos that maintain identity guided by multiple themes. The product first encourages the co-occurrence of multiple topics, and then designs a simple yet effective attention control strategy through a basic text-to-video diffusion model to unravel different topics in the latent space of the diffusion model. Additionally, the product helps the model focus on specific object regions by segmenting objects from a given reference image and providing corresponding object masks for attention learning. At the same time, they also collected a multi-topic text-to-video generation dataset as a comprehensive benchmark, which includes 69 individual topics and 57 meaningful pairs. Extensive qualitative, quantitative, and user study results demonstrate that our approach has significant advantages over previous state-of-the-art methods.