GameGen-X

GameGen-X is a diffusion transformation model designed for generating and interactively controlling open-world game videos. This model achieves high-quality, open-domain video generation by simulating various features of game engines, such as innovative characters, dynamic environments, complex actions, and diverse events. In addition, it provides interactive control capabilities to predict and change future content based on current video clips, enabling gameplay simulation. To realize this vision, we first collected and built from scratch an Open World Video Game Dataset (OGameData), the first and largest open world game video generation and control dataset, containing more than 1 million diverse gameplay video clips from more than 150 games, all equipped with informative subtitles from GPT-4o. GameGen-X has gone through a two-stage training process, including basic model pre-training and instruction tuning. First, the model is pre-trained with text-to-video generation and video sequels, giving it the ability to generate long-sequence, high-quality open-domain game videos. Further, in order to achieve interactive control capabilities, we designed InstructNet to integrate game-related multi-modal control signal experts. This enables the model to adapt latent representations based on user input, unifying character interaction and scene content control in video generation for the first time. During instruction tuning, only InstructNet is updated, while the pre-trained base model is frozen, allowing the integration of interactive control capabilities without losing the diversity and quality of the generated video content. GameGen-X represents a major leap forward in using generative models for open-world video game design. It demonstrates the potential of generative models as an adjunct to traditional rendering techniques, effectively combining creative generation with interactive capabilities.

The target audience is game developers, AI researchers, and video game enthusiasts. GameGen-X is suitable for them because it provides a powerful tool for generating and controlling open-world game videos, which can help them achieve more innovation and interactivity in the game design and development process.