AI game creation

GameNGen is a game engine driven entirely by neural models, enabling real-time interaction with complex environments and maintaining high quality over long-term trajectories. It can interactively simulate the classic game "DOOM" at more than 20 frames per second, and its next frame prediction PSNR reaches 29.4, which is comparable to lossy JPEG compression. Human evaluators were only slightly better than chance at distinguishing between gameplay clips and simulation clips. GameNGen is trained through two stages: (1) an RL-agent learns to play the game and records the actions and observations of the training session, which becomes the training data for the generative model; (2) a diffusion model is trained to predict the next frame, conditioned on the past sequence of actions and observations. Conditional boosting allows stable autoregressive generation over long time trajectories.