AI image generation

Text-to-image diffusion model is a type of deep generative model that exhibits excellent image generation capabilities. However, these models are susceptible to implicit biases from web-scale text-image training pairs and may not accurately model the image aspects we care about. This can lead to suboptimal samples, model bias, and images that are inconsistent with human ethics and preferences. This paper introduces an efficient and scalable algorithm that leverages reinforcement learning (RL) to improve diffusion models covering diverse reward functions such as human preference, compositionality, and fairness, covering millions of images. We illustrate how our approach substantially outperforms existing methods, aligning diffusion models with human preferences. We further illustrate how this significantly improves the pre-trained Stable Diffusion (SD) model, generating samples that are 80.3% preferred by humans, while improving the composition and diversity of the generated samples.