CM3leon by Meta

Introducing CM3leon, the ultimate generative model that revolutionizes text-to-image and image-to-text generation. This cutting-edge multimodal model combines the best of autoregressive models with remarkable training efficiency and inference speed.

CM3leon is meticulously trained using advanced techniques adapted from text-only language models, such as retrieval-augmented pre-training and multitask supervised fine-tuning. With just a fraction of the computational power required by previous transformer-based methods, CM3leon achieves state-of-the-art performance in text-to-image generation.

Unlike its predecessors, CM3leon is not limited to a single mode of generation. It excels at producing coherent sequences of text and images based on any combination of input content. In fact, CM3leon has surpassed Google's text-to-image model and set a new benchmark with its impressive Fréchet Inception Distance (FID) score of 4.88.

CM3leon's true capabilities shine in complex object generation and text-guided image editing tasks. With ease, it produces imagery that adheres to input prompts, even while incorporating constraints and compositional structures. Additionally, CM3leon performs exceptionally well in text-guided image editing, text-to-image generation with compositional prompts, and accurately answering questions about images.

Despite being trained on a relatively small dataset, CM3leon's zero-shot performance rivals that of larger models trained on extensive datasets. Its success demonstrates the power of retrieval augmentation and the impact of scaling strategies on autoregressive model performance. With its versatility, impeccable performance, and top-notch results, CM3leon is undoubtedly a game-changer for various vision-language tasks.