Janus

A unified multimodal understanding and generation model. Janus is a novel autoregressive framework that unifies multimodal comprehension and production. It addresses the limitations of previous approaches by decoupling visual encoding into separate pathways while still utilizing a single unified transformer architecture for processing. This decoupling not only alleviates the conflict between the roles of the visual encoder in understanding and generation but also enhances the framework's flexibility. Janus surpasses prior unified models and outperforms task-specific models in performance. Its simplicity, high flexibility, and effectiveness make it a strong candidate for the next generation of unified multimodal models. JanusFlow introduces a minimalist architecture that integrates autoregressive language models with rectified flows, a cutting-edge method in generative modeling. Our key finding demonstrates that rectified flows can be trained directly within the framework of large language models, eliminating the need for complex architectural modifications. Extensive experiments show that JanusFlow performs comparably or superiorly to specialized models in their respective domains while significantly outperforming existing unified approaches across standard benchmarks. This work represents a step toward more efficient and versatile vision-language models. Janus-Pro is an advanced version of the earlier Janus model. Specifically, Janus-Pro incorporates optimized training strategies, expanded training data, and scaling to larger model sizes. With these improvements, Janus-Pro achieves significant advancements in both multimodal understanding and text-to-image following capabilities, while also enhancing the stability of text-to-image generation.

Janus is a unified multimodal understanding and generation model that overcomes the limitations of traditional approaches by decoupling the visual encoding pathway, enhancing both flexibility and performance, enabling the comprehension and creation of multimodal content.