Cohere launches Aya Vision: Multilingual AI models now support 23 languages

Cohere For AI’s new Aya Vision models represent a significant breakthrough in multilingual capabilities for vision-language AI systems. The 8B and 32B parameter models support 23 languages, addressing a critical gap in the AI landscape where most vision models have focused primarily on English content. This advancement could democratize access to multimodal AI technologies across diverse linguistic communities and applications, potentially transforming how visual content is processed across global markets.

The big picture: Cohere For AI has released Aya Vision, a new family of open-weight vision-language models designed to bring robust multilingual capabilities to multimodal AI systems.

• The models build upon the foundation established by the company’s Aya Expanse multilingual language models while extending capabilities to visual content understanding across 23 languages.
• Available in both 8B and 32B parameter versions, Aya Vision represents a targeted effort to address language diversity in the rapidly evolving multimodal AI space.

Key technical details: Aya Vision employs SigLIP2-patch14-384 as its vision encoder and incorporates advanced processing techniques for efficient image handling.

• The architecture uses dynamic image resizing and Pixel Shuffle for optimization, striking a balance between performance and computational efficiency.
• Training followed a two-stage approach: vision-language alignment followed by supervised fine-tuning, with special attention to comprehensive language coverage.

Innovative approaches: The development team implemented several novel techniques to enhance multilingual performance in visual contexts.

• Synthetic data annotations and translation processes were employed to overcome limitations in available training data for non-English languages.
• The team leveraged multimodal model merging methods to efficiently scale capabilities from the 8B to 32B parameter versions.

Performance highlights: The models demonstrate strong capabilities across various vision-language tasks while maintaining multilingual support.

• Aya Vision excels in image captioning, visual question answering, and text generation tasks that involve visual reasoning.
• The models outperform several larger competitors specifically in multilingual evaluations, suggesting efficient architecture design.

Practical applications: Users can access and implement Aya Vision through several channels for immediate experimentation.

• The models and associated resources, including the new AyaVisionBench evaluation dataset, are available through Hugging Face repositories.
• Practical implementations include a Hugging Face Space demo, WhatsApp integration, and a Colab example to help users get started quickly.

Why this matters: Bringing robust multilingual capabilities to multimodal AI addresses a significant equity gap in current vision-language technologies.

• Most existing vision-language models have focused primarily on English, limiting access and utility for billions of non-English speakers worldwide.
• Democratizing these capabilities could accelerate AI adoption and application development across diverse global communities and markets.