GITHUB 🤗 Hugging Face｜ 🤖 ModelScope

The Introduction Video of Ming-Omni-TTS​

🚀 Featured Abilities​

Ming-omni-tts is a high-performance unified audio generation model that achieves precise control over speech attributes and enables single-channel synthesis of speech, environmental sounds, and music. Powered by a custom 12.5Hz continuous tokenizer and Patch-by-Patch compression, it delivers competitive inference efficiency (3.1Hz). Additionally, the model features robust text normalization capabilities for the accurate and natural narration of complex mathematical and chemical expressions.

• 🔊 Fine-grained Vocal Control: Enables precise control over speech rate, pitch, volume, emotion, and dialects via simple instructions. It achieves 93% accuracy for Cantonese and 46.7% for emotional control, outperforming CosyVoice3.
• 🌌 Intelligent Voice Design: Features 100+ premium built-in voices and supports zero-shot voice design through natural language descriptions. Its performance on the Instruct-TTS-Eval-zh benchmark is on par with Qwen3-TTS.
• 🎶 Immersive Unified Generation: The industry's first autoregressive model to jointly generate speech, ambient sound, and music in a single channel. Built on a custom 12.5Hz continuous tokenizer and a DiT head architecture, it delivers a seamless, "in-the-scene" auditory experience.
• ⚡ High-efficiency Inference: Introduces a "Patch-by-Patch" compression strategy that reduces the LLM inference frame rate to 3.1Hz. This significantly cuts latency and enables podcast-style audio generation while preserving naturalness and audio detail.
• 🧪 Professional Text Normalization: The model accurately parses and narrates complex formats, including mathematical expressions and chemical equations, ensuring natural-sounding output for specialized applications.

Model Structure​

Ming-omni-tts is a unified audio language model for the generation of speech, music, and sound, based on a unified continuous audio tokenizer.

Unified Continuous Audio Tokenizer.​

Unified Audio Language Model for Speech, Music and Sound Generation.​

Benchmark Evaluations​

Voice Control – Support Structured and Natural Command Control​

Basic Attributes Control: Speed, Volume and Pitch Control for Voice Generating​

Same Dialect/Cross-Dialect Control: Generating Cantonese and Sichuanese from Mandarin and Native Prompts​

Cross-Emotion Control: Cross-Emotion Synthesis Using a Single Neutral Prompt​

Built-in Premium Sounds: Over 100 Built-in, High-Quality Timbres​

Voice Design: Zero-Shot Synthesis of Custom Vocal Identities via Natural Language Descriptions​

Podcast: Multi-person Conversation​

Music Generation​

Speech/Music Mono Generation: Single-Channel Generation of Speech and Music​

Sound Generation(TTA)​

Speech/Sound Mono Generation: Single-Channel Generation of Speech and Sound​

GITHUB ARXIV 🤗 Hugging Face｜ 🤖 ModelScope

Omnimodal Ming-omni series update! Ming-flash-omni-Preview is the first open-source omnimodal large model with a parameter scale reaching the hundred billion-Scale level. Based on Ling 2.0's sparse MoE architecture, Ming-flash-omni-Preview has a total of 103B parameters with 9B activated. Compared to the previous version Ming-lite-omni-1.5, Ming-flash-omni-Preview has improved in both omnimodal understanding and generation capabilities. The overall performance across various modalities has reached a leading level among open-source omnimodal models, with particularly outstanding performance in controllable image generation, streaming video understanding, and speech recognition.

Capability Overview​

Controllable Image Generation​

For image generation, Ming-flash-omni-Preview pioneers the Generative Segmentation Paradigm, reframing "image segmentation" as a semantic-preserving editing task (Generative Segmentation-as-Editing), achieving fine-grained spatial semantic control. Ming-flash-omni-Preview achieved a score of 0.90 on the GenEval benchmark, surpassing all non-reinforcement learning generation methods and demonstrating exceptional controllability.

Streaming Video Understanding​

Users often have a need to engage in continuous dialogue with AI based on real-world scenarios and to use AI to understand those scenarios. Ming-flash-omni-Preview can effectively fulfill these needs. As shown in the video below, Ming-flash-omni-Preview can achieve fine-grained understanding of streaming video, recognizing objects and interactions within the video, and providing relevant understanding and explanations in real-time to support users in practical scenarios.

Speech and Dialect Understanding​

Ming-flash-omni-Preview can achieve Context-Aware Speech Recognition (ContextASR) and dialect recognition, achieving SOTA across all 12 ContextASR subtasks. Its understanding ability for 15 Chinese dialects, including Hunanese, Minnanese, and Cantonese, is significantly enhanced, effectively providing translation and real-time understanding support for users who might be lost in an unfamiliar dialect.

Voice Cloning​

Ming-flash-omni-Preview's speech generation has been upgraded from discrete tokenizers to continuous tokenizers, significantly enhancing voice cloning capabilities. It exhibits high stability in mixed Chinese-English pronunciation, and can effectively clone the voice from the original conversation into newly generated dialogue. The seed-tts-zh WER metric is 0.99, surpassing Qwen3-omni and seed-tts.

Model Architecture and Capability Introduction​

Model structure diagram of Ming-flash-omni-Preview:

Compared to Ming-lite-omni-1.5, Ming-flash-omni-Preview primarily features the following technical optimizations:

Omnimodal Training Based on Sparse Expert Architecture​

Ming-flash-omni-Preview extends the Ling-Flash-2.0 sparse MoE architecture to the omni-modality. It models the distribution and routing strategy of each modality based on the modality-level routing proposed by Ming-lite-omni, achieving "large capacity, small activation" for each modality. By introducing VideoRoPE in the Attention layer, it enhances spatiotemporal modeling for long videos, improving video interaction capability. Additionally, in terms of training strategy:

1. Stable Sparse Training: Utilizes a mixed expert balancing scheme (combining auxiliary load balancing loss with router bias updates) to ensure uniform activation and convergence of omnimodal training under the sparse MoE architecture;
2. Context-Aware ASR Training Paradigm: For speech training tasks, task/domain information input is used as the decoding condition, significantly improving proper noun recognition and transcription consistency. It also introduces high-quality dialect training corpora, leading to a significant increase in recognition accuracy for 15 Chinese dialects, including Hunanese, Minnanese, and Cantonese.

Unified Generative Segmentation and Editing​

The core challenge in building a unified multimodal model lies in how to efficiently integrate image understanding and generation capabilities. Our Ming-lite-omni-1.5 achieved this by freezing the language pathway and injecting hierarchical semantics using multi-scale QueryTokens, thereby allowing the generation objective to better integrate with the understanding task while preserving understanding performance. Although this training strategy improved stability, the fundamental differences between the learning objectives of understanding and generation mean that even with the introduction of hierarchical semantics, fine-grained visual knowledge (such as object attributes and spatial relationships) remains difficult to efficiently transfer to high-precision generation and editing tasks, thus limiting the improvement in model generation quality and controllability.

To overcome this bottleneck, Ming-flash-omni-Preview proposes the "Generative Segmentation-as-Editing" collaborative training paradigm. This paradigm reframes image segmentation as a semantic-preserving editing task (e.g., "paint the banana purple"). The key assistance provided by this design is: It forcibly unifies the understanding and generation objectives — successful editing must rely on precise understanding of the object's outline, and the editing quality directly provides supervision signals for understanding. This paradigm directly enhances the model's fine-grained spatiotemporal semantic control ability and indirectly solves the compositionality problem in pure text-to-image generation.

On the GenEval benchmark, Ming-flash-omni-Preview achieved a score of 0.90, surpassing all leading non-reinforcement learning (non-RL) methods; On the GEdit benchmark, the average score for precise editing tasks such as object deletion and object replacement improved from 6.9 to 7.9. These two results collectively prove that the fine-grained spatiotemporal semantic control capability gained through the "Generative Segmentation-as-Editing" training not only significantly improves performance in precise editing tasks but can also effectively generalize to pure text-driven image generation tasks.

Efficient Omnimodal Training Architecture​

Training omnimodal foundation models faces two major challenges: data heterogeneity (varied shapes of multi-modal inputs) and model heterogeneity (difficulty in parallelizing modality-specific encoders). These issues lead to load imbalance, memory fragmentation, and pipeline bubbles, severely slowing down the training speed. To address these problems, we made two key optimizations based on the Megatron-LM framework when training the Ming-flash-omni-Preview model:

1. Sequence Packing: Solves data heterogeneity. Varied-length sequences are densely packed into fixed-length batches, significantly improving memory utilization and computational density;
2. Flexible Encoder Sharding: Solves model heterogeneity. Extends Megatron-LM to support fine-grained sharding of modality encoders across DP/PP/TP, eliminating pipeline bubbles and achieving load balancing. These optimization measures resulted in a doubling of the training throughput of Ming-flash-omni-Preview compared to the baseline.

Getting Started with Ming-flash-omni-Preview​

Our model and code are open source. We welcome everyone to try, provide feedback, and exchange ideas:

• GitHub: https://github.com/inclusionAI/Ming
• Hugging Face: https://huggingface.co/inclusionAI/Ming-flash-omni-Preview
• ModelScope: https://www.modelscope.cn/models/inclusionAI/Ming-flash-omni-Preview

Future Plan​

The version released this time is the Ming-flash-omni-Preview, and the current version has some imperfections:

1. Visual-Text Understanding Capability: Although Ming-flash-omni-Preview's overall performance is leading among omnimodal models, there is still a gap compared to SOTA dedicated VL large models. We will continue to explore the performance upper limit of omnimodal models.
2. Speech Capability: Overall performance in speech recognition and speech synthesis is leading. The effects of multi-turn speech dialogue and high-quality voice cloning are our next optimization priorities.
3. Image Generation Capability: The model achieved a score of 0.90 on the GenEval benchmark, demonstrating good controllability, and already possesses text generation and editing capabilities. However, there is still room for improvement in rendering and editing text with complex layouts, as well as generating specific IP characters.

We are continuously optimizing the user experience of Ming-flash-omni-Preview. We welcome you to provide feedback via community discussion or issues. The official version will be released soon.

GITHUB 🤗 Hugging Face｜ 🤖 ModelScope

The Introduction Video of Ming-UniAudio​

Audio Edit Demo​

Editing Tasks Video demos​

🚀 Technical Highlights​

1. First unified continuous speech tokenizer for both understanding and generation tasks: MingTok-Audio is a unified continuous speech tokenizer MingTok-Audio based on a VAE framework with a causal Transformer architecture, the first continuous speech tokenizer to effectively integrate semantic and acoustic features, and enables a closed-loop system with LLMs through hierarchical feature representations, makes it suitable for both understanding and generation tasks.
2. First Speech LLM with unifed continuous tokenizer for both understanding and generation: Ming-UniAudio is an end-to-end unified speech language model with a single LLM backbone for both understanding and generation tasks, enhanced with a Diffusion Head to ensure high-fidelity speech synthesis.
3. First universal free-form speech editing model for semantic and acoustic tasks without temporal regime: We introduce the first instruction-guided, free-form speech editing framework that supports comprehensive semantic and acoustic edits without requiring explicit edit regions, along with Ming-Freeform-Audio-Edit, the first open-source evaluation set for such tasks.
4. First benchmark for free-form speech editing: We propose Audio-Edit-Benchmark, the first open-source free-form evaluation set comprising editing tasks of four semantic and five acoustic types, to evaluate the model's editing performance.

Instruction-Guided Free-Form Speech Editing​

Semantic Editing - Insert​

Semantic Editing - Substitute​

Semantic Editing - Delete​

Acoustic Editing - Dialect Conversion​

Acoustic Editing - Speed​

Acoustic Editing - Pitch​

Acoustic Editing - Volume​

Acoustic Editing - Denoise​

Acoustic Editing - Background Music​

Acoustic Editing - Emotion Conversion​

Audio Understanding​

Chinese and English ASR​

Dialect Understanding​

Context ASR​

Audio Generation​

Voice Clone​

Multi-lingual Synthesis​

GITHUB 🤗 Hugging Face｜ 🤖 ModelScope

🚀 Technical Highlights​

1. First Continuous Unified Tokenizer for Vision: MingTok seamlessly supports both image understanding and generation within a single continuous latent space—eliminating quantization and bridging modalities.
2. First NTP-style Autoregressive MLLM with Unified Continuous Visual Tokens: By building on MingTok, Ming-UniVision unifies vision and language under a shared next-token prediction framework, enabling end-to-end autoregressive modeling of diverse vision tasks.
3. Reduced Representational Competition → 3.5× Faster Convergence: The unified continuous representation aligns semantic understanding and generative dynamics, significantly accelerating joint training without performance trade-offs.
4. Multi-Round In-Context Learning in a Single Feature Space: All operations—understanding, generation, and editing—occur in the same continuous space, eliminating costly cross-space conversions and enabling simpler, more efficient training and inference.

The Challenge: The Inverse Nature of Seeing and Drawing​

Autoregression—the powerful paradigm of modeling the world by “predicting the next token”—has already unified diverse modalities like language and audio. The next frontier is to bring visual understanding (seeing) and visual generation (drawing) into this unified sequence‑to‑sequence framework.

However, this ambition encounters a deep challenge: in many respects, understanding and generation are inverse tasks.

• Understanding: Pixels → high‑dimensional, abstract semantic concepts
• Generation: Concepts → fine‑grained, high‑fidelity pixels

These tasks have drastically different—and often competing—preferences for their underlying visual representation.

Why Previous Approaches Fell Short​

Existing models attempt unification via two limited strategies:

1. Asymmetric Designs: Use different, heterogeneous feature spaces for each task. During multi‑turn interactions, this forces inefficient “round‑trips” between spaces, causing latency and complexity.
2. Shared Discrete Tokens: Unify the token space but introduce quantization errors. This hurts image fidelity and degrades understanding capability.

Our Solution: Ming-UniVision and MingTok​

To break this impasse, we introduce Ming-UniVision, a new generation of autoregressive vision‑language model built on a foundational innovation: MingTok.

MingTok is the first visual tokenizer based on a continuous latent space. It delivers a truly unified and efficient representation that serves as the bedrock for Ming‑UniVision’s unified NTP (Next‑Token Prediction) framework—harmonizing image understanding, generation, and editing in one in‑context multimodal loop.

The Core Design: A Three-Stage Architecture to Reconcile Competition​

At the heart of Ming-UniVision is the MingTok tokenizer, a three-stage sequential architecture elegantly designed to reconcile the competing representational demands of understanding and generation within a single framework.

Figure 1: (a) Existing models use separate visual representations. (b) MingTok, the engine of Ming-UniVision, uses a unified scheme for both semantic and generative representations. (c) This unified approach leads to over 3.5x faster training convergence.

1. Low-level Encoder: Maps an input image into a sequence of compact, continuous latent codes, optimized for high-quality and efficient autoregressive generation.
2. Semantic Decoder: Autoregressively "refines" the compact latent codes into high-dimensional, rich semantic features aligned with top-tier understanding models like CLIP.
3. Pixel Decoder: Serves as a quality-assurance module, ensuring the original image can be reconstructed with high fidelity, guaranteeing a high-fidelity representation process.

The Key Innovation: MingTok creates a unified, differentiable interface. The high-level features for understanding can be directly fed as conditional input for the next round of generation or editing. This completely eliminates the costly detour through pixel space.

The Breakthrough: A Fundamental Leap in Efficiency​

By integrating MingTok, Ming-UniVision achieves competitive results on both understanding and generation tasks. The shared continuous latent space unlocks two fundamental layers of efficiency, resolving bottlenecks that have plagued previous architectures.

Figure 2: On general recognition tasks, our method approaches the performance of models with separated representations and significantly outperforms other unified representation models. For generation, our model shows a clear advantage on fine-grained tasks.

1. A Revolution in Training: >3.5x Faster Convergence​

Traditional approaches expend massive resources aligning heterogeneous representations, creating an intrinsic "task competition" that slows learning. MingTok solves this at its root.

• Synergistic Enhancement: Our ablation studies show that using MingTok for both tasks fosters a synergy where understanding and generation capabilities enhance each other, rather than competing.
• >3.5x Speedup: By avoiding inefficient alignment, the model focuses its energy on learning, reaching the same performance level in a fraction of the time compared to traditional schemes.

Figure 3: The performance drop between generation-only training and joint training is minimal with MingTok, proving the advantage of our unified approach.

2. A Revolution in Interaction: Goodbye to the "Pixel Round-Trip"​

The efficiency of multi-turn interactions (e.g., generate → edit → re-generate) depends on the "understanding-generation" loop. This is precisely where traditional architectures falter.

As the table shows, any architecture with separated spaces is doomed to the inefficient Latent → Pixel → Feature round-trip. This "pixel detour" introduces massive latency and causes contextual information to decay.

Ming-UniVision achieves a direct Feature → Feature closed loop. High-level features from an understanding task can be directly consumed by the next generation task, unlocking truly coherent multimodal sequence modeling. This enables tasks that once required multiple specialized models to emerge naturally within a single, unified framework:

• Iterative Image Enhancement: Perform super-resolution, then directly continue with colorization or denoising.
• Generative Chain-of-Thought: Perform an understanding task (e.g., "segment the car"), then directly apply an editing command to that region.

Figure 4: Multi-turn tasks like "Super-resolution → Colorization" and "Segmentation → Editing" are now part of a seamless flow.

Understanding, generation, and editing are no longer isolated pipelines but are woven into a continuous visual conversation.

Conclusion and The Road Ahead​

We believe that a unified and continuous visual representation like MingTok opens up new possibilities for building more flexible and intuitive multimodal interactive systems.

We know this is just one step in a long journey. We have open-sourced our code and initial model weights, hoping to provide a useful foundation for the community and to inspire more discussion around unified representations. We look forward to collaborating with our peers to collectively advance the future of multimodal AI.

Get Involved​

• GitHub: (https://github.com/inclusionAI/Ming-UniVision)
• Technical Report: (https://arxiv.org/pdf/2510.06590)
• Model link: Hugging Face｜ 🤖 ModelScope

Try out our open-source model Ming-UniVision and MingTok-Vision on our GitHub Page / Demo Page. Please star our repo if you like it!

GITHUB 🤗 Hugging Face｜ 🤖 ModelScope

The Hype and the Hidden Question​

The multimodal AI world has been thriving.

From the debut of Qwen-Image to the interactive editing hype sparked by Nano Banana, image editing has rapidly become the next battlefield for generative AI.

Editing fundamentally requires two distinct skill sets:

• Know where, what, and how to change (understanding the image)
• Produce the change with high visual quality (generating the image)

Its rich gameplay and strong interactivity have pulled in users, developers, and creators alike.

But behind the noise, few are asking:

Beneath this prosperity, how close are we to a truly unified “understanding + generation” AI?

Understanding and Generation: Two Hands, Often Out of Sync​

For years, we’ve chased an ambitious goal:

Build a unified multimodal model that understands the world like a scientist (e.g., image segmentation) while creating it like an artist (e.g., image editing).

In theory, these abilities should be mutually reinforcing:

“The deeper the understanding, the better the creation; the more the creation, the deeper the understanding.”

Reality is messier.

In AI today:

• Understanding = the left hand: precise abstractions, semantic reasoning, boundaries.
• Generation = the right hand: coherent pixels, style, aesthetics.

But training a model to recognize 10,000 cat photos doesn’t magically make it capable of painting cats, and painting cats repeatedly doesn’t make it understand cats better.

Worse, in multitask training, the two often compete for resources — optimizations for understanding can hurt generation, and vice versa.

We’re missing a catalyst: a task that forces the left and right hands to evolve together.

The Struggle: 16% Segmentation and Out-of-Control Generation​

Before finding our solution, our unified model was struggling with generative segmentation:

Given an instruction like “segment the banana in the upper-right corner”, we wanted the model to output a segmentation mask directly.

The results were painful.

On RefCOCO-val, our cIoU plateaued at ~16%.

The root cause is the distribution gap.

Generative models thrive on natural, continuous image distributions. Segmentation masks, however, are synthetic, abstract, binary maps — as unnatural as it gets for an image generator.

It was like asking a painter to draw an X-ray: doable, but far from their artistic instincts.

Here, generation wasn’t helping segmentation — it was tripping it up.

We needed a new task that:

1. Met the precision demands of understanding.
2. Played to the strengths of generation.

The “Aha” Moment: Dressing Segmentation in Color​

Here’s the analogy that unlocked it for us:

If you want a child to mark an object, is it easier to have them draw a tight outline with a pencil, or fill it in with bright colors?

Obviously, the latter.

Instead of forcing our model to output abstract black-and-white masks, we turned the segmentation task into a color-editing task.

Example:

• Instruction: “segment the banana in the upper-right”
• Old way: Output a mask ❌
• New way: Directly edit the image: “paint the banana purple”, “make the banana red”, etc. ✅

This brought the task’s data distribution back to the realm of natural images — where generative models shine.

Why This Works: The Hidden Catalyst​

That small twist turned out to be exactly the catalyst we’d been searching for.

• Boosting Understanding: To color the banana without bleeding outside the boundary, the model must internally nail pixel-perfect segmentation. The segmentation step became an implicit prerequisite to editing.

• Unleashing Generation: No more awkward synthetic masks — the model is doing what it knows best: image-to-image editing. All its strengths in shading, texture, and edge blending go into making the change look natural.

For the first time, the left hand and right hand weren’t fighting — they were helping each other.

The Numbers: From 16% to 72.4% — and Beyond​

1. SOTA-level Segmentation​

The cIoU score didn’t just improve — it soared from 16% to 72.4% on RefCOCO-val, a relative gain of over 350%.

Qualitatively, the model outperformed competitors in pinpointing and segmenting targets, even in reasoning-heavy cases.

Against Qwen-Image and Nano Banana, our model:

• Located small or occluded targets more reliably.
• Produced boundaries that were visually and semantically aligned with instructions.

Our model (right) accurately locates and segments the target subject. Qwen-Image (second from left) fails to locate the correct target, while Nano-banana (third from left) fails to accurately segment the man's head and has loose boundary lines.

For the prompt "please segment the girl with red mask," our model (right) is precise. Qwen-Image (second from left) misses the feet, and Nano-banana (third from left) alters the subject's proportions.

During evaluation, thanks to the high consistency of non-edited regions in our model, we can directly derive the segmentation mask by calculating the difference between the edited result and the original image.

The results show that our model's performance on segmentation is now on par with specialized vision models.

For each test set, Nano-banana and Qwen-Image-Edit was evaluated on a randomly sampled subset of 500 images, to reduce computational cost while preserving the key statistical trends. We observed that Nano-banana frequently fails to accurately grasp the image segmentation intent during inference, leading to its comparatively lower evaluation metrics. This may be attributed to differences in training objectives and data emphasis.

2. Sharper, More Controllable Editing​

The beauty of this method is that it not only fixed the segmentation weakness but also dramatically enhanced the model's general editing capabilities.

Because the model has learned an unprecedented "respect for boundaries" through thousands of "precise coloring" exercises, this "muscle memory" for fine-grained control has transferred to all editing tasks. Our edit controllability score saw a significant jump from 7.69 to 8.12 across sub-tasks like background, color, and material changes.

Prompt: "remove the bow tie of the man on the far right." Our model (right) precisely removes only the target bow tie while maintaining background consistency. Qwen (second from left) incorrectly removes multiple bow ties and introduces inconsistencies. Nano-banana (third from left) also struggles with consistency.

3. Stronger ID Consistency​

A core challenge in portrait editing is maintaining identity. Our model excels here as well. Whether changing a hairstyle or adjusting an expression, the model skillfully preserves the person's core features.

Top Row (Turn head): Our model (right) maintains ID and background consistency, unlike competitors. Middle Row (Smile): Our model (right) correctly follows the prompt while preserving ID, avoiding distortions seen in others. Bottom Row (Change background): Our model (right) excels at preserving the subject's ID and appearance during a background swap.

See More Editing Consistency in Action:

An Honest Look: Where We Can Still Improve​

Despite the leap forward, challenges remain:

• Large pose changes (e.g., standing → running) need more reliability.
• Multi-step or compound instructions require better parsing and execution.
• Instruction diversity support needs expansion.

These are our next milestones.

Takeaway: The Next Catalysts Are Out There​

From 16% to 72.4% — this wasn’t driven by a massive architecture overhaul or billion-image datasets.

It came from one change in task design.

The lesson: Instead of gluing capabilities together after the fact, find naturally cooperative tasks — where solving the problem requires multiple abilities to mesh seamlessly.

“Segmentation-as-editing” is just the first example.

We suspect 3D understanding, video generation, and other domains have their own hidden catalysts, waiting to be discovered.

At last, AI’s left and right hands have learned to high-five.

And this is only the overture.

Try out our open-source model Ming-lite-omni 1.5 on our GitHub Page / Demo Page. Please star our repo if you like it!

📖 Technical Report | 🤗 Hugging Face｜ 🤖 ModelScope

Overview​

We present Ring-lite-2507, an upgraded version of our previously released lightweight reasoning model, Ring-lite (2506). Built upon a 16.8B Mixture-of-Experts (MoE) large language model with 2.75B activated parameters, Ring-lite-2507 further advances its reasoning capabilities while demonstrating superior performance across a comprehensive range of LLM benchmarks, including general text understanding, alignment, coding, logical, and agentic tasks. Thanks to our innovative and robust reinforcement learning training pipeline, Ring-lite-2507 distinguishes itself from the latest public dense models under 10B parameters by offering competitive performance across various tasks, despite activating only 1/3 of their parameter size.

To address the optimization instability of MoE RL training, we propose a novel approach, Constrained Contextual Computation Policy Optimization(C3PO), which enhances training stability and improves computational throughput via algorithm-system co-design. Additionally, we systematically investigate the dynamic relationship between long chain-of-thought SFT and RL training. Rather than relying solely on validation metrics, we explore optimal strategies for selecting the suitable fine-tuned model for RL scaling, yielding superior performance-efficiency trade-offs in our RL training pipeline. Last, we develop a two-stage training paradigm to harmonize multi-domain data integration, enhancing reasoning ability while effectively improving performance across various downstream general tasks.

Highlights

• 🚀 Superior performance across tasks: Ring-lite-2507 demonstrates outstanding performance across both reasoning and general tasks;
• 🔥 Only 2.75B activated parameters: Ring-lite-2507 is built upon a Mixture-of-Experts (MoE)-based large language model with only 2.75 billion activated parameters;
• ⛓️‍💥 Algorithm-system co-design: We proposed novel C3PO approach and employ token efficiency to improve training stability and effectiveness;
• 🔍 Publicly available: We fully release our training recipe and model weights.

Evaluation​

We conduct a comprehensive evaluation of our models across two main domains: reasoning and general. We utilize a diverse set of public benchmarks, organized according to the specific aspects they measure.

Knowledge Understanding​

Math​

Coding​

Reasoning & Agentic​

Alignment​

Constrained Contextual Computation Policy Optimization(C3PO)​

We introduce Constrained Contextual Computation Policy Optimization(C3PO), an innovative token-level optimization framework designed to mitigate training instability while enhancing throughput consistency. Different from sampling-level filtering, C3PO operates at the token level by sampling tokens to form a token-level global batch, each training step maintains consistent token input to optimizer, which results in reduced gradient variance and consequently achieving stable optimization.

C3PO

Balancing Token efficiency between Distillation and RL​

While distillation is effective, we find that it requires more training tokens to achieve comparable performance than RL training. Furthermore, we observe that varying the number of training epochs for the distilled model significantly influences the trend of entropy loss, thereby affecting the exploration scope for RL. Our experiments show that increasing the number of SFT training epochs leads to a rapid collapse in entropy, whereas insufficient SFT training inevitably results in inferior performance. To systematically quantify the choice of optimal SFT epoch, we employ token efficiency to determine the suitable checkpoint for RL scaling.

Training Data​

To ensure a high-quality training dataset for reinforcement learning, we established a comprehensive and meticulous data curation pipeline. This pipeline encompasses several key stages, such as data cleansing, answer verification, and data annotation, all designed to thoroughly decontaminate the data and ensure it is both suitable and informative for RL training.

Data Pipeline

Training Pipeline​

Training Pipeline

Reasoning RL​

Compared to our previously released Ring-lite-2506, we expanded our reasoning dataset by incorporating more challenging math, coding, and STEM data. Specifically, we adopted 67K math problems, 32K coding problems, and 9.9K scientific problems for reasoning RL training. In addition, we amplified our reasoning dataset by including more than 19K logical games, such as ARC-AGI, Countdown, Sudoku, AlphaMaze, etc. For each type of problem, we specifically designed suitable reward functions to ensure our training examples are verifiable.

General RL​

Apart from reasoning tasks, our Ring-lite-2507 has significantly expanded the collection of general datasets for RL training. Our general RL training does not compromise performance on reasoning tasks; instead, it enhances overall text understanding across a broad range of general benchmarks.

Our general RL training incorporates a variety of tasks, including instruction following, question answering, text summarization, and more. For open-ended questions, we employ a robust reward model to assign appropriate scores. Additionally, we have integrated a rule-based verifier to handle problems that can be easily validated, such as instruction-following tasks.

Citation​

@misc{lingteam2025ringlitescalablereasoningc3postabilized,
      title={Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs}, 
      author={Ling Team and Bin Hu and Cai Chen and Deng Zhao and Ding Liu and Dingnan Jin and Feng Zhu and Hao Dai and Hongzhi Luan and Jia Guo and Jiaming Liu and Jiewei Wu and Jun Mei and Jun Zhou and Junbo Zhao and Junwu Xiong and Kaihong Zhang and Kuan Xu and Lei Liang and Liang Jiang and Liangcheng Fu and Longfei Zheng and Qiang Gao and Qing Cui and Quan Wan and Shaomian Zheng and Shuaicheng Li and Tongkai Yang and Wang Ren and Xiaodong Yan and Xiaopei Wan and Xiaoyun Feng and Xin Zhao and Xinxing Yang and Xinyu Kong and Xuemin Yang and Yang Li and Yingting Wu and Yongkang Liu and Zhankai Xu and Zhenduo Zhang and Zhenglei Zhou and Zhenyu Huang and Zhiqiang Zhang and Zihao Wang and Zujie Wen},
      year={2025},
      eprint={2506.14731},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.14731}, 
}

GITHUB 🤗 Hugging Face｜ 🤖 ModelScope

Overview​

Ming-lite-omni v1.5 is a comprehensive upgrade to the full-modal capabilities of Ming-lite-omni(Github). It significantly improves performance across tasks including image-text understanding, document understanding, video understanding, speech understanding and synthesis, and image generation and editing. Built upon Ling-lite-1.5, Ming-lite-omni v1.5 has a total of 20.3 billion parameters, with 3 billion active parameters in its MoE (Mixture-of-Experts) section. It demonstrates highly competitive results in various modal benchmarks compared to industry-leading models.

Performance Comparison

Introduce Ming-lite-omni v1.5​

Controllable Image Generation: Pixel-Level Control, Infinite Creativity​

Ming-lite-omni v1.5 significantly optimizes Scene Consistency and ID Consistency (Character / Style Consistency) in image editing. When editing human figures, it demonstrates a clear advantage in maintaining scene and character ID. Furthermore, it expands support for perceptual tasks such as generative segmentation, depth prediction, object detection, and edge contour generation.

Generative Image Segmentation

User

Given the following instructions: little girl, pink, your monitors colors off friend p pink shirt girl; please perform referring segmentation on this image.

Ming-lite-omni v1.5

User

Please segment different classes in this image.

Ming-lite-omni v1.5

User

Please segment different instances in this image.

Ming-lite-omni v1.5

Depth and Edge Detection

Original Image	Generated Depth Map	Generated Bounding Boxes	Generated Edge Contours

Audio-Video Interactive Understanding​

Experience Alignment Optimization: Smarter Interaction That Understands You Better​

Visual Dialogue

User

This picture looks very design-oriented. Can you describe its various design elements in detail?

Ming-Lite-Omni

This image shows a high-end gaming mouse with a modern and technological design. The mouse body is black with a smooth metallic finish. There is a blue glowing logo on the top of the mouse, and below the logo is the mouse wheel and left and right button area. The side of the mouse has a blue light decoration, which increases the visual appeal. The bottom of the mouse has a non-slip texture design to ensure stability during use. The overall design is simple and futuristic, suitable for gamers.

Ming-Lite-Omni V1.5

This image showcases a gaming mouse with a striking design, featuring the following design elements:
1. **Color and Material**: - The main body of the mouse is black, with a smooth surface and a metallic finish. - The edges and button areas of the mouse have blue LED light strips, adding a high-tech feel and visual appeal.
2. **Shape and Structure**: - The mouse has an overall streamlined design, with a slightly sloped top that allows for natural hand placement. - The sides of the mouse feature pronounced curves, designed ergonomically to provide a comfortable grip.
3. **Buttons and Scroll Wheel**: - There is a scroll wheel on the left side of the mouse, with a blue LED light strip above it. - There are additional buttons on the right side of the mouse, which also have blue LED light strips, enhancing both functionality and aesthetics.
4. **Brand Logo**: - In the center of the mouse's palm rest, there is an illuminated "G" logo, which is a signature design element of the brand.
5. **Lighting Effects**: - The blue LED light strips on the mouse stand out prominently against the dark background, enhancing its high-tech and futuristic feel.
6. **Texture and Details**: - The top of the mouse has a textured design, likely for an anti-slip effect. - The sides of the mouse feature fine, elongated textured patterns, which add a sense of visual depth.
Overall, the design of this mouse combines a modern, high-tech aesthetic with ergonomic design, making it both beautiful and practical, and well-suited for gamers.

User

Why is the calligraphy in the picture good?

Ming-Lite-Omni

This calligraphy work showcases superb calligraphic skills and profound cultural heritage. The brushwork is fluid and natural, the character structures are rigorous, and the layout is well-proportioned, reflecting the calligrapher's deep skill and artistic cultivation.

Ming-Lite-Omni V1.5

This piece of calligraphy displays several notable characteristics that make it an excellent work of art:
1. **Exquisite Brushwork**: The calligraphy's brushwork is fluid and natural, with rich variations in line thickness, demonstrating the calligrapher's profound skill and proficient technique.
2. **Rigorous Structure**: The structure of each character is meticulously designed, with natural connections between strokes, maintaining a good balance and harmony between characters.
3. **Composition and Layout**: The overall composition of the piece is well-arranged, with even line and character spacing, creating a sense of order and neatness.
4. **Ink Tone Variation**: The variations in ink tone, from dark to light, are natural, adding layers and a three-dimensional feel to the work.
5. **Elegant Content**: The content of the calligraphy is a beautiful classical text, rich in philosophical and literary value, which enhances the cultural depth of the work.
6. **Seal Embellishment**: The red seals on the work not only add a decorative element but also serve to embellish and balance the composition.
Together, these features constitute the high artistic value of this calligraphy piece, making it a masterpiece worthy of appreciation and study.

User

Is the phone in the picture suitable to buy for parents?

Ming-Lite-Omni

This phone has a simple and stylish appearance, suitable for parents who prefer a minimalist style. The camera configuration is also quite advanced, making it suitable for users who enjoy taking photos.

Ming-Lite-Omni V1.5

This phone is suitable for buying for parents for the following reasons:
1. **Appearance Design**: The phone features a simple and stylish design. The white body looks clean and neat, aligning with modern aesthetics.
2. **Camera Configuration**: The rear camera has a unique and neat arrangement, which may provide a good photography experience.
3. **Brand**: The Honor (荣耀) brand has a good reputation in the smartphone market, offering stable and reliable products.
4. **Features**: Phones of this type are typically equipped with a large-capacity battery and long battery life, suitable for daily use.
5. **Ease of Use**: It may offer a simple user interface and easy-to-operate functions, suitable for users who are not familiar with complex operations.
6. **Value for Money**: Honor phones are generally known for their high cost-performance ratio, making them suitable for users on a limited budget who still want a good experience.
Of course, whether it is truly suitable depends on the actual needs and preferences of the parents.

User

How to buy a plane ticket using the software in the picture? Please list the specific steps

Ming-Lite-Omni

The specific steps to buy a plane ticket using the software in the picture are as follows:
1. Open the Ctrip app. 2. Click on the "Flights" option. 3. On the flight page, select the departure city, destination, departure date, and return date. 4. Click the search button to view available flight information. 5. After selecting a suitable flight, click the "Book" button. 6. Fill in passenger information and choose a payment method. 7. Confirm the order details and submit the payment to complete the purchase.

Ming-Lite-Omni V1.5

The specific steps to purchase a plane ticket using the Ctrip app are as follows:
1. **Download and Install the Ctrip App**: - If you haven't installed the Ctrip App, you can search for "Ctrip Travel" (携程旅行) in the app store (like the Apple App Store or Android app markets) and download it.
2. **Open the Ctrip App**: - After installation is complete, open the Ctrip app.
3. **Select the "Flights" Option**: - On the homepage navigation bar, tap the "Flights" option.
4. **Enter Departure and Destination Cities**: - On the flight booking page, enter your departure and destination cities.
5. **Select Departure and Return Dates**: - Choose your departure date and return date (if booking a round trip).
6. **Select Passenger Information**: - Enter the number of passengers and their types (adult, child, infant).
7. **Choose a Flight**: - Browse and select your preferred flight.
8. **Fill in Passenger Information**: - Enter the passenger's name, ID number, contact information, etc.
9. **Select a Payment Method**: - Choose your preferred payment method (e.g., Alipay, WeChat Pay, bank card, etc.).
10. **Confirm Order and Pay**: - After confirming that the order information is correct, tap the "Pay" button to complete the payment.
11. **Receive Your E-ticket**: - After successful payment, you will receive an e-ticket, which you can view or download from the "My Orders" section within the app.
By following these steps, you can easily purchase plane tickets using the Ctrip App.

User

Based on the chart, under which primer is the Δct difference between samples the smallest? | Based on the chart, the Δct difference between samples is smallest under the cGAS-qF+qR primer. (Image source: Publicly available online - 舒桐科技公众号)

Ming-Lite-Omni

Based on the chart, the Δct difference between samples is smallest under the cGAS-qF+qR primer.

Ming-Lite-Omni V1.5

The difference in Δct between samples is smallest under the **EGFP-cGAS monoclonal** primer.

**Calculation Process:**
**1. Understand the meaning of Δct:** Δct is the difference in relative expression between the target gene (cGAS-qF+qR or EGFP-cGAS) and the reference gene (cTubulin). A smaller Δct indicates a smaller difference in expression between the target and reference genes.
**2. Compare Δct under different primers:**
- **cGAS-qF+qR primer:** - Wild type: Δct = 8.06 - EGFP-cGAS monoclonal: Δct = -0.15 - Difference: 8.06 - (-0.15) = 8.21
- **EGFP-cGAS monoclonal primer:** - Wild type: Δct = 10.44 - EGFP-cGAS monoclonal: Δct = -2.9 - Difference: 10.44 - (-2.9) = 13.34
- **EGFP-cGAS-qF+qR primer:** - Wild type: Δct = 21.43 - EGFP-cGAS monoclonal: Δct = -13.45 - Difference: 21.43 - (-13.45) = 34.88
**3. Conclusion:** Under the **EGFP-cGAS monoclonal** primer, the difference in Δct between samples is the smallest (Δct difference is 13.34), indicating the smallest difference in expression between samples under this primer.

**Summary:** The difference in Δct between samples is smallest under the **EGFP-cGAS monoclonal** primer.

Model Architecture Upgrade and Capability Evaluation​

The Ming-lite-omni v1.5 model architecture is outlined below. The core design references the structure of Ming-lite-omni V1. However, a key distinction is the upgrade of the Vision head to support reference image feature input, specifically to enhance character and scene consistency in image editing.

Mode Architecture

The model's capabilities have been significantly optimized and upgraded across three key areas: enhanced Omni-modal comprehension, precise visual editing control, and improved user experience.

Enhanced Omni-Modal Comprehension​

Thanks to optimized data quality, Ming-lite-omni v1.5 shows significant improvements in tasks such as vision-text comprehension (including image-text, document, and video understanding) and speech understanding. It has reached an industry-leading level for models of comparable scale.

Vision-text Comprehension

Speech Understanding

Precise Visual Editing Control​

Ming-lite-omni v1.5 employs the following optimization strategies to address the issues of character ID and scene ID consistency during image editing:

1. ID and Scene Consistency Loss: This is achieved by increasing the weight of the edited region in the target image and the reference strength of the non-edited region in the reference image, while simultaneously decreasing the reference strength of the edited region in the reference image. This approach enhances image editing consistency.
2. Incorporating Generative Detection and Segmentation Tasks to Boost Perceptual Capabilities: By supporting generative segmentation and keypoint detection, the model's understanding of image details and spatial relationships is improved. This enhances the structural controllability of the editing and generation processes, leading to significant increases in evaluation metrics related to position, structure, and quantity.
3. Multi-Task Collaborative Learning Strategy: Through a joint training pipeline, generation and editing mutually reinforce each other. Segmentation tasks are transformed into colorization editing tasks, which significantly improves segmentation metrics and the precision and controllability of local image editing, resulting in smoother edges for edited regions.

Based on these optimizations, Ming-lite-omni v1.5 shows a significant improvement in image editing capabilities, achieving a GenEval score of 0.87.

Optimized User Experience​

Thanks to the construction of high-quality alignment preference data, Ming-lite-omni v1.5 demonstrates a certain advantage over leading models in terms of correctness, relevance, format aesthetics, and fluency of expression for image-text Q&A. Ming-lite-omni v1.5 achieved a win rate of 87.07% against Ming-lite-omni V1 on internal adversarial evaluation sets, indicating a significant optimization in user experience.

Get Started with Ming-lite-omni v1.5​

The model and code for Ming-lite-omni v1.5 are now open-source, and we invite everyone to try it out, share feedback, and join the discussion. Looking ahead, we're excited to announce that a quantized and accelerated version is on the way. This future release will not only further enhance omni-modal performance but also make the model even more lightweight, all while strengthening its multimodal reasoning and generation capabilities. Stay tuned for more updates!

• Github: https://github.com/inclusionAI/Ming
• Hugging Face: https://huggingface.co/inclusionAI/Ming-Lite-Omni-1.5
• ModelScope: https://www.modelscope.cn/models/inclusionAI/Ming-Lite-Omni-1.5

📖 Technical Report | 🤗 Hugging Face｜ 🤖 ModelScope

Introduction​

We introduce M2-Reasoning-7B, a model designed to excel in both general and spatial reasoning. Our approach integrates two key innovations: (1) a novel data pipeline that generates 294.2K high-quality data samples (168K for cold-start fine-tuning and 126.2K for RLVR), which feature logically coherent reasoning trajectories and have undergone comprehensive assessment; and (2) a dynamic multi-task training strategy with step-wise optimization to mitigate conflicts between data, and task-specific rewards for delivering tailored incentive signals. This combination of curated data and advanced training allows M2-Reasoning-7B to set a new state-of-the-art (SOTA) across 8 benchmarks, showcasing superior performance in both general and spatial reasoning domains.

📌 Updates​

• [2025.07.14] 🔥 Our Technical Report is in public on arxiv.
• [2025.07.11] 🔥 We release M2-Reasoning on 🤗 Hugging Face and 🤖 ModelScope.

Key Features​

• A High-quality Data Construction Pipeline: We design and implement a multi-stage data synthesis and curation pipeline that generates vast amounts of reasoning data.
• A Dynamic Multi-Task Training Strategy: We propose a sophisticated training strategy that effectively handles data heterogeneity. It features step-wise dynamic optimization to mitigate conflicts between different data sources and a task-specific reward formulation to provide tailored incentive signals.
• Unified General and Spatial Reasoning Model: We propose M2-Reasoning-7B, an MLLM uniquely engineered for both abstract and spatial reasoning. Extensive evaluations on 8 distinctbenchmarks demonstrate that, by leveraging our custom data and training pipelines, M2-Reasoning establishes new state-of-the-art (SOTA) results across both general and spatial reasoning domains.

Evaluation​

We conduct a comprehensive evaluation of our models across two key domains: general and spatial reasoning. Our evaluation utilizes a diverse set of public benchmarks, grouped by the primary capability they measure:

• General Reasoning (Mathematical & Logical): To evaluate this capability, we employ six benchmarks: MathVista, MathVision, MathVerse, DynaMath, WeMath, and LogicVista.

• Spatial Reasoning: We assess this skill using 2 benchmarks: CV-Bench and VSI-Bench

  • CV-Bench:

  Models	Count	Relation	Depth	Distance	Avg.
  Large-Scale Models
  GPT-4O	65.9	85.7	87.8	78.2	78.9
  Gemini-1.5-pro	70.4	85.2	82.4	72.8	77.4
  Base-Scale Models
  InternVL3-8B	74.0	90.6	84.3	81.0	82.0
  Qwen2.5-VL-7B-Instruct	65.2	86.6	70.6	79.8	75.0
  LLava-NEXT-Video-7B	59.3	77.0	71.3	54.7	65.2
  Our Models
  M2-Reasoning-7B	66.6	92.8	89.3	84.3	82.3

  • VSI-Bench:

  	OC	AD	OS	RS	RDs	RDr	RP	AO	Avg.
  Large-Scale Models
  Gemini-1.5-pro	56.2	30.9	64.1	43.6	51.3	46.3	36.0	34.6	45.4
  GPT-4O	46.2	5.3	43.8	38.2	37.0	41.3	31.5	28.5	34.0
  Base-Scale Models
  InternVL3-8B	68.1	39.0	48.4	33.6	48.3	36.4	27.3	35.4	42.1
  Video-R1-7B	-	-	-	-	-	-	-	-	37.1
  Qwen2.5-VL-7B-Instruct	37.7	20.1	49.7	37.4	38.5	40.4	31.4	32.0	35.9
  LLava-NeXT-Video-7B	48.5	14.0	47.8	24.2	43.5	42.4	34.0	30.6	35.6
  Our Models
  M2-Reasoning-7B	41.0	34.0	60.9	55.4	40.7	47.3	29.9	28.8	42.3

Model Downloads​

You can download the model from both Hugging Face and ModelScope.

If you're in mainland China, we strongly recommend you to download our model from ModelScope.

Example Usage​

The basic environment is python=3.10, torch=2.6.0+cu124, transformers=4.49.0

We provide a small example on the usage of this repo.

import os
import torch

from transformers import (
    AutoProcessor,
    AutoTokenizer,
)

import warnings
import argparse
from modeling_bailing_qwen2_5 import Bailing_qwen2_5NativeForConditionalGeneration
from processing_bailing_qwen2_5 import Bailing_qwen2_5Processor

warnings.filterwarnings("ignore")

class BailingMMInfer:
    def __init__(self,
        model_name_or_path,
        device="cuda",
        max_pixels=None,
        min_pixels=None,
        video_max_pixels=768 * 28 * 28,
        video_min_pixels=128 * 28 * 28,
        generation_config=None
    ):
        super().__init__()
        self.model_name_or_path = model_name_or_path

        self.device = device

        self.device_map = device

        self.video_max_pixels = video_max_pixels if video_max_pixels is not None else 768 * 28 * 28
        self.video_min_pixels = video_min_pixels if video_min_pixels is not None else 128 * 28 * 28

        self.model, self.tokenizer, self.processor = self.load_model_processor()
        if max_pixels is not None:
            self.processor.max_pixels = max_pixels
        if min_pixels is not None:
            self.processor.min_pixels = min_pixels
        if generation_config is None:
            generation_config = {
                "num_beams": 1,
                "do_sample": True,
                "temperature": 0.9
            }

        self.generation_config = generation_config


    def load_model_processor(self):

        model = Bailing_qwen2_5NativeForConditionalGeneration.from_pretrained(
            self.model_name_or_path,
            torch_dtype=torch.bfloat16,
            device_map=self.device_map,
            _attn_implementation="flash_attention_2"
        ).eval()

        tokenizer = AutoTokenizer.from_pretrained(self.model_name_or_path, add_bos_token=True, trust_remote_code=True)
        processor = Bailing_qwen2_5Processor.from_pretrained(self.model_name_or_path, trust_remote_code=True)

        return model, tokenizer, processor

    def generate(self, messages, max_new_tokens=512):
        text = self.processor.apply_chat_template(
            messages, tokenize=False, add_generation_prompt=True, use_system=True
        )

        image_inputs, video_inputs = self.processor.process_vision_info(messages)


        inputs = self.processor(
            text=[text],
            images=image_inputs,
            videos=video_inputs,
            return_tensors="pt",
        )
        # print(inputs)
        print(self.tokenizer.decode(inputs['input_ids'][0]))

        inputs = inputs.to(self.device)

        for k in inputs.keys():
            if k == "pixel_values" or k == "pixel_values_videos":
                inputs[k] = inputs[k].to(dtype=torch.bfloat16)

        with torch.no_grad():
            generated_ids = self.model.generate(
                inputs,
                max_new_tokens=max_new_tokens,
                eos_token_id=self.processor.tokenizer.eos_token_id,
                **self.generation_config,
            )

        generated_ids_trimmed = [
            out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
        ]

        output_text = self.processor.batch_decode(
            generated_ids_trimmed, skip_special_tokens=False, clean_up_tokenization_spaces=False
        )[0]

        return output_text

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--model_name_or_path', type=str, default="inclusionAI/M2-Reasoning")
    parser.add_argument('--max_pixels', type=int, default=401408)
    parser.add_argument('--min_pixels', type=int, default=401408)
    parser.add_argument('--max_new_tokens', type=int, default=4096)

    args = parser.parse_args()

    device = "cuda" if torch.cuda.is_available() else "cpu"
    # model_name_or_path = os.path.join(args.input_dir, args.model_name_or_path)
    bailing2 = BailingMMInfer(
        args.model_name_or_path,
        device=device,
        max_pixels=args.max_pixels,
        min_pixels=args.min_pixels
    )

    messages = [
        {
            "role": "system",
            "content": [
                {"type": "text", "text": "You are a helpful assistant. When the user asks a question, your response must include two parts: first, the reasoning process enclosed in <think>...</think> tags, then the final answer enclosed in <answer>...</answer> tags. The critical answer or key result should be placed within \\boxed{}."}]},
        {
            "role": "user",
            "content": [
                {"type": "image", "image": "./assets/example1.png"},
                {"type": "text", "text": "\nQuestion:\n\nRhombus $QRST$ has an area of 137.9 square meters. If $RT$ is 12.2 meters, find $QS$.\nA. 11.3\nB. 22.4\nC. 22.6\nD. 25.6"},
            ],
        },
    ]
    output_text = bailing2.generate(messages, max_new_tokens=args.max_new_tokens)
    print(output_text)



'''
[Output]:

<think>
To find the length of \( QS \) in the rhombus \( QRST \), we can use the formula for the area of a rhombus, which is given by:

\[
\text{Area} = \frac{1}{2} \times d_1 \times d_2
\]

where \( d_1 \) and \( d_2 \) are the lengths of the diagonals. In this problem, we are given:
- The area of the rhombus is 137.9 square meters.
- One of the diagonals,

🌟 Overview​

ABench is an evolving open-source benchmark suite designed to rigorously evaluate and enhance Large Language Models (LLMs) on complex cross-domain tasks. By targeting current model weaknesses, ABench provides systematic challenges in high-difficulty specialized domains, including physics, actuarial science, logical reasoning, law, and psychology.

🎯 Core Objectives​

1. Address Evaluation Gaps: Design high-differentiation assessment tasks targeting underperforming question types
2. Establish Unified Standards: Create reliable, comparable benchmarks for multi-domain LLM evaluation
3. Expand Capability Boundaries: Drive continuous optimization of knowledge systems and reasoning mechanisms through challenging innovative problems

📊 Dataset Release Status​

"Self-awareness: the hardest problem isn't solving within limits, it's discovering the own limitations"

Table of Contents​

• News — Latest updates and announcements.
• Introduction — Overview and purpose of the project.
• Installation — Step-by-step setup instructions.
• Quick Start — Get started with usage examples.
• Architecture — Explore the multi-agent system design.
• Demo — See the project in action with demonstrations.
• Contributing — How to get involved and contribute.
• License — Project licensing details.

News​

• 🦤 [2025/07/07] AWorld, as a runtime, is now ready for agentic training. See Self-Improvement section for details. We have updated our score to 77.08 on the GAIA test. Learn how to construct a GAIA runtime in the Demo section.
• 🦩 [2025/06/19] We have updated our score to 72.43 on the GAIA test. Additionally, we have introduced a new local running mode. See ./README-local.md for detailed instructions.
• 🐳 [2025/05/22] For quick GAIA evaluation, MCP tools, AWorld, and models are now available in a single Docker image. See ./README-docker.md for instructions and youtube video for demo.
• 🥳 [2025/05/13] AWorld has updated its state management for browser use and enhanced the video processing MCP server, achieving a score of 77.58 on GAIA validation (Pass@1 = 61.8) and maintaining its position as the top-ranked open-source framework. Learn more: GAIA leaderboard
• ✨ [2025/04/23] AWorld ranks 3rd on GAIA benchmark (69.7 avg) with impressive Pass@1 = 58.8, 1st among open-source frameworks. Reproduce with python examples/gaia/run.py

Introduction​

AWorld (Agent World) is a multi-agent playground that enables agents to collaborate and self-improve. The framework supports a wide range of applications, including but not limited to product prototype verification, foundation model training and Multi-Agent System (MAS) design meta-learning.

Runtime Key Features​

Self-Improvement with Diverse Runtimes​

By constructing diverse runtime environments (with tools, agents, or models in them), AWorld aims to find the limitations of a model and push intelligence forward. Here we will record some of our work to prove the effectiveness of our proposal.

Category	Runtime	Performance	Key Information
Tool Use	Function call runtime to be released	Competitive on BFCL benchmark
Deep Search	Search runtime to be released	SOTA on HotpotQA benchmark

Demo of GAIA Agent-Runtime​

Here we first introduce the GAIA runtime, which can be constructed on your local computer. It can be used for:

• Product prototype verification
• Self-improvement training (See training pipeline for details)

Follow the instructions in ./examples/gaia/README.md to initialize the GAIA agent runtime and run the demo shown above.

Want to build your own multi-agent system? Check out the detailed tutorials below to get started! ⬇️⬇️⬇️

Installation​

Python>=3.11:

git clone https://github.com/inclusionAI/AWorld
cd AWorld
python setup.py install

Quick Start​

Here's a quick start guide to: (1) create your first agent; (2) equip it with a MCP tool; (3) assign a teammate; and (4) answer a user query through teamwork.

from aworld.config.conf import AgentConfig
from aworld.agents.llm_agent import Agent
from aworld.runner import Runners
from aworld.core.agent.swarm import Swarm

if __name__ == '__main__':
    agent_config = AgentConfig(
        llm_provider="openai",
        llm_model_name="gpt-4o",

        # Set via environment variable or direct configuration
        # llm_api_key="YOUR_API_KEY",
        # llm_base_url="https://api.openai.com/v1"
    )

    # Register the MCP tool here, or create a separate configuration file.
    mcp_config = {
        "mcpServers": {
            "amap-amap-sse": {
                "type": "sse",
                "url": "https://mcp.amap.com/sse?key=YOUR_API_KEY",
                "timeout": 5,
                "sse_read_timeout": 300
            }
        }
    }

    # Create your first agent equipped with an MCP tool
    search = Agent(
        conf=agent_config,
        name="search_agent",
        system_prompt="You are a helpful agent.",
        mcp_servers=["amap-amap-sse"], # MCP server name for agent to use
        mcp_config=mcp_config
    )

    # Add a new teammate to the agent
    summary = Agent(
        conf=agent_config,
        name="summary_agent",
        system_prompt="You are a helpful summary agent."
    )

    # Collaborate as a team; the default is a static workflow
    swarm = Swarm(search, summary)

    # Run agent team
    res = Runners.sync_run(input="Hotels within 1 kilometer of West Lake in Hangzhou",
                     swarm=swarm)
    print(res)

Architecture​

AWorld is designed to achieve two primary objectives: (1) provide an efficient forward process, and (2) facilitate diverse backward processes, including but not limited to foundation model training and system design meta-learning.

Forward​

An illustration of the runtime, showing the message workflow when Agent1 receives a query from a user.

Backward​

During training, an action-state rollout demonstration using AWorld's distributed environments.

Demo​

Running Pre-defined Agents (e.g., see demo code). Below are demonstration videos showcasing AWorld's capabilities across various agent configurations and environments.

Mode	Type	Demo
Single Agent	Browser use	▶️ Watch Browser Demo on YouTube
	Phone use	▶️ Watch Mobile Demo on YouTube
Multi Agent	Cooperative Teams	▶️ Watch Travel Demo on YouTube
	Competitive Teams	▶️ Watch Debate Arena on YouTube
	Mixed of both Teams	Coming Soon 🚀

Contributing​

We warmly welcome developers to join us in building and improving AWorld! Whether you're interested in enhancing the framework, fixing bugs, or adding new features, your contributions are valuable to us.

For academic citations or wish to contact us, please use the following BibTeX entry:

@software{aworld2025,
  author = {Agent Team at InclusionAI},
  title = {AWorld: Enabling Agent Self-Improvement through Interactive Experience with Dynamic Runtime},
  year = {2025},
  url = {https://github.com/inclusionAI/AWorld},
  version = {0.1.0},
  publisher = {GitHub},
  email = {chenyi.zcy at antgroup.com}
}

License​

This project is licensed under the MIT License - see the LICENSE file for details.

Star History​