VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct
VeriEvol employs iterative prompt evolution and offline answer verification to scale visual math reasoning data from 10K to 250K, boosting accuracy by 19.31%.
Key Findings
Methodology
VeriEvol introduces a dual-axis scalable framework combining route-specific prompt evolution and offline hypothesis-testing answer verification. The prompt evolution module employs problem-category routing to generate progressively harder, image-grounded prompts by applying constrained, route-specific operators that enhance reasoning complexity while maintaining visual grounding. The HTV-Agent verifier operates through multiple stages: generating independent solver hypotheses, seeking counter-evidence via programmatic and visual channels, resolving conflicts through a decider, and applying a deterministic acceptance gate. This process ensures only answers that withstand multi-source refutation are accepted, forming high-reliability training data. The framework decouples prompt difficulty and answer reliability, allowing independent scaling and ablation, and integrates seamlessly with existing RL pipelines by producing standard prompt-answer-reward tuples for supervised fine-tuning (SFT) and reinforcement learning (RL).
Key Results
- On five visual math benchmarks, evolving SFT data from 10K to 250K samples increased mean accuracy from 35.42% to 54.73%, a gain of 19.31%. In RL training, verified data from 10K to 130K samples yielded a cumulative performance boost of +3.88 points over the un-evolved baseline, with +1.82 attributable to evolved prompts and +2.06 to the verifier. The approach demonstrated monotonic data scaling and robustness across tasks.
- The multi-source verification mechanism effectively reduced visual grounding failures and shortcut answers, leading to higher answer correctness and model reasoning robustness. The modular design allows adding new verification channels or evolution routes without re-engineering the entire pipeline, facilitating extensibility.
- Compared to state-of-the-art 7B models like OpenMMReasoner-7B and ReVisual-R1-7B, VeriEvol achieved superior external benchmark scores, with an average improvement of approximately 8-10 percentage points, confirming its effectiveness in large-scale multimodal reasoning tasks.
Significance
This work addresses a critical bottleneck in scaling multimodal reasoning models: ensuring data quality and answer reliability at large volumes. By integrating prompt evolution with offline answer falsification, VeriEvol provides a systematic way to generate high-quality, verifiable training data, significantly improving the robustness and generalization of large models. This approach paves the way for more reliable AI systems capable of complex, multi-step reasoning in diverse visual contexts, with broad implications for scientific computing, intelligent tutoring, and automated reasoning applications. It also offers a blueprint for scalable, transparent data construction pipelines that can be audited and extended by the community, fostering reproducibility and trustworthiness in AI research.
Technical Contribution
The primary technical innovation lies in combining route-specific prompt evolution with a multi-stage, multi-source answer verification process. The prompt evolution module employs a schema-constrained classifier to route each seed into problem families, applying tailored operators that incrementally increase difficulty while preserving visual grounding. The HTV-Agent verifier integrates multiple independent solvers, programmatic checks, and visual evidence channels, with a conflict-aware decider that ensures only answers passing all channels are accepted. This design enables scalable, traceable data generation compatible with existing RL frameworks, with explicit control over prompt difficulty and answer reliability. The approach introduces a new paradigm for data construction that explicitly decouples challenge escalation from correctness assurance.
Novelty
This research uniquely combines iterative prompt evolution with offline, multi-source answer falsification, addressing the limitations of prior methods that rely solely on generator-based augmentation or in-loop verification. The explicit decoupling of prompt difficulty and answer reliability axes, along with the multi-stage, multi-channel verification pipeline, represents a novel contribution to scalable data construction in multimodal reasoning. It is the first to demonstrate that such a dual-axis, verifiable data pipeline can substantially enhance both supervised fine-tuning and reinforcement learning performance, setting a new standard for data quality assurance in large-scale multimodal models.
Limitations
- Despite the robustness of the verification process, the system may still encounter failure modes in highly ambiguous or noisy visual scenes, where visual grounding or OCR may produce uncertain results, leading to potential false negatives or positives.
- The computational overhead of multi-stage verification, especially the multiple solver runs and evidence checks, is significant, which may limit scalability in resource-constrained environments. Future work should focus on optimizing efficiency.
- The current routing schema relies on accurate problem classification; misclassification could lead to suboptimal prompt evolution or verification, reducing diversity and coverage. Improving routing accuracy remains an open challenge.
Future Work
Future directions include developing more efficient verification algorithms, possibly leveraging self-supervised or few-shot learning to reduce computational costs. Extending the framework to incorporate additional modalities like video or speech could broaden its applicability. Adaptive, dynamic routing strategies and reinforcement-based feedback mechanisms could further enhance the difficulty control and verification robustness, enabling autonomous scaling to even larger datasets and more complex reasoning tasks.
AI Executive Summary
In recent years, large multimodal models have demonstrated remarkable progress in tasks requiring visual understanding and reasoning. However, scaling these models effectively remains a challenge due to the difficulty of acquiring high-quality, reliable training data. Traditional data pipelines often rely on human annotations or synthetic generation, which can introduce noise and limit the models' reasoning capabilities. Moreover, as models grow larger, the impact of incorrect labels becomes more pronounced, especially in reinforcement learning settings where reward signals are derived directly from data labels.
Li et al. address this bottleneck with VeriEvol, a novel framework that combines iterative prompt evolution with offline answer verification. The core idea is to generate progressively harder, visually grounded prompts from simple seeds, while simultaneously verifying answers through a multi-source, multi-stage process that seeks to falsify solutions before they are used as training signals. This approach decouples the escalation of question difficulty from answer correctness, enabling scalable, high-reliability data construction.
The prompt evolution component employs a type-aware routing system that classifies each seed into problem categories such as geometry, charts, or OCR, and applies tailored operators to produce more challenging prompts. These operators are designed to amplify reasoning complexity while maintaining visual grounding, ensuring that the generated questions are both difficult and verifiable. The verification component, HTV-Agent, involves multiple independent solvers generating hypotheses, followed by rigorous cross-channel checks—programmatic assertions and visual grounding tests—to identify contradictions or errors. A conflict-aware decider then synthesizes these signals, rejecting answers that cannot be confidently verified.
Experimental results on five benchmark datasets reveal the effectiveness of VeriEvol. When expanding the training data from 10K to 250K samples, the average accuracy on visual math tasks increased by over 19 percentage points. In reinforcement learning, verified data from 10K to 130K samples yielded a performance boost of nearly 4 points, with the contribution split between evolved prompts and the verification process. Notably, VeriEvol outperformed several strong baselines, including models like OpenMMReasoner-7B and ReVisual-R1-7B, across multiple metrics.
This work has significant implications for the development of reliable, scalable multimodal AI systems. By ensuring data quality through explicit verification, it mitigates the risk of reinforcing incorrect policies and enhances model robustness. The modular design allows easy extension with new verification channels or evolution strategies, fostering transparency and auditability. Looking ahead, further improvements in efficiency, multimodal integration, and adaptive difficulty control could unlock even broader applications, from scientific research to intelligent tutoring. Overall, VeriEvol marks a substantial step forward in the quest for trustworthy, high-capacity multimodal reasoning models.
Deep Dive
Abstract
Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions: as data volume grows, the reward labels themselves must remain reliable. Yet existing data pipelines scale supervision while trusting the labeller, and policy-side methods assume the underlying answers are already correct. We instead treat scaling as a verifiable data-construction problem and decouple two axes before any policy update: prompt difficulty, expanded by route-specific evolution operators, and answer reliability, enforced by offline hypothesis-test falsification. We instantiate this as VeriEvol, an iterative framework with two extensible components: a type-aware evolution module that rewrites low-difficulty image-question seeds into harder, image-grounded prompts; and HTV-Agent, a verifier that accepts an answer only after multi-source counter-evidence has failed to refute it. The resulting verified data scales in volume, extends by adding evolution routes or verifier channels, and plugs directly into existing GRPO-style RL recipes. On a five-benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73; then, with backbone, SFT initialization, and GRPO recipe held fixed, VeriEvol adds a cumulative +3.88 over an un-evolved RL baseline, of which +1.82 comes from evolved prompts and +2.06 from the HTV-Agent verifier. We release the prompts, data, models, code, and the full verifier trace of every sample, so that downstream work can scale and audit the pipeline rather than only inspect its outputs.
References (20)
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Adam Suma, Sam Dauncey
DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
Ziwei Zheng, Michael Yang, Jack Hong et al.
Qwen2.5-VL Technical Report
Shuai Bai, Ke-qin Chen, Xuejing Liu et al.
Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
Ke Wang, Junting Pan, Weikang Shi et al.
MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning
Fanqing Meng, Lingxiao Du, Zongkai Liu et al.
Judge Anything: MLLM as a Judge Across Any Modality
Shu Pu, Yaochen Wang, Dongping Chen et al.
MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
Sicong Leng, Jing Wang, Jiaxi Li et al.
See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoning
Shuoshuo Zhang, Yizhen Zhang, Jingjing Fu et al.
MR. Judge: Multimodal Reasoner as a Judge
Renjie Pi, Felix Bai, Qibin Chen et al.
V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation
Han Wang, Yi Yang, Jingyuan Hu et al.
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
Jinguo Zhu, Weiyun Wang, Zhe Chen et al.
Let's Verify Step by Step
H. Lightman, Vineet Kosaraju, Yura Burda et al.
We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?
Runqi Qiao, Qiuna Tan, Guanting Dong et al.
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
Chaoqun He, Renjie Luo, Yuzhuo Bai et al.
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Zhibin Gou, Zhihong Shao, Yeyun Gong et al.
Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models
Zeyu Liu, Yuhang Liu, Guanghao Zhu et al.
MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Xiang Yue, Yuansheng Ni, Kai Zhang et al.
More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models
H. Just, Yifei Fan, Handong Zhao et al.
Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning
Rui Liu, Dian Yu, Tong Zheng et al.
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
Wenxuan Huang, Bohan Jia, Zijie Zhai et al.