Error-Conditioned Neural Solvers
Error-conditioned neural solver (ENS) uses PDE residuals as direct inputs, achieving 10× accuracy improvement and avoiding costly optimizers.
Key Findings
Methodology
This paper introduces the Error-Conditioned Neural Solver (ENS), a novel framework that incorporates the PDE residual field directly into the neural network at each iteration. ENS consists of a predictor network (Pθ) that generates an initial solution estimate, and a learned corrector (Cϕ) that reads the current solution and residual field to produce a nonlinear correction. During training, the model is supervised with a reconstruction loss, enabling it to learn a robust correction policy across diverse residual distributions. In inference, ENS performs multiple recursive updates, iteratively refining the solution without explicit numerical optimization. This approach circumvents the residual minimization pitfalls of hybrid methods, such as instability and high computational cost, especially in ill-conditioned systems. The residual field as input allows the network to adaptively read the spatial structure of its own errors, leading to superior accuracy and generalization, including zero-shot parameter shifts and cross-equation transfer.
Key Results
- Across four PDE families—linear/nonlinear Helmholtz, Darcy flow, Poisson, Navier-Stokes—ENS consistently outperforms baselines like PINO, POSEIDON, and PCFM. For example, in the turbulent Kolmogorov flow, ENS achieves an L2 error of 1.98e-2, which is roughly 10× lower than PINO-TTOP’s 1.68e-1, with residuals maintained around 3.9e-1. In the Helmholtz problem, ENS’s error is approximately 3.04e-3, significantly better than PINO’s 4.19e-2, while residuals are comparable. In out-of-distribution tests, such as parameter extrapolation and cross-equation transfer, ENS maintains low errors and residuals, demonstrating strong generalization. The method also converges robustly from diverse initializations, reducing residuals to a stable floor regardless of starting point.
- In terms of efficiency, ENS runs approximately 0.1 seconds per sample for static PDEs and 0.19 seconds for Navier-Stokes flows, vastly faster than hybrid methods like PCFM (over 200 seconds). Despite the speed, ENS achieves the highest accuracy, making it a Pareto-optimal solution balancing computational cost and precision. Ablation studies confirm that feeding residuals as inputs, rather than optimizing residuals directly, is key to performance gains. The learned correction policy continues improving beyond training steps, indicating effective iterative refinement.
- Theoretical analysis reveals a residual-reconstruction gap (Prop. 1): minimizing residuals alone does not guarantee low solution error, especially in ill-conditioned systems with small Jacobian singular values. ENS’s approach directly addresses this by reading the residual’s spatial structure, enabling stable convergence even when residual minimization fails. Extensive experiments validate this, showing that residual-based hybrid methods often produce low residuals but high errors, whereas ENS maintains low errors and residuals simultaneously across diverse scenarios.
Significance
This work marks a significant advancement in physics-informed machine learning for PDEs. By directly integrating the residual field as an input, ENS overcomes the instability and inefficiency of traditional residual minimization techniques like Gauss-Newton or gradient descent. Its ability to generalize under distribution shifts and across different equations addresses a long-standing challenge in scientific computing—robust, fast, and accurate PDE solutions in complex, ill-conditioned regimes. The approach paves the way for scalable, real-time simulation in engineering, climate modeling, and fluid dynamics, where traditional solvers are computationally prohibitive. ENS’s framework also opens new avenues for integrating learned correction policies into classical numerical methods, fostering a hybrid paradigm that combines data-driven adaptability with physical fidelity. Overall, this research significantly narrows the gap between neural approximations and classical solvers, promising transformative impacts on computational science and engineering.
Technical Contribution
The primary technical innovation lies in treating the PDE residual as a direct input rather than an optimization target. This enables the network to learn a nonlinear correction policy that reads the spatial structure of its own errors, effectively bypassing the residual-reconstruction gap highlighted in Prop. 1. The model architecture combines a predictor (e.g., Fourier Neural Operator or Transformer backbone) with a recurrent corrector trained via end-to-end supervision on multi-step residuals. The training scheme exposes the corrector to realistic residual distributions, ensuring robustness. Theoretically, the authors derive the residual-reconstruction gap, explaining why residual minimization can be unreliable in ill-conditioned systems, and demonstrate how ENS mitigates this through learned nonlinear corrections. Empirically, ENS achieves up to 10× accuracy improvement over existing methods, with robust convergence from diverse initializations, and maintains low residuals and errors under distribution shifts.
Novelty
This work is the first to embed the PDE residual field directly as an input to a neural network for iterative correction, rather than using residual minimization as an optimization objective. Unlike PINNs or hybrid residual minimization methods, ENS learns a nonlinear correction policy that reads the spatial error structure, enabling stable convergence even in ill-conditioned regimes. The theoretical analysis of the residual-reconstruction gap (Prop. 1) provides a new understanding of the limitations of residual-based methods, highlighting the importance of reading the residual as a dynamic error signal. The combination of residual field input, learned nonlinear correction, and multi-step recursive refinement constitutes a novel paradigm in neural PDE solving, bridging the gap between classical numerical methods and modern deep learning approaches.
Limitations
- ENS’s performance may degrade in extremely high-dimensional or highly noisy systems where residual fields become too complex for the network to interpret accurately. The method also relies on sufficient training data and hyperparameter tuning, which could limit scalability.
- The current architecture, while expressive, may not fully capture multi-scale phenomena or complex boundary conditions without further modifications. Extending ENS to handle such scenarios remains an open challenge.
- Training involves substantial computational resources, especially for large-scale problems or when incorporating complex physics. Future work should focus on reducing training costs and improving robustness in diverse environments.
- Although ENS demonstrates excellent generalization, its performance in real-world noisy data or uncertain boundary conditions needs further validation. Incorporating uncertainty quantification and robustness mechanisms will be crucial for practical deployment.
Future Work
Future research will explore integrating ENS with multi-scale models and physics-informed priors to enhance robustness in highly nonlinear or multi-physics systems. Extending the framework to handle stochastic PDEs and uncertain boundary conditions is another promising direction. Additionally, developing more efficient training strategies, such as transfer learning or unsupervised pretraining, could reduce data requirements. Real-time deployment in industrial applications, like aerodynamic design or climate prediction, will be a key goal. Lastly, combining ENS with hardware acceleration and parallel computing will enable scalable, high-fidelity simulations for large-scale scientific problems.
AI Executive Summary
偏微分方程（PDE）在描述自然和工程现象中扮演着基础角色，从气候变化到流体动力学，几乎无处不在。传统的数值方法如有限元和有限差分虽然在精度上有保障，但在高维复杂系统中计算成本极高，难以满足实时仿真的需求。近年来，深度学习的崛起带来了神经网络在偏微分方程求解中的新希望，特别是神经算子（Neural Operators）和物理信息神经网络（PINNs），它们通过学习参数到解的映射，大大提升了预测速度。然而，这些方法在物理一致性、泛化能力和稳定性方面仍存在瓶颈。神经网络通常将偏微分方程的求解视作回归任务，缺乏对物理约束的主动反馈，导致在超出训练分布或复杂系统中表现不佳。混合方法试图在推理阶段引入残差优化，结合数值方法的物理保障，但其依赖昂贵的梯度和线性求解，容易引发数值不稳定和计算瓶颈。本文提出的误差条件神经求解器（ENS）创新性地将偏微分方程残差场作为网络的直接输入，通过多次递归修正，显著提升了复杂系统中的预测精度和泛化能力。ENS由预测器和学习的修正器组成，训练过程中利用重建监督，确保模型在不同残差分布下都能学习到稳健的修正策略。在多个复杂的PDE任务中，ENS展现出优异的性能，误差比传统方法低10倍，且在参数迁移和跨方程转移中表现出强大的泛化能力。该方法不仅在计算效率上优于混合优化方法，还显著提升了预测的物理一致性，为偏微分方程的快速、准确求解开辟了新路径。这一突破为科学计算和工程应用提供了强有力的工具，有望推动大规模仿真、实时控制和多物理场模拟的发展。未来，结合多尺度机制和物理先验，ENS有望在更复杂、更高维的系统中实现广泛应用，开启深度学习在科学计算中的新纪元。
Deep Analysis
Background
偏微分方程（PDE）作为描述自然界各种物理现象的数学工具，已成为科学与工程中的基础模型。从早期的有限差分、有限元等数值方法到近年来兴起的深度学习方法，研究者不断探索更高效、更准确的求解策略。神经算子（Neural Operators）如DeepONet和Fourier Neural Operator（FNO）通过学习参数到解的映射，实现了跨问题的快速预测。PINNs（Physics-Informed Neural Networks）则结合了物理约束与神经网络，允许在少量数据下进行高精度求解。然而，这些方法在实际应用中仍面临诸多挑战：神经算子在高维问题中的泛化能力有限，PINNs在复杂几何和强非线性系统中训练困难，且都缺乏对物理约束的主动反馈机制。传统数值方法虽具有严格的物理一致性，但在大规模仿真中计算成本高昂，成为瓶颈。近年来，混合方法试图结合数值与神经网络的优势，通过在推理阶段引入残差优化，改善物理一致性，但其依赖昂贵的数值优化，容易引发数值不稳定和计算瓶颈。本文的研究背景正是在此基础上，旨在突破残差最小化的局限，提出一种新颖的神经求解框架，提升复杂系统中的预测精度和泛化能力。
Core Problem
现有神经偏微分方程求解方法在面对高条件数、强非线性或不稳定系统时，表现出明显的性能瓶颈。神经算子和PINNs等方法虽然在速度上优于传统数值方法，但在物理约束的满足和泛化能力方面仍不足。尤其是在超出训练分布的参数变化或跨方程迁移中，模型容易出现大误差和物理不一致的问题。混合方法通过在推理阶段优化残差，试图弥补这一缺陷，但其依赖昂贵的数值优化，容易引发数值不稳定和计算瓶颈，限制了其实际应用。更重要的是，残差最小化在高条件数系统中可能存在“残差-重建差距”，即低残差并不意味着高精度。这些问题严重制约了神经方法在复杂偏微分方程中的应用推广，亟需一种既能保持物理一致性，又具备良好泛化能力的求解策略。
Innovation
本文的核心创新在于将偏微分方程的残差场作为神经网络的直接输入，而非传统的优化目标。这一设计使得模型可以在每次迭代中“自主感知”自身的误差空间结构，从而学习出非线性修正策略，逐步逼近真实解。具体而言，ENS由预测器（Pθ）生成初始估计，修正器（Cϕ）根据残差场学习非线性映射，训练过程中通过端到端的重建损失优化。该方法避免了残差最小化带来的数值不稳定和“残差-重建差距”，实现了在高条件数系统中的稳健逼近。技术上，ENS结合了Transformer和Fourier神经算子等强表达能力架构，增强了模型对复杂空间结构的捕捉能力。理论上，作者提出残差-重建差距（Prop. 1）分析，揭示了残差最小化在高条件数系统中的局限性，为理解传统混合方法的不足提供了数学依据。实验验证显示，ENS在多个复杂PDE任务中均优于现有方法，验证了其技术创新的有效性。
Methodology
- �� 预测器（Pθ）根据输入的PDE参数和边界条件，生成初始解估计。• 计算当前预测的残差场 r(k) = F(û(k); f)，作为空间误差的直接指标。• 修正器（Cϕ）接收当前解 û(k) 和残差场 r(k)，学习非线性映射，输出修正 δû(k)。• 在每次迭代中，更新解：û(k+1) = û(k) + β·δû(k)，其中β为步长超参数。• 训练过程中，端到端优化所有参数，目标为多步修正的重建损失：L = Σ_{k=0}^{K} ||û(k) - u_{gt}||^2。• 采用多步递归策略，模型在训练中学习到稳健的修正策略，推理时可持续改善解的精度。• 网络架构方面，静态方程采用改良的Fourier神经算子（FNO），湍流流动采用Transformer backbone。• 训练过程中，模型在不同残差分布下进行端到端优化，确保其在推理中的泛化能力。• 推理阶段，模型通过多次迭代，不断读取残差场，逐步逼近真实解，避免了传统线性化和二阶优化的数值不稳定性。
Experiments
实验设计涵盖四个代表性PDE家族：线性与非线性Helmholtz、Darcy流、Poisson和Navier-Stokes方程。训练数据来自对应的标准模拟数据集，测试在未见过的参数和几何配置上进行。基线方法包括FNO、PINN、POSEIDON、DiffusionPDE和PCFM，均采用官方实现和默认参数。评估指标包括相对L2误差和PDE残差均方误差（MSE），以全面衡量解的准确性和物理一致性。超参数方面，训练采用K=5修正步骤，步长β通过验证集调优。 在不同的迁移场景（如参数超出训练范围、跨方程转移、超分辨率）下，模型表现被详细比较，特别关注残差与重建误差的差异。实验还进行了消融实验，验证残差作为网络输入的关键作用，以及不同网络架构对性能的影响。
Results
ENS在所有测试场景中均优于基线方法，误差最低，残差保持在合理范围。具体而言，在非线性Helmholtz问题中，ENS的L2误差约为3.04e-3，远优于PINN和PINO-TTOP的误差（分别为4.19e-2和6.44e-2），且残差为3.9e-1，显示出良好的物理一致性。在湍流Kolmogorov流中，ENS误差达到1.98e-2，较PINO-TTOP的1.68e-1低近10倍。在参数迁移和跨方程转移中，ENS依然保持低误差（如Poisson到非线性Helmholtz的误差为2.59e-2），验证了其强大的泛化能力。实验还显示，残差作为输入的设计显著优于仅优化残差目标的策略，模型在多次修正后能持续逼近真实解，超越单次前馈预测的限制。性能方面，ENS推理速度在静态问题中约为0.1秒/样本，远快于混合优化方法（如PCFM的251秒），在复杂流动模拟中仍保持优异的效率和准确性。
Applications
ENS的核心优势在于快速、准确地求解复杂偏微分方程，适用于气候模拟、流体动力学、材料科学等领域的高性能仿真。其无需昂贵的数值线性求解，能在大规模模拟中实现实时或准实时的物理场预测。未来可结合硬件加速（如GPU、TPU）应用于工业级仿真平台，支持复杂边界条件和多物理场耦合问题。此外，ENS的泛化能力使其在参数优化、逆问题、控制与优化等场景中具有潜在应用价值，有望推动科学计算的智能化和自动化发展。
Limitations & Outlook
尽管ENS在多场景中表现优异，但在极端高维或极端非线性系统中仍可能面临训练困难。模型对残差场的表达能力依赖较强，若网络架构不足或残差噪声较大，性能可能下降。此外，训练过程对大量标注数据和超参数调优的依赖，存在一定的计算成本。未来需探索更高效的训练策略和模型结构，以减少数据和计算需求。同时，模型在极端复杂边界条件和强不确定性环境下的鲁棒性仍需验证，未来工作将结合物理先验和多尺度机制，提升模型的泛化和稳定性。
Glossary
Neural Operator (神经算子)
一种通过神经网络学习从参数空间到函数空间映射的方法，能实现偏微分方程的快速预测。
在本文中，神经算子用于替代传统数值方法，快速预测不同参数下的解。
PDE Residual (偏微分方程残差)
偏微分方程在某个解上的偏差，反映解是否满足方程的物理约束。
ENS将残差场作为输入，用于引导修正预测。
Residual-Conditioned Neural Solver (误差条件神经求解器)
一种利用偏微分方程残差作为输入，递归修正偏微分方程解的神经网络框架。
本文提出的核心方法，提升了复杂系统中的预测精度和泛化能力。
Gauss-Newton (高斯-牛顿法)
一种用于非线性最小二乘问题的二阶优化算法，依赖于雅可比矩阵的近似逆。
混合方法中采用，但在高条件数系统中表现不稳定。
Fourier Neural Operator (FNO, 傅里叶神经算子)
一种利用傅里叶变换实现偏微分方程快速预测的神经网络架构。
ENS中的静态方程采用FNO架构。
Transformer Backbone (变换器骨架)
基于Transformer结构的深度学习模型，擅长捕捉复杂空间关系。
ENS在湍流流动任务中采用Transformer作为修正器架构。
Reconstruction Loss (重建损失)
衡量预测解与真实解差异的指标，常用L2范数。
训练目标，确保模型输出逼近真实解。
Distribution Shift (分布迁移)
模型在训练数据分布之外的测试数据上的表现变化。
ENS在参数超出训练范围时仍保持性能，体现出良好的迁移能力。
Iterative Refinement (迭代修正)
通过多次递归调整预测解，逐步逼近真实解的方法。
ENS采用多步递归修正策略，超越单次前馈预测的限制。
Physics-Informed Neural Networks (PINNs, 物理信息神经网络)
结合物理约束与神经网络的偏微分方程求解方法。
作为传统方法的代表，存在训练成本高、泛化差的问题。
Open Questions Unanswered questions from this research
- 1 ENS在极端高维或极端非线性系统中的表现尚未充分验证，未来需探索其在更复杂场景中的鲁棒性和稳定性。
- 2 模型对残差场的表达能力依赖较强，如何设计更高效的网络架构以应对更复杂的空间结构仍是挑战。
- 3 训练过程中对大量标注数据和超参数调优的依赖，限制了其在实际大规模应用中的推广，需要开发更高效的训练策略。
- 4 ENS在极端边界条件和强不确定性环境中的表现仍需验证，未来应结合物理先验和多尺度机制提升泛化能力。
- 5 如何将ENS与硬件加速技术结合，实现实时偏微分方程求解，是未来的重要研究方向。
Applications
Immediate Applications
气候模拟与天气预报
ENS可以快速逼近复杂大气模型的解，支持实时天气预报和气候变化预测，减少传统数值模拟的计算成本。
工程结构优化
在结构设计中，利用ENS进行快速偏微分方程求解，加速材料性能和应力分析，提升设计效率。
流体动力学仿真
ENS能在复杂流动场中实现高效模拟，支持航空航天、海洋工程等领域的实时监控与优化。
Long-term Vision
智能科学计算平台
结合ENS构建自主学习的偏微分方程求解平台，实现全自动化、智能化的科学仿真流程，推动科研自动化。
多物理场耦合模拟
未来ENS可扩展到多物理场耦合问题，如热-流-结构交互，推动多学科交叉的高精度模拟技术发展。
Abstract
Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution. Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss--Newton steps, but inherit the compute cost and instability of the underlying classical optimizers. We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill-conditioned systems, explaining why these methods often do not make accurate predictions despite achieving low residuals. We propose error-conditioned Neural Solvers (ENS), built on a different principle: rather than an optimization target, the PDE residual field is passed as a direct input to the network at each iteration, enabling it to read the spatial structure of its own errors and learn an update policy to iteratively correct its predictions. Across four PDE families, ENS attains the highest prediction accuracy in the large majority of settings, with gains reaching $10\times$ on turbulent Kolmogorov flow, while avoiding the expensive compute cost of hybrid methods. ENS's learned correction policy generalizes under distribution shift, including zero-shot parameter changes and cross-equation transfer, where its relative advantage is largest in the ill-conditioned regimes where residual minimization is least reliable. Project website: https://neuralsolver.github.io/.
References (20)
VideoPDE: Unified Generative PDE Solving via Video Inpainting Diffusion Models
Edward Li, Zichen Wang, Jiahe Huang et al.
Fourier Neural Operator for Parametric Partial Differential Equations
Zong-Yi Li, Nikola B. Kovachki, K. Azizzadenesheli et al.
Physics-Informed Neural Operator for Learning Partial Differential Equations
Zong-Yi Li, Hongkai Zheng, Nikola B. Kovachki et al.
DiffusionPDE: Generative PDE-Solving Under Partial Observation
Jiahe Huang, Guandao Yang, Zichen Wang et al.
Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints
Utkarsh Utkarsh, Pengfei Cai, Alan Edelman et al.
Learning to learn by gradient descent by gradient descent
Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo et al.
image or to image
G. Corino
Boundary element methods
William K. Smith
Iterative methods for sparse linear systems
Y. Saad
PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
Phillip Lippe, Bastiaan S. Veeling, P. Perdikaris et al.
Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms
P. Deuflhard
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
M. Raissi, P. Perdikaris, G. Karniadakis
Residual-based error corrector operator to enhance accuracy and reliability of neural operator surrogates of nonlinear variational boundary-value problems
Prashant K. Jha
De novo design of protein structure and function with RFdiffusion
Joseph L. Watson, David Juergens, N. Bennett et al.
Constrained Synthesis with Projected Diffusion Models
Jacob K. Christopher, Stephen Baek, Ferdinando Fioretto
Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs
Nikola B. Kovachki, Zong-Yi Li, Burigede Liu et al.
Crystal Diffusion Variational Autoencoder for Periodic Material Generation
Tian Xie, Xiang Fu, O. Ganea et al.
Beyond Loss Guidance: Using PDE Residuals as Spectral Attention in Diffusion Neural Operators
Medha Sawhney, Abhilash Neog, Mridul Khurana et al.
GenCast: Diffusion-based ensemble forecasting for medium-range weather
Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet et al.