Generative Bayesian Computation as a Scalable Alternative to Gaussian Process Surrogates
arXiv:2602.21408v1 Announce Type: new Abstract: Gaussian process (GP) surrogates are the default tool for emulating expensive computer experiments, but cubic cost, stationarity assumptions, and Gaussian predictive distributions limit their reach. We propose Generative Bayesian Computation (GBC) via Implicit Quantile Networks (IQNs) as a surrogate framework that targets all three limitations. GBC learns the full conditional quantile function from input--output pairs; at test time, a single forward pass per quantile level produces draws from the predictive distribution. Across fourteen benchmarks we compare GBC to four GP-based methods. GBC improves CRPS by 11--26\% on piecewise jump-process benchmarks, by 14\% on a ten-dimensional Friedman function, and scales linearly to 90,000 training points where dense-covariance GPs are infeasible. A boundary-augmented variant matches or outperforms Modular Jump GPs on two-dimensional jump datasets (up to 46\% CRPS improvement). In active learni
arXiv:2602.21408v1 Announce Type: new Abstract: Gaussian process (GP) surrogates are the default tool for emulating expensive computer experiments, but cubic cost, stationarity assumptions, and Gaussian predictive distributions limit their reach. We propose Generative Bayesian Computation (GBC) via Implicit Quantile Networks (IQNs) as a surrogate framework that targets all three limitations. GBC learns the full conditional quantile function from input--output pairs; at test time, a single forward pass per quantile level produces draws from the predictive distribution. Across fourteen benchmarks we compare GBC to four GP-based methods. GBC improves CRPS by 11--26\% on piecewise jump-process benchmarks, by 14\% on a ten-dimensional Friedman function, and scales linearly to 90,000 training points where dense-covariance GPs are infeasible. A boundary-augmented variant matches or outperforms Modular Jump GPs on two-dimensional jump datasets (up to 46\% CRPS improvement). In active learning, a randomized-prior IQN ensemble achieves nearly three times lower RMSE than deep GP active learning on Rocket LGBB. Overall, GBC records a favorable point estimate in 12 of 14 comparisons. GPs retain an edge on smooth surfaces where their smoothness prior provides effective regularization.
Executive Summary
This article proposes Generative Bayesian Computation (GBC) as a scalable alternative to Gaussian Process (GP) surrogates for emulating expensive computer experiments. GBC learns the full conditional quantile function from input-output pairs and produces draws from the predictive distribution with a single forward pass per quantile level. The approach is compared to four GP-based methods across fourteen benchmarks, demonstrating improved performance and scalability, particularly in cases with non-smooth surfaces or high-dimensional data.
Key Points
- ▸ GBC targets the limitations of GP surrogates, including cubic cost, stationarity assumptions, and Gaussian predictive distributions
- ▸ GBC learns the full conditional quantile function from input-output pairs
- ▸ GBC scales linearly to 90,000 training points, outperforming dense-covariance GPs
Merits
Scalability
GBC demonstrates linear scalability to large datasets, making it a viable alternative to GP surrogates in high-dimensional or complex problems
Flexibility
GBC can handle non-smooth surfaces and non-Gaussian predictive distributions, making it a more robust approach than traditional GP methods
Demerits
Smoothness Prior
GBC may not perform as well as GP surrogates on smooth surfaces, where the smoothness prior provides effective regularization
Computational Cost
While GBC scales linearly, the computational cost of training and evaluating the model may still be prohibitively expensive for very large datasets
Expert Commentary
The proposed GBC approach represents a significant advancement in the field of surrogate modeling, offering a scalable and flexible alternative to traditional GP methods. While GBC may not be suitable for all applications, its ability to handle non-smooth surfaces and non-Gaussian predictive distributions makes it a promising tool for a wide range of complex problems. Further research is needed to fully explore the potential of GBC and to develop more efficient and effective training methods.
Recommendations
- ✓ Further investigation of GBC's performance in active learning scenarios, particularly with respect to its ability to balance exploration and exploitation
- ✓ Development of more efficient training methods for GBC, such as parallelization or distributed computing approaches, to reduce computational cost and improve scalability
