Online student-t processes with an overall-local scale structure for modelling non-stationary data
Sha, T. and Zhang, M. M. (2025). In The 28th International Conference on Artificial Intelligence and Statistics (AISTATS 2025).
Presented as a poster at AISTATS 2025 & AABI 2025
Abstract
Mixture-of-expert (MOE) models are popular methods in machine learning, since they can model heterogeneous behaviour across the space of the data using an ensemble collection of learners. These models are especially useful for modelling dynamic data as time-dependent data often exhibit non-stationarity and heavy-tailed errors, which may be inappropriate to model with a typical single expert model. We propose a mixture of Student-t processes with an adaptive structure for the covariance and noise behaviour for each mixture. Moreover, we use a sequential Monte Carlo (SMC) sampler to perform online inference as data arrive in real time. We demonstrate the superiority of our proposed approach over other models on synthetic and real-world datasets to prove the necessity of the novel method.
Greedy Empirical Bayes for Density Estimation
Sha, T., Yu, S., and Fong, E.∗ (2026). Preprint.
∗ corresponding author
Presented as a poster at ISBA 2026 (International Society for Bayesian Analysis World Meeting)
Abstract
Bayesian nonparametric density estimation is challenging in high dimensions, where traditional approaches like the Dirichlet process (DP) mixture rely on computationally intensive Markov chain Monte Carlo or approximations. We introduce the partition posterior bootstrap (PPB), where a flexible prior is constructed by smoothing a DP over a partition of the sample space. Conditioned on this partition, exact posterior sampling reduces to a simple Bayesian bootstrap-like algorithm. Importantly, we select the partition using a greedy empirical Bayes (GEB) algorithm which scales to high dimensions and large sample sizes. GEB greedily maximizes the closed-form marginal likelihood via local split decisions arising naturally from its factorization, yielding an adaptive binary tree partition. Our key theoretical result is that the GEB tree belongs to an admissible family with high probability, splitting all identifiable signals while avoiding over-splitting noise. As a result, the PPB posterior with the GEB tree attains adaptive contraction rates. Our theory holds under weak, verifiable assumptions, which we demonstrate in numerous examples. Empirically, PPB delivers accurate density estimates and uncertainty quantification at a fraction of the computational cost of other approaches, making it a practical off-the-shelf choice for high-dimensional Bayesian density estimation with theoretical guarantees.