Approximate Bayesian inference, and associated theory

A long-standing open problem is approximate Bayes inference methods (variational inference in particular) is to understand statistical properties of the estimation procedure. We are able to provide general conditions for obtaining optimal risk bounds for point estimates acquired from mean-field variational Bayesian inference. The conditions pertain to the flexibility of the variational family and existence of certain test functions for the distance metric on the parameter space and minimal assumptions on the prior concentration around the true parameter.

• Yang Y., Pati D., Bhattacharya A. (2020). α-Variational inference with statistical guarantees, The Annals of Statistics, 48, (2) 886–905.
• Pati D., Bhattacharya A., Yang Y (2018) On statistical optimality of Variational Bayes. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (AISTATS), 1579–1588.
• Bhattacharya A., Pati D., Yang Y (2019). Bayesian fractional posteriors. The Annals of Statistics, 47 (1): 39-66.
• Ghosh I., Bhattacharya A., Pati D. (2022) Statistical optimality and stability of tangent transform algorithms in logit models. , Journal of Machine Learning Research, to appear.
• Plummer S., Zhou S., Bhattacharya A., Dunson D.B., Pati D. (2021) Statistical Guarantees for Transformation Based Models with applications to Implicit Variational nference. AISTATS 2021, to appear.
• Guha B., Bhattacharya A., Pati D. (2021) Statistical Guarantees and Algorithmic onvergence Issues of Variational Boosting, IEEE-ICTAI.
• Wang H., Bhattacharya A., Yang Y., Pati D. (2022) Structured Variational Inference in Bayesian State-Space Models, AISTATS 2022, to appear.

Developing hierarchical models for non-Euclidean objects

A collaboration with the Duke Brain Center about 10 years ago sparked my interest in probabilistic modeling of shapes to intelligently combine expert traces of brain tumors to construct a high-credibility region, prior to excising them. Apart from solving the specific problem using a multiscale Bayesian model for closed curves, my research during the last 10 years has tackled how to model closed surfaces, cluster protein sequences, model densities and conditional densities using deformations and model shape restricted densities that often arise in engineering and biomedical applications. The articles are a successful synergy of two key ideas: i) novel parameterized models for the underlying object (closed curves, surfaces, deformable template) ii) a novel metric on the shape-space that is invariant to shape-preserving transformations, useful for comparing shapes. These made coherent probabilistic inference on shapes possible, which was previously missing in the literature.

• Gu K, Pati D. and Dunson D.B. (2014) Bayesian multiscale modeling of closed curves in point clouds, Journal of the American Statistical Association, 109 (508): 1481-1494.
• Binette O, Pati D. and Dunson D.B. (2020) Bayesian fitting of closed surfaces hrough tensor products, The Journal of Machine Learning Research, 21, (119) 1–26.
• Dasgupta S., Pati D., Jermyn I, Srivastava A.(2020) Modality-Constrained Density Estimation via Deformable Templates. Technometrics, to appear.
• Dasgupta S. Pati D., Jermyn I, Srivastava A. (2018) Shape-Constrained and Unconstrained Density Estimation Using Geometric Exploration, 2018 IEEE Statistical Signal Processing Workshop (SSP), to appear.
• Dasgupta S., Pati D., Srivastava A. (2019) Bayesian Shape-Constrained Density Estimation, Quarterly of Applied Mathematics.
• Dasgupta S., Pati D., Srivastava A. (2019) A two-step geometric framework for density modeling. Statistica Sinica, to appear.

Measurement error models in nutritional epidemiology

Development of flexible and efficient Bayesian semiparametric methodology for measurement error problems is another direction of my collaborative research with Abhra Sarkar, Ray Carroll and Bani Mallick. In previously existing literature, solutions to even the most fundamental measurement error problems, like density deconvolution and regression with errors-in-predictors, were available only under numerous simplifying and unrealistic assumptions. Addressing these problems using a Bayesian framework that can accommodate measurement errors through natural hierarchies, we proposed efficient semiparametric solutions that relaxed many restrictive assumptions of previously existing techniques but also encompassed diverse simplified scenarios as special cases. The methodology we developed vastly outperformed its competitors even in simplified scenarios for which the competitors were specifically designed.

• Sarkar A., Mallick B., Staudenmayer J., Pati D., Carroll R.J. (2014) Bayesian Semiparametric Density Deconvolution in the Presence of Conditionally Heteroscedastic Measurement Errors. Winner of the SBSS student paper award. Journal of Computational and Graphical Statistics, 23 (4): 1101-1125.
• Sarkar A., Pati D., Chakrabarty A., Mallick B., Carroll R.J. (2018) Bayesian semiparametric multivariate density deconvolution. Journal of the American Statistical Association, 113 (521): 401-416
• Sarkar A., Pati D., Mallick B., Carroll R.J. (2021) Bayesian Copula Density Deconvolution for Zero-Inflated Data with Applications in Nutritional Epidemiology. Journal of the American Statistical Association, to appear.

Bayesian shrinkage priors for high-dimensional data

With the recent flurry of activities in high-throughput data, we now routinely encounter complex outcomes in the form of high-dimensional arrays with the number of model parameters p greatly exceeding the sample-size even for the simplest parametric models.  The seemingly impossible task of inferring (p >> n) parameters compels us to investigate and exploit lower dimensional structures underlying the data generating process.  A Bayesian approach allows us to incorporate prior knowledge about sparsity of model parameters and obtain a probabilistic characterization of uncertainty through the posterior distribution. However, the choice of the prior distribution on a high-dimensional space poses a tricky challenge, and one is likely to obtain very misleading inferences unless care is exercised while applying commonly used priors to high-dimensional settings. This is another of my long-term research direction which focuses on identifying these areas, and developing optimal prior distributions.

• Bhattacharya , Pati D., Pillai N.S. and Dunson D.B. (2016) Sub-optimality of some continuous shrinkage priors. Stochastic Processes and their Applications, 126(12):  382–3842.  (Invited paper in memoriam: Evarist Gine´ )
• Bhattacharya A., Pati D., Pillai N.S. and Dunson D.B. (2015) Dirichlet Laplace priors for optimal shrinkage. Journal of the American Statistical Association, 110 (512): 1479-1489.
• Li H., Pati D. (2017) Variable selection using shrinkage priors. Computational Statistics & Data Analysis, 7, 107–119.
• Pati D., Bhattacharya , Pillai N.S. and Dunson D.B. (2014) Posterior contraction in sparse Bayesian factor models for massive covariance matrices. The Annals of Statistics, 42 (3): 1102-1130.
• Sabnis G., Pati D., Bhattacharya A. (2018) Compressed covariance estimation with automated dimension learning. Sankhya (Series A).
• Zhou S., Ray P., Pati D., Bhattacharya A. (2022) Mass-shifting phenomenon of truncated multivariate normal priors. Journal of the American Statistical Association, to appear.

Gaussian process theory

While Gaussian processes (GPs) are highly successful for function estimation, we observed that in high dimensions, when the underlying true function has anisotropic smoothness, a GP with isotropic smoothness is sub-optimal. We proposed a Gaussian process with dimension specific scalings with joint a Dirichlet induced shrinkage that can adapt to the true anisotropy. The construction of this joint prior is motivated to overcome the concentration of measure trap (often present in independent prior specification) through a hierarchical specification, where the inverse-bandwidth parameters in lower level of the hierarchy are assigned independent Gamma priors with exponents jointly drawn from a Dirichlet distribution. This idea of breaking the prior independence to avoid unwanted concentration of measure has a substantial impact (refer to the previous line of research); as a shrinkage prior for vectors,, covariance matrices and later in BART (Linero, 2018, JASA) for variable selection and interaction recovery. In addition, we have worked on sparse Gaussian processes in high dimensions, extending Gaussian process theory for random covariate design and approximating posteriors obtained from Gaussian process models in Wasserstein metric.

• Bhattacharya A., Pati D. and Dunson D.B. (2014) Anisotropic function estimation using multi-bandwidth Gaussian processes. The Annals of Statistics, 32 (1): 352-381.
• Pati D., Bhattacharya A., Cheng G. (2015) Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior. The Journal of Machine Learning Research, 16, 2837-2851.
• Bhattacharya A., Pati D. (2017) Posterior contraction in Gaussian process regression using Wasserstein approximations. Information and Inference, 6, 416–440.

Bayes theory for network and graphical models

Understanding the dependence structure in complex high-dimensional objects is a fundamental problem in modern statistical theory and methods. I am specifically interested discovering communities in large networks.

• Geng J., Bhattacharya A., Pati D. (2019) Probabilistic community detection with unknown number of communities. Journal of the American Statistical Association. 114, (526) 893–905.
• Ghosh P., Pati D., Bhattacharya A. (2019) Optimal Bayesian estimation in stochastic block models. Sankhya Series A, (invited for special volume in honor of Prof. J.K. Ghosh) to appear.
• Niu Y., Pati D., Mallick, B. (2021) Bayesian Graph Selection Consistency Under Model Mis- specification. Bernoulli. 27, (1), 637–672. Winner of SETCASA Poster competition 2019 Second prize.

Applications in Nuclear Physics

A collaboration with Pablo Giulani and Jorge Piekarewicz at Florida State got me interested in nuclear Physics applications. The datasets often have very small amount of experimental data and hence rely heavily on constraints imposed by the laws of Physics. Bayesian statistics provides a coherent way to combine this prior belief with experimental data to obtain meaningful inference.

• Zhou, S., Giuliani, P., Piekarewicz, J., Bhattacharya A., Pati, D. (2019) Reexamining the proton-radius problem using constrained Gaussian processes. Winner of SETCASA Poster competition 2019 Second prize. Physical Review C, 99 (5): 055202
• Lim Y., Bhattacharya A., Holt J. W., Pati D. (2021) Revisiting constraints on the maximum neutron star mass in light of GW190814. Letter in Physical Review C, to appear.

Collaborative projects in Biomedicine

Some of my collaborative works evolved out of collaborations with biomedical scientists, particularly in the areas of tumor tracking and oral health-care.

• Cervone D., Pillai N.S., Pati D., Berbecko R. and Lewis J.H. (2014) A location-mixture auto-regressive model for online forecasting of lung-tumor motion. The Annals of Applied Statistics, 8 (3): 1341-1371.
• Bandyopadhyay D., Hilden P., Pati D., Fernandes J., Russell S. L., Fellows J. L., Nagarajan R. (2021). Correlated tooth-level caries status in a Type-2 diabetic Gullah population, Modern Approaches in Dentistry and Oral Health Care, to appear.