Global implicit function theorems and the online expectation –maximisation algorithm

SummaryThe expectation –maximisation (EM) algorithm framework is an important tool for statistical computation. Due to the changing nature of data, online and mini-batch variants of EM and EM-like algorithms have become increasingly popular. The consistency of the estimator sequences that are produced by these EM varian ts often rely on an assumption regarding the continuous differentiability of a parameter update function. In many cases, the parameter update function is not in closed form and may only be defined implicitly, which makes the verification of the continuous differentiability property difficult. We dem onstrate how a global implicit function theorem can be used to verify such properties in the cases of finite mixtures of distributions in the exponential family, and more generally, when the component-specific distributions admit data augmentation schemes, within the exponential family. We then illu strate the use of such a theorem in the cases of mixtures of beta distributions, gamma distributions, fully visible Boltzmann machines and Student distributions. Via numerical simulations, we provide empirical evidence towards the consistency of the online EM algorithm parameter estimates in such ca ses.

Source: Australian and New Zealand Journal of Statistics - January 28, 2022 Category: Statistics Authors: Hien Duy Nguyen, Florence Forbes Tags: Geoff McLachlan Festschrift Source Type: research