Daniel Paulin (University of Oxford)

“Connections between optimization and sampling”

Abstract: In this talk, I am going to look at some connections between optimization and sampling. In “Hamiltonian descent methods’’, we introduce a new optimization method is based on conformal Hamiltonian dynamics. It was inspired by the literature on Hamiltonian MCMC methods. The use of general kinetic energies allows us to obtain linear rates of convergence for a much larger class than strongly convex and smooth functions. “Dual Space Preconditioning for Gradient Descent” applies a similar idea to gradient descent. We introduce a new optimization method based on nonlinear preconditioning of gradient descent, with simple and transparent conditions for convergence. “Randomized Hamiltonian Monte Carlo as Scaling Limit of the Bouncy Particle Sampler and Dimension-Free Convergence Rates” studies the high dimensional behaviour of the Bouncy Particle Sampler (BPS), a non-reversible piecewise deterministic MCMC method. Although the paths of this method are straight lines, we show that in high dimensions they converge to a Randomised Hamiltonian Monte Carlo (RHMC) process, whose paths are determined by the Hamiltonian dynamics. We also give a characterization of the mixing rate of the RHMC process for log-concave target distributions that can be used to tune the parameters of BPS.