In 1788, Lagrange presented a set of equations of motion that, unlike Newtonian mechanics, are independent of the choice of coordinates of the physical system, and ultimately led to the formulation of general relativity. Hamilton came up with a different set of equations of motion in 1833 that arguably led to the development of quantum mechanics. Remarkably, in classical mechanics, these sets of equations turn out to be equivalent via a beautiful duality due to Legendre.

A portrait of Legendre by H. Rousseau, E. Thomas, Augustin Challamel, and Desire Lacroix via Wikimedia Commons

Lagrangian and Hamiltonian dynamics have inspired several promising optimization and sampling algorithms such as first-order methods in optimization,  Hamiltonian Monte Carlo (see also this paper that will appear in NIPS 2018). Legendre duality also appears in convex optimization as Fenchel duality.  This note, written primarily for optimization folks, introduces Lagrangian dynamics, Hamiltonian dynamics, and proves the duality that connects them.

I hope that these fundamental ideas inspire you as well to think about optimization from a physics perspective!