#
28 Feb 2018
Mean Field Methods for Exponential Families

\[\newcommand{\E}{\mathbb{E}}
\newcommand{\brac}[1]{\left[#1\right]}
\newcommand{\paren}[1]{\left(#1\right)}
\newcommand{\set}[1]{\left\{#1\right\}}
\newcommand{\KL}{\text{KL}}
\DeclareMathOperator*{\minimize}{min.}
\DeclareMathOperator*{\maximize}{max.}
\renewcommand{\L}{\mathcal{L}}
\newcommand{\A}{\mathcal{A}}
\newcommand{\Expect}{\mathbb{E}}
\newcommand{\Q}{\mathcal{Q}}
\newcommand{\X}{\mathcal{X}}
\newcommand{\D}{\mathcal{D}}
\newcommand{\giv}{\mid}\]
I recently started reading Wainwright and Jordan’s “book” on Exponential Families as part of a reading group. I was in charge of presenting Chapter 5 (Mean Field Methods) today and wrote up some notes to facilitate the presentation. I think this chapter is really nice as it showcases how people thought about variational methods before black-box variational inference with deep neural networks became popular. I’m sharing my chapter summary here in case you’re interested in understanding the classical approach to variational inference. And if you’re comfortable with the level of math in this summary, I highly recommend reading the seminal book; it’s a difficult but rewarding read!

Mean Field Methods Chapter Summary

End of post