How to derive Variational Free Energy: A Software Engineer's Guide - Part 1.

The static case: one observation, one hidden cause.

Scope: This derives the simplest case from Bogacz (2017): inferring a single fixed quantity from a single observation, with a linear-Gaussian model and fixed precisions. No dynamics, no time, no action yet — those will come later in the series. (For the mathematically inclined: this is the Laplace-approximated free energy, where the approximate posterior collapses to a point estimate — which is why the KL/entropy term you may have seen elsewhere doesn't appear here.)

Skip my ramblings, this isn't some hippy cookbook.

1. Introduction

If you've ever been like me, lying awake at night contemplating how to implement continuous generative models within the framework of Active Inference, then I'm genuinely surprised because I thought I was weird. In my spare time I am building Active Inference POMDP models with my partner in crime Kev, mostly using the pymdp python toolbox. Although this toolbox is brilliant and has been a joy to use, it currently doesn't support the creation of Continuous Generative Models (CGMs). This makes my current ultimate goal of creating a Mixed Generative Model (MGM) somewhat more difficult. To do this I need to understand how the fundamental equations of these models are derived, hence this post and subsequent posts. Due to its nature, there isn't a large amount of easily accessible content on Active Inference in general, so whilst learning how to code this stuff, I thought I might as well document it as I go.

So what is Variational Free Energy, before we drown in Greek? It is a mathematically tractable upper bound on sensory "surprise" (or negative log-evidence).

In this derivation we actually maximise negative VFE instead of the classic literature move of minimising positive VFE. They are approximately identical for the contents of this post.

This post follows the structure of Rafal Bogacz's 2017 paper, "A tutorial on the free-energy framework for modelling perception and learning". Brilliant, but dense enough at times to make this pleb software engineer's to make your eyes water. My job is to slow it right down, swap the heavy notation for runnable Python and plain variable names, and generally make it survivable to an none-academic. It's also the first in a series I'm writing to drag Active Inference into the engineering spotlight.

Note: There will be a lot of acronyms and symbols to deal with in this post. There is an appendix for both.

Our goal is to go from first principles, based on a simple organism, to this monstrosity:

It looks frightening now. By the end it'll look like an old friend who happens to owe you money. The whole thing is just two competing pulls wearing the latest Greek-algebra summer collection, and once you've seen what each piece is doing, the rest is bookkeeping. We'll build it up one honest step at a time, the same way I did in my notebook (mistakes and false starts included, because pretending I got it first try would be dishonest and hopefully you can laugh at me to give your brain a break).

Here's the roadmap. We set up a simple organism trying to guess one number, in this case the size of a food item. We write down the textbook-correct way to solve it, notice that the textbook-correct way involves an integral nobody wants to compute, and then quietly walk away from it. We then do the thing the brain (allegedly) actually does, which is find the single best guess instead of the whole distribution. That shortcut is what gives us Free Energy. Then we differentiate it, get an update rule, and stare at the two terms until they make sense.

2. The Simplest Case

2.1 Setting up the problem

Bogacz starts with a single value of a single variable being inferred by a single observation. One number in, one number out.

The story is this. We have a simple animal (Bogacz uses "animal" so I will too to keep it consistent, but I find "animal" does a lot of heavy lifting) that wants to infer the size of a food item. We'll call this size $v$. The animal can't see size directly. What it can do is sense light intensity, because it has a single light-sensitive receptor, and bigger things reflect more light. The sensed light intensity is $u$, and crucially $u$ has noise.

So our two characters:

• $v$ — the size of the food (the hidden cause we want to infer)
• $u$ — the noisy light intensity the receptor actually reports (the observation)

Now we need to connect them. Light reflected off an object scales with its area, so Bogacz picks a nice clean non-linear function relating average light intensity to size:

Keep an eye on that $g$. It's the animal's internal model of "how my sensor responds to the world".

Next, the noise. The animal's receptor doesn't report $g(v)$ exactly; it reports something scattered around it. Bogacz assumes the perceived light intensity is normally distributed with mean $g(v)$ and variance $\Sigma_u$. In symbols, the likelihood of an observation given a size is:

where $f(x; \mu, \Sigma)$ is the density of a normal distribution which is shown with this formula:

Read $f(x; \mu, \Sigma)$ as "the bell curve evaluated at $x$, for a distribution whose mean is $\mu$ and whose variance is $\Sigma$". The semicolon is just separating the value we're plugging in (on the left) from the two settings that define the curve's shape and position (on the right). I'll use the $f$ notation throughout — whenever you see it, it's the same formula, just with different things slotted into the blanks.

Side note for Bogacz, and worth repeating here: a Gaussian isn't really the right distribution for light intensity, because a Gaussian happily applies probabilities to negative numbers and there's no such thing as negative brightness. But it's clean, it's tractable, and it makes the maths behave, so let's just agree to look the other way, for now. This is a recurring theme in modelling: pick the distribution that lets you finish the derivation, apologise in the footnotes.

The last important piece we need for setup is the idea of prior knowledge. A habitual expectation of how big food usually is, before it sees anything at all. This is the bit that makes it Bayesian rather than just "trust the wonky sensor blindly".

For simplicity we assume the animal expects size to be normally distributed too, with mean $v_p$ and variance $\Sigma_p$, where the subscript $p$ stands for prior:

So now we have both parts that Bayes needs: a likelihood $p(u \mid v)$ (how observations relate to causes) and a prior $p(v)$ (what we believe before looking).

2.2 Stating the result

Here's where we're headed, so you know what the destination looks like before we start the journey.

The exact answer to "given what I saw, how likely is each possible size?" is the posterior $p(v \mid u)$, delivered by Bayes' theorem:

That's the honest, complete, textbook-correct answer. And as we'll see in a second, it contains a landmine in the denominator. So instead of computing the whole posterior, we'll settle for finding the single most likely size (the peak of the posterior) which we'll call $\phi$. The quantity we maximise to find $\phi$ turns out to be (the log of) the numerator of Bayes' rule, and that quantity is what gets called Free Energy:

Maximise $F$, find your best guess. If your head hurts, don't worry, mine did too — first deriving it myself, then trying to think of how to write about how I derived it. I've got double your headaches...loser.

To make "the peak of the posterior" concrete, here's that curve actually plotted — the exact posterior $p(v \mid u)$ for the food-size example, computed by solving Bogacz Exercise 1 (Python in Appendix C). This is the whole target: the single value of $v$ sitting under the peak is $\phi$, our best-guess size, and everything from here on is about reaching that peak without computing the whole curve.

2.3 Derivation

The exact solution, and the integral that we run away from

To work out how likely the range of different sizes $v$ is, given the observed input $u$, we use Bayes' theorem, equation (4) above. The numerator, $p(v)\,p(u \mid v)$, is just prior multiplied by likelihood. The problem is the denominator, $p(u)$.

The denominator is the normaliser. It guarantees that the posterior probabilities over all possible sizes integrate to 1. To achieve it, you have to integrate the numerator over every possible size:

This is the landmine. For our manufactured simple case with friendly Gaussians, it's doable. Try Exercise 1 in the Bogacz paper; the Python solution is in Appendix C if you get stuck. In any realistic model (many variables, non-linear $g$) this integral is intractable (meaning very difficult or impossible to control, manage, or solve). It's the wall the entire free-energy framework exists to climb over. So rather than smashing our heads against it, we change the question.

Finding the most likely value (the MAP shortcut)

Instead of finding the whole curve $p(v \mid u)$, we go after the single value of $v$ that maximises $p(v \mid u)$. We call that value $\phi$, our best-guess size, and we call $p(\phi \mid u)$ the posterior probability density at that point.

Bogacz notes (and it's a fairly heavy assumption) that it's reasonable to think the brain represents, at any given moment, only the most likely values of features rather than full distributions. The single best guess, not the entire belief. Bogacz uses binocular rivalry as evidence of this assumption. If you don't know what that is, expand the section below.

This next bit is the pivot the whole derivation hinges on, so I'm going to lay it out slowly. Here's the logic, lightly paraphrased and then broken down:

We look for the value $\phi$ which maximises the posterior $p(\phi \mid u)$. By equation (4), that posterior depends on a ratio of two quantities, but the denominator $p(u)$ does not depend on $\phi$. Therefore the value of $\phi$ that maximises the posterior is the same value that maximises the numerator. We denote the logarithm of that numerator by $F$ (it

's related to negative free energy, as we'll see).

Let me unpack that, because it's doing three separate clever things at once:

• "We look for a value which maximises..." — The animal wants its single best guess for food size. This is the peak of the curve from before. Instead of calculating the whole curve, it just wants to know where the top is.

• "The posterior depends on a ratio, but the denominator doesn't depend on $\phi$." — Bayes' rule is prior x likelihood / normaliser. That normaliser, $p(u)$, is the integral. But it does not depend on the guess $\phi$. It's a fixed number once $u$ is observed.

• "Thus the value which maximises it is the same one which maximises the numerator." — This is the crux. Dividing everything by the same constant just rescales the curve; it doesn't move where the peak is. Like that one friend at a party that's a downer. They don't change the location of the party but they bring everybody down. So we can completely ignore that friend, I mean, the hard-to-compute denominator $p(u)$, and just maximise the numerator, keeping the party alive. The location of the maximum is unchanged.

Then one final convenience: we take the log of the numerator before maximising. This is safe because log is monotonic (a fancy way of saying it never reorders things, so it also doesn't move the peak). Like everyone wearing a hat at the party...something something...worn out party analogy. If $a > b$ then $\ln a > \ln b$, always. Taking logs also turns all our products into sums (via $\ln(ab) = \ln a + \ln b$), which is going to make the calculus nicer in about thirty seconds.

In summary: the denominator is fixed scaling that does not affect where the peak is, so we throw it away. We take the log of what's left (products become sums, peak stays put), and we call the result $F$, for Free Energy, and we maximise that.

That gives us equation (7):

Two terms: the log-prior and the log-likelihood. That's the whole objective. Now we substitute in the actual Gaussians and grind.

Substituting in the Gaussians (a wall of maths, honestly)

The plan: Take equation (7), sub in the prior (3) and the likelihood (1), each of which is the Gaussian density (2), and simplify using log rules until something clean falls out. Bogacz basically says "do this" and then shows a wall of algebra. So here's that wall. Feel free to try it out yourself.

Starting from (7) and substituting the two Gaussian densities:

Each $\ln$ of a product splits into a log of the front fraction plus a log of the exponential. The log of an exponential is just its exponent, so the $\exp$ and $\ln$ annihilate and leave the quadratic behind:

Now look at those two leftover log terms, $\ln\!\left(\frac{1}{\sqrt{2\pi\Sigma_p}}\right)$ and $\ln\!\left(\frac{1}{\sqrt{2\pi\Sigma_u}}\right)$. We're eventually going to differentiate $F$ with respect to $\phi$. Neither of those terms contain a $\phi$. So when we differentiate, they vanish.

So Bogacz groups them into a single constant term $C$:

That's our objective function. A log-prior penalty, a log-likelihood penalty, and a constant we'll never think about again. Well done, go grab a drink.

Confession: the first time I did this I tried to expand all the $\ln\!\left(\frac{1}{\sqrt{2\pi\Sigma}}\right)$ terms out fully — splitting them into $-\tfrac{1}{2}\ln(2\pi\Sigma)$, then into $-\tfrac{1}{2}\ln(2\pi) - \tfrac{1}{2}\ln\Sigma$, and so on, dragging every $2\pi$ along for the ride. I filled half a page, wrote "Wrong, started again" in big letters, and underlined it. It wasn't actually wrong, it was just a waste of effort, because all those $2\pi$ bits are about to get swept into a constant anyway. Learn from my pain: don't expand things you're about to throw away.

Differentiating to get the update rule

We've got $F$ as a function of our guess $\phi$. To find the $\phi$ that maximises it, we need the gradient $\frac{\partial F}{\partial \phi}$. The constant $C$ goes away. The two $-\ln\Sigma$ terms also go, since they don't contain a $\phi$ either. That leaves the two quadratics.

Differentiating $-\frac{(\phi - v_p)^2}{2\Sigma_p}$ with respect to $\phi$ gives $-\frac{(\phi - v_p)}{\Sigma_p}$, which flips to $\frac{v_p - \phi}{\Sigma_p}$.

The second term, $-\frac{(u - g(\phi))^2}{2\Sigma_u}$, needs the chain rule, because $g(\phi)$ is a function tucked inside another function.

The inner function is $g(\phi)$, so its derivative $g'(\phi)$ gets pulled out front. Working it through gives $\frac{(u - g(\phi))}{\Sigma_u} \cdot g'(\phi)$.

Putting both together:

Quick note on that $g'(\phi)$ — it matters enormously. Remember the animal's sensor model was $g(v) = v^2$. So:

$$g(\phi) = \phi^2 \quad\Rightarrow\quad g'(\phi) = 2\phi$$

This is the only place the specific shape of the sensor enters the update. Swap in a different $g$ and only $g'(\phi)$ changes — everything else in the equation is structural.

Gradient ascent: letting the guess move

Equation (9) tells us the slope of $F$ at our current guess. To actually find the peak, we apply gradient ascent. Bogacz writes it about as simple as it can be written:

In English this means "let your guess $\phi$ change over time at the rate of the gradient". If $F$ slopes upwards at your current guess, $\dot\phi$ is positive and $\phi$ moves in the upward direction. At the peak, $\frac{\partial F}{\partial\phi} = 0$, so $\dot\phi = 0$, and the guess stops moving. It's settled on the best estimate.

Worth flagging: this is gradient ascent, not descent — notice there's no minus sign. But hang on, doesn't everyone bang on about minimising free energy? They do, and there's no contradiction; it's a naming mismatch that we addressed in an earlier paragraph. The quantity we've been building, $F$, came straight out of logging the numerator of Bayes' rule, so it's a goodness score we want to push up (most probable guess = top of the hill). The free energy the literature tells you to minimise is defined the other way up, as a proxy for surprise, which you obviously want down. The two are just negatives of each other: $F_{\text{literature}} \approx -F_{\text{ours}}$. Climbing our hill is descending their valley. Bogacz didn't flip his terms to match the convention because his derivation hands him this version for free; negating everything by hand just to agree with a sign convention would've added minus signs to every equation that follows, for zero benefit. So: we ascend, and we call it negative free energy to keep our consciences clear.

The two terms and their meaning

We're going to take a break from the algebra for a second and go back to the narrative. The update to $\phi$ is driven by two terms in equation (9). The first pulls the guess towards the prior; the second pulls it according to the sensory input. Let's take a minute to recap each.

That's the whole engine. The animal's belief is a tug-of-war between "this is what I usually expect" (the prior) and "this is what I'm actually seeing right now" (the evidence), each weighted by how much it trusts the respective source. Inference is just letting that tug-of-war settle. Stitch the two terms back together and you've got the monstrosity from the very top of the post:

Appendix A — The full algebra

For completeness, here's the full expansion I shortcut in the main text. If you want every $2\pi$ accounted for rather than swept into $C$, this is for you.

Start from equation (7) with both Gaussians substituted in. Take one term at a time. Using $\ln(ab) = \ln a + \ln b$ on the prior term:

Now expand that front log. Rewrite the fraction as a power, $\frac{1}{\sqrt{2\pi\Sigma_p}} = (2\pi\Sigma_p)^{-1/2}$, and pull the exponent out with $\ln(b^x) = x\ln b$:

Split the product inside with $\ln(ab) = \ln a + \ln b$ again:

The $-\tfrac{1}{2}\ln(2\pi)$ piece contains no $\phi$ and no variance worth tracking — it's a pure constant. The same expansion applies identically to the likelihood term with $\Sigma_u$ and $g(\phi)$ in place of $\Sigma_p$ and $v_p$. Doing both and collecting:

The two $\ln\!\left(\tfrac{1}{\sqrt{2\pi}}\right)$ terms are constants. Roll them — and only them — into $C$, factor out the $\tfrac12$, and you land exactly on the boxed result from the main text:

The moral, again: the $\Sigma$-logs survive the simplification but disappears under differentiation (they're constant in $\phi$ when precisions are fixed), and the $2\pi$ junk was never worth carrying. Expand only what you need.

Appendix B — Bogacz Exercises

Bogacz peppers the tutorial with exercises, and they're genuinely worth doing. They're the difference between following along and actually building the thing. I left them out of the main flow to not repeat their work and to keep this easier to read. But I would recommend taking a look at them yourself. They live in the relevant chapters of the tutorial:

• Exercise 1 (Chapter 2) — plotting the exact posterior $p(v \mid u)$ for the food-size example. This is the curve whose peak we spent the whole post chasing.
• Exercise 2 (Chapter 2.2 / 2.3) — implementing the gradient ascent on $\phi$ and watching it converge.

My solutions, written in Python, can be seen in Appendix C

Appendix C - Python solutions to Bogacz Exercises

import matplotlib.pyplot as plt
import numpy as np
from scipy.stats import norm


def exercise_1():
    v_p = 3
    sigma_p = 1
    sigma_u = 1

    u = 2

    vrange = np.arange(0.01,5,0.01)

    numerator = norm.pdf(vrange, v_p, sigma_p) * norm.pdf(u, vrange**2, sigma_u) # Posterior = Prior * likelihood
    denominator = sum(numerator*0.01)
    p = numerator/denominator

    plt.plot(vrange, p)
    plt.ylim(ymin=0)
    plt.xlabel("v (size)")
    plt.ylabel("posterior probability")
    plt.show()

exercise_1()

import matplotlib.pyplot as plt
import numpy as np


def exercise_2():
    v_p = 3
    sigma_p = 1
    sigma_u = 1

    u = 2

    phi=[]
    phi: list[int|float] = [v_p]
    Dt = 0.01
    max_t = 5
    n_steps = int(max_t/Dt)

    for _ in range(n_steps):
        current = phi[-1]
        prior_expectation_pull = (v_p - current)/sigma_p
        observation_pull = ((u - current**2)/sigma_u)*2*current

        # Using Euler's method phi(t+Dt) = phi(t) + Dt*(dF/dphi)
        phi.append(current + Dt*(prior_expectation_pull+observation_pull))

    time = np.arange(len(phi)) * Dt

    plt.plot(time, phi, "k")
    plt.ylim(ymin=0)
    plt.xlabel("time")
    plt.ylabel(r"$\phi$ (guess of v)")
    plt.show()

exercise_2()

Notation Summary

As promised up top, here's the appendix for the symbols and acronyms. Skim it once now if you like, or just flick back whenever a piece of Greek catches you out.

Acronyms

Symbols

References

1. Bogacz, R. (2017). A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology, 76, 198–211.

2. Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020). Active inference on discrete state-spaces: A synthesis. Journal of Mathematical Psychology, 99, 102447.

3. Smith, R., Friston, K. J., & Whyte, C. J. (2022). A step-by-step tutorial on active inference and its application to empirical data. Journal of Mathematical Psychology, 107, 102632.