What's the deal with free energy?
To cite this page
@misc{swpkim2024what's,
author={P. Kim, Sun Woo},
title={What's the deal with free energy?},
year={2023},
howpublished={\url{https://sunwoo-kim.github.io/en/posts/free-energy/}},
}
The term ‘free energy’ confused me for a long time. One one hand, when we learn thermodynamics, people talk about the Helmholtz free energy, $F=U-TS$. On the other hand, in the literature, people seem to describe the logarithm of a partition function to be the ‘free energy’, too. They must be related, but how?
In this post, I’m going to try to make the connection clear as I can, starting from the basic assumptions of statistical mechanics. To do this I will essentially re-derive the canonical ensemble, and point out some things along the way. I will also work in units of energy where $k_B=1$. I assume that you have read some notes on statistical physics, but was left confused as I certainly was.
Consider a system $S$ that is in contact with with a bath $R$, and that $S$ and $R$ together are isolated. Let’s say we know what the total energy, $E_T$, is. We know that $E_T$ is fixed since we said $S$ and $R$ together are isolated.
Let’s say that $S$ can have energies ${ E_S}$, and that the number of states (some people call it the statistical weight) that $S$ has with energy $E_S$ is $\Omega_S(E_S)$. $R$ has energies ${E_R}$, and the number of states that $R$ has with energy $E_R$ is $\Omega_R(E_R)$.
We are going to apply the ‘fundamental postulate of statistical mechanics’ on the combined system $S$ and $R$, which just means that we will assume that any state of the combined system, as long as it has total energy $E_T$, is equally likely.
The total number of states for the joint system $S$ and $R$ is then 1 $$ \Omega(E_{T}) = \sum_{E_S} \Omega_R(E_R=E_T - E_S) \Omega_S(E_S). $$ This is counting all the different ways that the total energy can be shared between $S$ and $R$. Entropy is defined as $S := \ln \Omega$. So then we have $$ \Omega(E_T) = \sum_{E_S} e^{S_R(E_T - E_S) + S_S(E_S)}. $$ Let us assume that the bath is large, and so its energy is much larger than the system’s $E_S \ll E_T$. Taylor expanding to first order, we have $$ \Omega (E_T) \approx \sum_{E_S} e^{S_S(E_S)} \exp \left(S_R(E_T)- \frac{\partial S_R}{\partial E_R} E_S\right). $$
Inverse temperature is defined as $T^{-1} = \partial_{E_R} S_R$. So the total number of states is given by
Here, we made the assumption that $\partial_{E_R} S_R$ is positive. This means that we assume that the entropy of the bath at some energy increases with its energy, or put in another way, that the number of states for the bath increase with energy. This is a sensible assumption; we expect there to be more states at higher energies.
Now consider $E^*$, which is the energy that maximises the Boltzmann weight $e^{-F(E)/T}$ and therefore minimises $F(E)$ and $f(E)$. We can write the probability as $$ p(E) = \frac{e^{N(f(E^*)-f(E))/T}}{1 + \sum_{E’ \neq E^*} e^{-N(f(E’)-f(E^*))/T}}. $$ In the thermodynamic limit, $N \rightarrow \infty$, $p(E)$ sharply peaks around $E^*$, since the numerator is zero unless $E=E^*$, and the denominator converges to $1$, since inside the sum, we always have $f(E’) - f(E^*) > 0$.
We can also write the partition function in a similar fashion, $$ Z = e^{-F(E^*)/T} \left(1 + \sum_{E’ \neq E^*} e^{-N(f(E’)-f(E^*))/T} \right). $$ Let’s define a mystery quantity $\tilde{f} = -\frac{T}{N} \ln Z$. Then we have $$ \begin{align} \tilde{f} & = f(E^*) - \frac{T}{N} \ln \left(1 + \sum_{E’ \neq E^*} e^{-N(f(E’)-f(E^*))/T} \right) \\ & \mathop{\rightarrow}_{N \rightarrow \infty} f(E^*). \end{align} $$ So, in the thermodynamic limit, we ‘select’ energies that minimise the free energy (density) $f(E)$. But we also see that $f(E^*)$ also tends to $\tilde{f}=-\frac{T}{N} \ln Z$ in the thermodynamic limit. This is why physicists refer to both $F(E)$ and $\tilde{F} := N \tilde{f}$ as the ‘free energy’. What’s even more confusing is that sometimes we even call anything proportional to $\ln Z$ as the ‘free energy’, and don’t even distinguish between free energy densities $f$ and free energies $F$.
Above, we defined the partition function as a sum over energies. But when we study statistical models, such as the Ising model and so on, we write it as a sum over configurations $\sigma$. In fact, they are one and the same. Notice that we can also write $Z = \sum_\sigma e^{-E(\sigma)/T} = \sum_E (\sum_{\sigma \vert E(\sigma) = E} 1) e^{-E/T} = \sum_E e^{S(E)} e^{-E/T}$.
Back to the Helmholtz free energy, which we can write as $F(E) = T (E/T- S(E))$. Hopefully with the previous discussion it is more clear why this quantity represents a ‘fight’ between entropy maximisation and energy minimisation.
The first term is proportional to $E/T$, which appeared when expanding the entropy of the bath. When the system given energy $E$, this is due to the bath losing the energy by that much. The more energy the bath has to give to the system, the less likely that it will occur. So, increasing the system’s energy decreases its probability, and hence increase the free energy. As $T \rightarrow 0$, the cost for energy becomes more and more expensive, and as $T \rightarrow \infty$, energy becomes free, so only the entropy matters.
The second term is entropy. Remember that the entropy $S(E)$ appeared in the partition function due to the number of configurations with energy $E$. It increases the weight of energies with many configurations and correspondingly reduces the free energy of such energies.
