Science is rarely tidy. We ultimately seek a unified explanatory framework characterized by elegance and simplicity; along the way, however, our aesthetic impulses must occasionally be sacrificed to the desire to encompass the largest possible range of phenomena (i.e., to fit the data). It is often the case that an otherwise compelling theory, in order to be brought into agreement with observation, requires some apparently unnatural modification. Some such modifications may eventually be discarded as unnecessary once the phenomena are better understood; at other times, advances in our theoretical understanding will reveal that a certain theoretical compromise is only superficially distasteful, when in fact it arises as the consequence of a beautiful underlying structure.
General relativity is a paradigmatic example of a scientific theory of impressive power and simplicity. The cosmological constant, meanwhile, is a paradigmatic example of a modification, originally introduced  to help fit the data, which appears at least on the surface to be superfluous and unattractive. Its original role, to allow static homogeneous solutions to Einstein’s equations in the presence of matter, turned out to be unnecessary when the expansion of the universe was discovered , and there have been a number of subsequent episodes in which a nonzero cosmological constant was put forward as an explanation for a set of observations and later withdrawn when the observational case evaporated. Meanwhile, particle theorists have realized that the cosmological constant can be interpreted as a measure of the energy density of the vacuum. This energy density is the sum of a number of apparently unrelated contributions, each of magnitude much larger than the upper limits on the cosmological constant today; the question of why the observed vacuum energy is so small in comparison to the scales of particle physics has become a celebrated puzzle, although it is usually thought to be easier to imagine an unknown mechanism which would set it precisely to zero than one which would suppress it by just the right amount to yield an observationally accessible cosmological constant.
This checkered history has led to a certain reluctance to consider further invocations of a nonzero cosmological constant; however, recent years have provided the best evidence yet that this elusive quantity does play an important dynamical role in the universe. This possibility, although still far from a certainty, makes it worthwhile to review the physics and astrophysics of the cosmological constant (and its modern equivalent, the energy of the vacuum).
There are a number of other reviews of various aspects of the cosmological constant; in the present article I will outline the most relevant issues, but not try to be completely comprehensive, focusing instead on providing a pedagogical introduction and explaining recent advances. For astrophysical aspects, I did not try to duplicate much of the material in Carroll, Press and Turner , which should be consulted for numerous useful formulae and a discussion of several kinds of observational tests not covered here. Some earlier discussions include [85, 50, 221], and subsequent reviews include [58, 218, 246]. The classic discussion of the physics of the cosmological constant is by Weinberg , with more recent work discussed by [58, 218]. For introductions to cosmology, see [149, 160, 189].
Einstein’s original field equations are:
The energy-momentum sources may be modeled as a perfect fluid, specified by an energy density and isotropic pressure in its rest frame. The energy-momentum tensor of such a fluid is
Einstein was interested in finding static () solutions, both due to his hope that general relativity would embody Mach’s principle that matter determines inertia, and simply to account for the astronomical data as they were understood at the time. (This account gives short shrift to the details of what actually happened; for historical background see .) A static universe with a positive energy density is compatible with (5) if the spatial curvature is positive () and the density is appropriately tuned; however, (6) implies that will never vanish in such a spacetime if the pressure is also nonnegative (which is true for most forms of matter, and certainly for ordinary sources such as stars and gas). Einstein therefore proposed a modification of his equations, to
The discovery by Hubble that the universe is expanding eliminated the empirical need for a static world model (although the Einstein static universe continues to thrive in the toolboxes of theorists, as a crucial step in the construction of conformal diagrams). It has also been criticized on the grounds that any small deviation from a perfect balance between the terms in (9) will rapidly grow into a runaway departure from the static solution.
Pandora’s box, however, is not so easily closed. The disappearance of the original motivation for introducing the cosmological constant did not change its status as a legitimate addition to the gravitational field equations, or as a parameter to be constrained by observation. The only way to purge from cosmological discourse would be to measure all of the other terms in (8) to sufficient precision to be able to conclude that the term is negligibly small in comparison, a feat which has to date been out of reach. As discussed below, there is better reason than ever before to believe that is actually nonzero, and Einstein may not have blundered after all.
The cosmological constant is a dimensionful parameter with units of (length)–2. From the point of view of classical general relativity, there is no preferred choice for what the length scale defined by might be. Particle physics, however, brings a different perspective to the question. The cosmological constant turns out to be a measure of the energy density of the vacuum – the state of lowest energy – and although we cannot calculate the vacuum energy with any confidence, this identification allows us to consider the scales of various contributions to the cosmological constant [277, 33].
Consider a single scalar field , with potential energy . The action can be written
It is not necessary to introduce scalar fields to obtain a nonzero vacuum energy. The action for general relativity in the presence of a “bare” cosmological constant is
Classically, then, the effective cosmological constant is the sum of a bare term and the potential energy , where the latter may change with time as the universe passes through different phases. Quantum mechanics adds another contribution, from the zero-point energies associated with vacuum fluctuations. Consider a simple harmonic oscillator, i.e. a particle moving in a one-dimensional potential of the form . Classically, the “vacuum” for this system is the state in which the particle is motionless and at the minimum of the potential (), for which the energy in this case vanishes. Quantum-mechanically, however, the uncertainty principle forbids us from isolating the particle both in position and momentum, and we find that the lowest energy state has an energy (where I have temporarily re-introduced explicit factors of for clarity). Of course, in the absence of gravity either system actually has a vacuum energy which is completely arbitrary; we could add any constant to the potential (including, for example, ) without changing the theory. It is important, however, that the zero-point energy depends on the system, in this case on the frequency .
A precisely analogous situation holds in field theory. A (free) quantum field can be thought of as a collection of an infinite number of harmonic oscillators in momentum space. Formally, the zero-point energy of such an infinite collection will be infinite. (See [264, 48] for further details.) If, however, we discard the very high-momentum modes on the grounds that we trust our theory only up to a certain ultraviolet momentum cutoff , we find that the resulting energy density is of the form.)
The net cosmological constant, from this point of view, is the sum of a number of apparently disparate contributions, including potential energies from scalar fields and zero-point fluctuations of each field theory degree of freedom, as well as a bare cosmological constant . Unlike the last of these, in the first two cases we can at least make educated guesses at the magnitudes. In the Weinberg-Salam electroweak model, the phases of broken and unbroken symmetry are distinguished by a potential energy difference of approximately (where ); the universe is in the broken-symmetry phase during our current low-temperature epoch, and is believed to have been in the symmetric phase at sufficiently high temperatures at early times. The effective cosmological constant is therefore different in the two epochs; absent some form of prearrangement, we would naturally expect a contribution to the vacuum energy today of order
As we will discuss later, cosmological observations imply
This work is licensed under a Creative Commons License.