### 3.2 Coxeter groups – The general theory

We have just shown that the billiard group in the case of pure gravity in four spacetime dimensions is the group . This group is generated by reflections and is a particular example of a Coxeter group. Furthermore, as we shall explain below, this Coxeter group turns out to be the Weyl group of the (hyperbolic) Kac–Moody algebra . Our first encounter with Lorentzian Kac–Moody algebras in more general gravitational theories will also be through their Weyl groups, which are, exactly as in the four-dimensional case just described, particular instances of (non-Euclidean) Coxeter groups, and which arise as the groups of billiard reflections.

For this reason, we start by developing here some aspects of the theory of Coxeter groups. An excellent reference on the subject is [107], to which we refer for more details and information. We consider Kac–Moody algebras in Section 4.

#### 3.2.1 Examples

Coxeter groups generalize the familiar notion of reflection groups in Euclidean space. Before we present the basic definition, let us briefly discuss some more illuminating examples.

#### The dihedral group

Consider the dihedral group of order 6 of symmetries of the equilateral triangle in the Euclidean plane.

This group contains the identity, three reflections , and about the three medians, the rotation of about the origin and the rotation of about the origin (see Figure 3),

The reflections act as follows,
where is here the Euclidean scalar product and where is a vector orthogonal to the hyperplane (here, line) of reflection.

Now, all elements of the dihedral group can be written as products of the two reflections and :

Hence, the dihedral group is generated by and . The writing Equation (3.21) is not unique because and are subject to the following relations,
The first two relations merely follow from the fact that and are reflections, while the third relation is a consequence of the property that the product is a rotation by an angle of . This follows, in turn, from the fact that the hyperplanes (lines) of reflection make an angle of . There is no other relation between the generators and because any product of them can be reduced, using the relations Equation (3.22), to one of the 6 elements in Equation (3.21), and these are independent.

The dihedral group is also denoted because it is the Weyl group of the simple Lie algebra (see Section 4). It is isomorphic to the permutation group of three objects.

#### The infinite dihedral group

Consider now the group of isometries of the Euclidean line containing the symmetries about the points with integer or half-integer values of ( is a coordinate along the line) as well as the translations by an integer. This is clearly an infinite group. It is generated by the two reflections about the origin and about the point with coordinate ,

The product is a translation by while the product is a translation by , so no power of or gives the identity. All the powers and are distinct (translations by and , respectively). The only relations between the generators are
This infinite dihedral group is also denoted by because it is the Weyl group of the affine Kac–Moody algebra .

#### 3.2.2 Definition

A Coxeter group is a group generated by a finite number of elements () subject to relations that take the form

and
where the integers associated with the pairs fulfill
Note that Equation (3.25) is a particular case of Equation (3.26) with . If there is no power of that gives the identity, as in our second example, we set, by convention, . The generators are called “reflections” because of Equation (3.25), even though we have not developed yet a geometric realisation of the group. This will be done in Section 3.2.4 below.

The number of generators is called the rank of the Coxeter group. The Coxeter group is completely specified by the integers . It is useful to draw the set pictorially in a diagram , called a Coxeter graph. With each reflection , one associates a node. Thus there are nodes in the diagram. If , one draws a line between the node and the node and writes over the line, except if is equal to 3, in which case one writes nothing. The default value is thus “3”. When there is no line between and (), the exponent is equal to 2. We have drawn the Coxeter graphs for the Coxeter groups , and for the Coxeter group of symmetries of the icosahedron.

Note that if , the generators and commute, . Thus, a Coxeter group is the direct product of the Coxeter subgroups associated with the connected components of its Coxeter graph. For that reason, we can restrict the analysis to Coxeter groups associated with connected (also called irreducible) Coxeter graphs.

The Coxeter group may be finite or infinite as the previous examples show.

#### Another example:

It should be stressed that the Coxeter group can be infinite even if none of the Coxeter exponent is infinite. Consider for instance the group of isometries of the Euclidean plane generated by reflections in the following three straight lines: (i) the -axis (), (ii) the straight line joining the points and (), and (iii) the -axis (). The Coxeter exponents are finite and equal to () and (). The Coxeter graph is given in Figure 7. The Coxeter group is the symmetry group of the regular paving of the plane by squares and contains translations. Indeed, the product is a reflection in the line parallel to the -axis going through and thus the product is a translation by in the -direction. All powers of are distinct; the group is infinite. This Coxeter group is of affine type and is called (which coincides with ).

#### The isomorphism problem

The Coxeter presentation of a given Coxeter group may not be unique. Consider for instance the group of order 12 of symmetries of the regular hexagon, generated by two reflections and with

This group is isomorphic with the rank 3 (reducible) Coxeter group , with presentation

the isomorphism being given by , , . The question of determining all such isomorphisms between Coxeter groups is known as the “isomorphism problem of Coxeter groups”. This is a difficult problem whose general solution is not yet known [10].

#### 3.2.3 The length function

An important concept in the theory of Coxeter groups is that of the length of an element. The length of is by definition the number of generators that appear in a minimal representation of as a product of generators. Thus, if and if there is no way to write as a product of less than generators, one says that has length .

For instance, for the dihedral group , the identity has length zero, the generators and have length one, the two non-trivial rotations have length two, and the third reflection has length three. Note that the rotations have representations involving two and four (and even a higher number of) generators since for instance , but the length is associated with the representations involving as few generators as possible. There might be more than one such representation as it occurs for . Both involve three generators and define the length of to be three.

Let be an element of length . The length of (where is one of the generators) differs from the length of by an odd (positive or negative) integer since the relations among the generators always involve an even number of reflections. In fact, is equal to or since and . Thus, in , there can be at most one simplification (i.e., at most two elements that can be removed using the relations).

#### 3.2.4 Geometric realization

We now construct a geometric realisation for any given Coxeter group. This enables one to view the Coxeter group as a group of linear transformations acting in a vector space of dimension , equipped with a scalar product preserved by the group.

To each generator , associate a vector of a basis of an -dimensional vector space . Introduce a scalar product defined as follows,

on the basis vectors and extend it to by linearity. Note that for , implies for all . In the case of the dihedral group , this scalar product is just the Euclidean scalar product in the two-dimensional plane where the equilateral triangle lies, as can be seen by taking the two vectors and respectively orthogonal to the first and second lines of reflection in Figure 3 and oriented as indicated. But in general, the scalar product (3.28) might not be of Euclidean signature and might even be degenerate. This is the case for the infinite dihedral group , for which the matrix reads
and has zero determinant. We shall occasionally use matrix notations for the scalar product, .

However, the basis vectors are always all spacelike since they have norm squared equal to . For each , the vector space splits then as a direct sum

where is the hyperplane orthogonal to ( iff ). One defines the geometric reflection as
It is clear that fixes pointwise and reverses . It is also clear that and that preserves ,
Note that in the particular case of , we recover in this way the reflections and .

We now verify that the ’s also fulfill the relations . To that end we consider the plane spanned by and . This plane is left invariant under and . Two possibilities may occur:

1. The induced scalar product on is nondegenerate and in fact positive definite, or
2. the induced scalar product is positive semi-definite, i.e., there is a null direction orthogonal to any other direction.

The second case occurs only when . The null direction is given by .

• In Case 1, splits as and is clearly the identity on since both and leave pointwise invariant. One needs only to investigate on , where the metric is positive definite. To that end we note that the reflections and are, on , standard Euclidean reflections in the lines orthogonal to and , respectively. These lines make an angle of and hence the product is a rotation by an angle of . It follows that also on .
• In Case 2, is infinite and we must show that no power of the product gives the identity. This is done by exhibiting a vector for which for all integers different from zero. Take for instance . Since one has and , it follows that unless .

As the defining relations are preserved, we can conclude that the map from the Coxeter group generated by the ’s to the geometric group generated by the ’s defined on the generators by is a group homomorphism. We will show below that its kernel is the identity so that it is in fact an isomorphism.

Finally, we note that if the Coxeter graph is irreducible, as we assume, then the matrix is indecomposable. A matrix is called decomposable if after reordering of its indices, it decomposes as a non-trivial direct sum, i.e., if one can slit the indices in two sets and such that whenever or . The indecomposability of follows from the fact that if it were decomposable, the corresponding Coxeter graph would be disconnected as no line would join a point in the set to a point in the set .

#### 3.2.5 Positive and negative roots

A root is any vector in the space of the geometric realisation that can be obtained from one of the basis vectors by acting with an element of the Coxeter group (more precisely, with its image under the above homomorphism, but we shall drop “” for notational simplicity). Any root can be expanded in terms of the ’s,

If the coefficients are all non-negative, we say that the root is positive and we write . If the coefficients are all non-positive, we say that the root is negative and we write . Note that we use strict inequalities here because if for all , then is not a root. In particular, the ’s themselves are positive roots, called also “simple” roots. (Note that the simple roots considered here differ by normalization factors from the simple roots of Kac–Moody algebras, as we shall discuss below.) We claim that roots are either positive or negative (there is no root with some ’s in Equation (3.33) and some other ’s ). The claim follows from the fact that the image of a simple root by an arbitrary element of the Coxeter group is necessarily either positive or negative.

This, in turn, is the result of the following theorem, which provides a useful criterion to tell whether the length of is equal to or .

Theorem: if and only if .

The proof is given in [107], page 111.

It easily follows from this theorem that if and only if . Indeed, is equivalent to , i.e., and thus, by the theorem, . But since , this is equivalent to .

We have seen in Section 3.2.3 that there are only two possibilities for the length . It is either equal to or to . From the theorem just seen, the root is positive in the first case and negative in the second. Since any root is the Coxeter image of one of the simple roots , i.e., can be written as for some and , we can conclude that the roots are either positive or negative; there is no alternative.

The theorem can be used to provide a geometric interpretation of the length function. One can show [107] that is equal to the number of positive roots sent by to negative roots. In particular, the fundamental reflection associated with the simple root maps to its negative and permutes the remaining positive roots.

Note that the theorem implies also that the kernel of the homomorphism that appears in the geometric realisation of the Coxeter group is trivial. Indeed, assume where is an element of the Coxeter group that is not the identity. It is clear that there exists one group generator such that . Take for instance the last generator occurring in a reduced expression of . For this generator, one has , which is in contradiction with the assumption .

Because is an isomorphism, we shall from now on identify the Coxeter group with its geometric realisation and make no distinction between and .

#### 3.2.6 Fundamental domain

In order to describe the action of the Coxeter group, it is useful to introduce the concept of fundamental domain. Consider first the case of the symmetry group of the equilateral triangle. The shaded region in Figure 4 contains the vectors such that and . It has the following important property: Any orbit of the group intersects once and only once. It is called for this reason a “fundamental domain”. We shall extend this concept to all Coxeter groups. However, when the scalar product is not positive definite, there are inequivalent types of vectors and the concept of fundamental domain can be generalized a priori in different ways, depending on which region one wants to cover. (The entire space? Only the timelike vectors? Another region?) The useful generalization turns out not to lead to a fundamental domain of the action of the Coxeter group on the entire vector space , but rather to a fundamental domain of the action of the Coxeter group on the so-called Tits cone , which is such that the inequalities continue to play the central role.

We assume that the scalar product is nondegenerate. Define for each simple root the open half-space

We define to be the intersection of all ,
This is a convex open cone, which is non-empty because the metric is nondegenerate. Indeed, as is nondegenerate, one can, by a change of basis, assume for simplicity that the bounding hyperplanes are the coordinate hyperplanes . is then the region (with appropriate orientation of the coordinates) and is . The closure
is then a closed convex cone.

We next consider the union of the images of under the Coxeter group,

One can show [107] that this is also a convex cone, called the Tits cone. Furthermore, is a fundamental domain for the action of the Coxeter group on the Tits cone; the orbit of any point in intersects once and only once [107]. The Tits cone does not coincide in general with the full space and is discussed below in particular cases.