Cheeger’s inequality

Let \(G\) be a \(d\)-regular graph with vertex set \(V=[n]\). For a vertex subset \(S\subseteq V\), we define its expansion \(\varphi_G(S)\) as: \[ \varphi_G(S) = \frac{\bigabs{E(S,V\setminus S)}}{\tfrac d n\cdot\bigabs{S}\cdot \bigabs{V \setminus S}}\,. \label{eq:expansion} \] Another way to say it is that the expansion of a set \(S\) is the number of edges between \(S\) and its complement in \(G\) as a fraction of the expected number of edges in a random graph with average degree \(d\).Up to a constant factor this is the same as the probability that we leave the set \(S\) if we start at a random vertex in \(S\) and go to one of its \(d\) neighbors at random. Can you see why?.

It is not difficult to check that the expansion of any set \(S\) is a number between \(0\) and \(2\).For example, in a bipartite \(d\)-regular graph each side of the bipartition has expansion \(2\). Most sets in a graph have expansion close to \(1\).Concretely, the expected expansion of a random vertex subset is close to \(1\). Therefore, an interesting question about a graph is whether it contains exceptional sets with expansion close to \(0\) or whether all sets have expansion bounded away from \(0\).

The expansion of graph \(G\), denoted \(\varphi(G)\), is the minimum expansion \(\varphi_G(S)\) over all sets \(S\subseteq[n]\).The literature on graph expansion defines several closely related quantities such as sparsest cut, expansion, and conductance that are all equivalent up to constant factors. We do not distinguish between these notions here. The problem of computational the expansion of a graph (and finding the corresponding set) is a fundamental graph problem, with a wide variety of applications to network design, analyzing Markov chains, and more. It is also widely used as a tool in many “divide and conquer” algorithms.

For every \(\epsilon>0\), in a random regular graph of sufficiently large degree, \(\varphi_G(S)\) will be at least \(1-\epsilon\). On the other hand, if we “plant” a non-expanding set in a random graph by selecting a set \(S\) of half the vertices and conditioning the random edges touching \(S\) to stay inside it with probability \(1-\epsilon\), it might not be a priori clear how one can detect this set. For this reason, like in the max cut case, it is not a priori clear how one can certify that a highly expanding graph (such as a random \(d\)-regular graph) has expansion \(\varphi(G)\) smaller than \(1\) nor is it clear how to find any set with \(\varphi_G(S) \ll 1\), even if \(\varphi(G) = o(1)\). Nevertheless, like in the case of max cut, it turns out that one can in fact beat the “combinatorial” (or linear-programming based) algorithms.

Bounding rational functions using sum of squares

A priori it might not be clear how to apply the sum-of-squares algorithm to Min Expansion. So far we have talked about the problem of minimizing polynomials over the hypercube, but the expansion of a set \(S\) is a rational function of the characteristic vector of \(S\). In particular, if we let \(f_G(x)=\sum_{\set{i,j}\in E(G)}(x_i-x_j)^2\) and \(\abs{x}=\sum_{i=1}^n x_i\), then \[ \varphi(G) = \min_{x\in \bits^n} \frac{f_G(x)}{\tfrac dn \cdot \abs{x}\cdot (n-\abs{x})} \] The following observation allows us to apply sum-of-squares also for minimizing rational functions: in order to certify that for every \(x\in\bits^n\), a rational function of the form \(P(x)/Q(x)\) is at least \(\e>0\), all we need to do is to show that the polynomial \(P - \e \cdot Q\) is always non-negative.

The following theorem, known as the discrete Cheeger’s Inequality (obtained by Dodziuk (1984), and independently by Alon and Milman (1985) and Alon (1986) as a discrete version of (Cheeger 1970)), shows that degree-2 sum-of-squares does provide such a certificate, in particular, showing that we can efficiently certify that \(\varphi(G)\ge 0.001\) for every graph that satisfies \(\varphi(G)\ge 0.1\).

The proof of the above theorem also shows that there is a polynomial-time algorithm to find \(S\) with \(\varphi_G(S) = O(\sqrt{\varphi(G)})\). Leighton and Rao (1988) gave a polynomial-time algorithm based on linear programming to find \(S\) with \(\varphi_G(S) = O(\log n )\varphi(G)\), that is, the algorithm achieves approximation ratio \(O(\log n)\).It is instructive to verify that the approximation guarantees of the algorithms based on degree-2 sum-of-squares and linear programming are incomparable. For small values of \(\varphi(G)\), the linear programming approach has stronger guarantees. For larger values of \(\varphi(G)\) (say \(\varphi(G)\ge 1/\log n\)) the guarantees of degree-2 sum-of-squares are stronger. In a breakthrough work, Arora, Rao, and Vazirani (2004) improved this approximation ratio \(O(\sqrt{\log n})\). Their algorithm uses the degree-\(4\) SOS algorithm, and we will see it later in this course. Shortly thereafter, Agarwal et al. (2005) gave the analogous result for Max Cut, namely an algorithm that given \(G\) with \({\mathrm{maxcut}}(G)=1-\e\), outputs \(S\) with \(\varphi_G(S) \geq 1 - O(\sqrt{\log n})\e\).

Rounding pseudo-distributions for Min Expansion

We now show how Reference:degree-2-sos-certificates-for-expansion is implied by the standard formulation of the discrete Cheeger’s inequality.

The proof, which we omit here, is not extremely complicated and can be found in several sources (e.g., see handouts 3 and 4 in Luca Trevisan’s course).

For every vector \(x\in \R^n\), \[ \iprod{x,L_G x}=\sum_{i=1}^n x_i^2 - \tfrac 2 d \sum_{\set{i,j}\in E(G)} x_i x_j = \tfrac{1}{d}f_G(x)\,. \] Moreover, the minimum eigenvector of \(L_G\) is always the all ones vector \(\Ind\). If the second smallest eigenvector of \(L_G\) is \(\lambda\) then for every vector \(x\in\R^n\), its projection \(y = x - \tfrac{1}{n}\iprod{\Ind,x}\Ind\) into the subspace orthogonal to \(\Ind\) satisfies \(f_G(y) \geq \lambda \cdot \norm{y}^2\).

These above facts together with the observation that \(|x| = \sum_{i=1}^n x_i^2\) over \(\bits^n\) are enough to derive Reference:degree-2-sos-certificates-for-expansion from Reference:thm-cheeger. We leave the details as an exercise.

The following exercises asks you to prove the corresponding statement about pseudo-distributions.

It turns out the proof of Cheeger’s inequality is constructive, and this can be used to show an efficient rounding algorithm that takes any pseudo-distribution satisfying \(\pE n f_G \leq \epsilon \pE d|x|(n-|x|)\) and obtains from it an actual set \(S\) with \(\varphi_G(S) \leq O(\sqrt{\epsilon})\).