Defining CSP’s

A CSP is characterized by the family of constraints that are allowed. These constraints are always local in the sense that they only depend on a finite number of coordinates in the input, and then apply one of a family of predicates to these coordinates.

One example of a CSP is when the family \(\cP\) is generated by applying one or more predicates to the literals in the input which are the variables and their negations in the \(\Sigma=\bits\) case (or their shifts in the larger alphabet case):

The basic computational problem associated with a CSP is to compute or approximate its value. For simplicity we will be focused on two kinds of decision problems:

• The exact computation problem for \(CSP_\Sigma(\cP)\) is the task of determining, given \(I\in CSP_\Sigma(\cP)\), whether \(\val(I)=1\) or \(\val(I)<1\).
• The \(c\) vs \(s\) approximate computation problem for \(CSP_\Sigma(\cP)\) is the task of distinguishing, given \(I\in CSP_\Sigma(\cP)\), between the case that \(\val(I)\geq c\) and the case that \(\val(I) \leq s\).

Examples:

Here are some classical examples of CSP’s

• \(CSP(\generated{x_1 \vee x_2 \vee \cdots \vee x_k})\) is the \(k-SAT\) problem where one is given a \(k\)-CNF (e.g., \((x_{17} \vee \overline{x}_9 \vee x_{52}) \wedge (\overline{x}_52 \vee x_5 \vee \overline{x}_89)\wedge (x_9 \vee \overline{x}_{22} \vee x_{89})\)) and wants to know if one can satisfy it or, barring that, how close can we get.
• \(CSP(\generated{x_1 \oplus \cdots \oplus x_k})\) is the \(k-XOR\) (also known as \(k\)LIN\((2)\) ) problem.
• If \(\Sigma=\{0,\ldots,k-1\}\) then \(CSP_\Sigma(\{ \neq \})\) is the \(k\) coloring problem. In particular for \(k=2\) this is the Maximum Cut problem
• \(CSP(\{ (x_1 \wedge \cdots \wedge x_{k-1}) \Rightarrow x_k \})\) is known as the HORN sat problem.
• For finite \(\Sigma\) and \(d\in \N\), a \(d\) to \(1\) projection constraint is a predicate \(P\from \Sigma'\times \Sigma \to \bits\) where \(|\Sigma'|=d|\Sigma|\) and where for every \(y\in \Sigma\) there exist exactly \(d\) \(x\)’s in \(\Sigma'\) such that \(P(x,y)=1\). Clearly we can also think of \(P\) as a predicate taking two inputs from the larger alphabet \(\Sigma'\).We can also characterize a \(d\) to \(1\) game as a \(k\) ary predicate \(P\from\Sigma^k\to\bits\) such that \(\Sigma'=P^{-1}(1)\) satisfies that for every \(i\in [k]\) and \(\sigma\in\Sigma\), \(|\{ x\in \Sigma \;|\; x_i = \sigma \}|=d\). The \(d\) to \(1\) problem with alphabet \(\Sigma\) is the CSP corresponding to the set of all \(d\) to \(1\) projection constraints. The \(1\) to \(1\) problem is also known as the unique games problem. The union of the \(d\) to \(1\) problem over all \(d\) is known as the label cover problem.

Classes of CSP’s

At a rough and imprecise level, our intuition from the study of CSP’s is that there are three “canonical prototypes” of CSP problems:

• Linear predicates: These are CSP’s such as \(k\)-XOR where the class of predicates admits some linear or other algebraic structure to allows a Gaussian elimination type algorithm to solve their exact version. The general notion is quite subtle and involves the notion of a polymorphism of the predicate. (Indeed, the technical content of the algebraic dichotomy conjecture is that every predicate admitting such a polymorphism can be solved by a Gaussian elimination type algorithm.)
• Propagation predicates: These are CSP’s such as 2-SAT, Horn-SAT, and Max-Cut where “guessing” few coordinates of the assignment allows us to propagate values for the other coordinates. In particular there is a simple linear time algorithm to solve the exact satisfiability problem of such CSP’s.
• Hard predicates: These are CSP’s such as \(3\)-SAT or \(3\)-COL (as well as their versions with larger arity) where neither an algebraic nor propagation structure is present.

The algebraic dichotomy conjecture can be phrased as the conjecture that the exact decision problem for linear predicates can be solved with a \(poly(n)\) (in fact with exponent at most \(3\)) time algorithm, propagation predicates can be solved with a linear time algorithm, while the exact decision problem for all other predicates is NP hard. In fact, the reduction showing NP hardness for the latter case only has a linear blowup from the 3SAT problem and so, under the exponential time hypothesis, it rules out \(2^{o(n)}\) time algorithms for the exact decision problem for these predicates. The latter two parts (propagation based algorithm and NP hardness reduction) of the dichotomy conjecture have already been proven, and it is the first part that is yet open. To some extent this is not surprising: the Gaussian elimination algorithm is a canonical example of an algorithm using non trivial algebraic structure (and in particular, one that is not captured by the sos algorithm) and understanding its power is a subtle question.

Approximating CSP’s

Considering the notion of approximation of CSP’s makes understanding their complexity harder in some respects and easier in others. Let us focus on the \(1-\epsilon\) vs \(f(\epsilon)\) approximation problem for small \(\epsilon>0\). One aspect in which approximation makes classification easier is that the Gaussian elimination type algorithm are extremely brittle and so the linear predicates actually become hard under noise. This is epitomized by the following theorem of Håstad (2001):

Moshkovitz and Raz (2010) improved this to give a \(\tilde{O}(n)\) blowup reduction, as well as showing that \(\epsilon\) can tend to zero as fast \(1/polylog(n)\). As far as we know, the fastest algorithm in this problem might require time \(\exp(poly(\epsilon) n)\).

We note that the “brittleness” of Gaussian elimination under noise is also crucially used in cryptography where Lattice based cryptosystems all require for their security the conjecture that solving noisy linear equations is hard, c.f. (Regev 2005) (and in fact for levels of noise that are below those where we can expect to get NP hardness, c.f. (Aharonov and Regev 2004)).

In contrast, the propagation predicates do have non trivial approximation of the form \(1-\epsilon\) vs \(1-f(\epsilon)\) for \(f\) that tends to \(0\) as \(\epsilon\rightarrow 0\). However, understanding the shape of this function \(f(\cdot)\) is an open question, and is to a large extent the content of the Unique Games conjecture. For example, for the Max-Cut problem, the Goemans and Williamson (1995) algorithm yields a \(1-\epsilon\) vs \(1-C\sqrt{\epsilon}\) approximation (for some absolute constant \(C\)) while the best known NP hardness by Trevisan et al. (2000) (which uses a gadget that was found via a computer search) rules out a \(1-\epsilon\) vs \(1-C'\epsilon\) approximation for some absolute \(C'\). Khot et al. (2004) and Mossel, O’Donnell, and Oleszkiewicz (2010) showed that if the Unique Games Conjecture is true, then there is no polynomial-time algorithm beating the guarantee of the Goemans and Williamson (1995) algorithm.

The UGC itself can be phrased as follows:

Unique Games Conjecture: For every \(\epsilon>0\) and \(\delta>0\) there is some \(\Sigma\) such that the \(1-\epsilon\) vs \(\delta\) problem for unique games on alphabet \(\Sigma\) is NP hard.

While the UGC itself is a constraint satisfaction problem, it has been shown to imply interesting results for other problems as well. Nonetheless, CSP’s provide a very useful lens under which to examine the results and open questions surrounding the UGC.

Illustration of the conjectural complexity of CSP’s

If we assume the exponential time hypothesis (Impagliazzo, Paturi, and Zane 1998) and the algebraic dychotomy conjecture (Andrei A. Bulatov, Jeavons, and Krokhin 2005) (both of which are widely believed, especially if we do not distinguish between \(\exp(\Omega(n))\) and \(\exp(n^{1-o(1)})\) running time) then for every exact constraint satisfaction problem can either be solved in \(poly(n)\) (in fact \(O(n^3)\)) time or requires \(\exp(\Omega(n))\) time. The unique games conjecture, coupled with the sub-exponential time algorithm Arora, Barak, and Steurer (2015) shows that the picture for approximation problems is very different. For some CSP’s, such as the “hard” or “linear” CSP’s that contain a pairwise independent distribution, there is a threshold effect similar to the case of exact solution and (up to some measure zero sets) for every two numbers \(0\leq s<c \leq 1\) the task of coming up can either be solved in \(poly(n)\) and or requires \(\exp(n^{1-o(1)})\) time. For others, such as the unique games problem itself (and likely others, such as max cut) there is a region of parameters where the problem can be solved in subexponential time.

⊕ A rough (and not to scale) sketch of the known and/or very likely time complexity of finding an assignment satisfying \(s\) fraction of the constraints given a non propagation instance (i.e., containing a pairwise independent distribution) “linear” or “hard” type CSP with optimal value \(c\) for some \(c<1\). This relies on the results of Håstad (2001), Moshkovitz and Raz (2010), Chan (2016), Barak, Chan, and Kothari (2015)

⊕ A rough (and not to scale) sketch of the known and conjectured time complexity of finding an assignment satisfying \(s\) fraction of the constraints given an instance of a “propagation” type CSP with optimal value \(c\) for some \(c<1\). This relies on the results of Håstad (2001), Raghavendra (2008), Arora, Barak, and Steurer (2015), Moshkovitz and Raz (2010)

⊕ Conjectured complexity curve for 3SAT.

⊕ Conjectured complexity curve for 3XOR

⊕ Conjectured complexity curve for Max Cut assuming the UGC and a generalization of the subexponential algorithm. The curve for exponential hardness is a rough approximation, the curve for UGC hardness is taken from O’Donnell and Wu (2008)

Algorithmic results known about the unique games problem.

We denote the unique games problem on alphabet \(\Sigma\) as \(\Sigma\)-UG for short. The following algorithmic results are known about it:

• The basic SDP algorithm (which is a generalization of the Goemans and Williamson (1995) max cut algorithm) can solve the \(1-\epsilon\) vs \(1-O(\sqrt{\epsilon \log |\Sigma|})\) algorithm for the \(\Sigma\)-UG problem, as well as the \(1-\epsilon\) vs \(\Omega(|\Sigma|^{-\epsilon/(2-\epsilon))}\) version of this problem (Charikar, Makarychev, and Makarychev 2006). In particular this means that the \(\Sigma\)-UG problem is not approximation resistant in the sense that there is an algorithm for the \(1-o(1)\) vs \(\tfrac{1}{|\Sigma|}+o(1)\) variant of this problem. It also means that the order of quantifiers in the UGC formulation is crucial, and (unlike the case of 3XOR) we cannot have hardness if the completeness condition holds with parameter \(1-\epsilon\) for \(\epsilon\) tending to zero independently of the alphabet size.
• When \(|\Sigma| \gg 1/\epsilon\) (in particular \(|\Sigma| = 2^{\Omega(1/\epsilon)}\)) then no polynomial time algorithm is known. However, Arora, Barak, and Steurer (2015) showed a \(\exp(\poly(|\Sigma|)n^{\poly(\epsilon)})\) time algorithm for the \(1-\epsilon\) vs \(1/2\) \(\Sigma\)-UG problem.

The approximation guarantees of the Basic SDP algorithm are known to be optimal for \(\Sigma\)-UG if the unique games conjecture is true (Charikar, Makarychev, and Makarychev 2006).

The relation of UGC and the sum of squares algorithm

The UGC and sum of squares algorithm seem to have a sort of “love hate relationship”. For starters, one of the most striking predictions of the UGC is the result of Raghavendra (2008) that if the UGC is true than sos is optimal for all constraint satisfaction problems:

In fact Raghavendra (2008) showed that this holds for a restricted class of pseudo-distributions where the most interesting constraint (namely that \(\pE_\mu p^2 \geq 0\) for all degree \(d/2\) polynomials) needs to hold only for \(d=2\). (These corresponds to solutions of what is known as the “Basic SDP program” for \(\cC\).) Thus the UGC is a “friend” of the sos algorithm in the sense that it shows that it is optimal for a broad class of optimization problems. (Even if it shows that for this class, we don’t really need to use some of the more interesting constraints of the sos algorithm.)

However, the sos algorithm hasn’t always been a friend of the UGC. Arguably the best evidence for the UGC comes from candidate hard instances, which are instances of the unique games problem (or other CSP’s or computational problems) for which the predictions of the UGC hold true for some natural algorithms. That is, the performance of these algorithm is not better than the performance that the UGC predicts is the best possible for polynomial time algorithms. However, the sos algorithm has been used to give some results that call the UGC into questions:

• For the Unique Games problem itself, the sos algorithm gives an \(\exp(n^{poly(\epsilon)})\) time algorithm for the \(1-\epsilon\) vs \(1/2\) unique games problem which is predicted to be NP hard via the UGC. This means that even if the UGC is true, reductions based on it cannot rule out, say, a \(\exp(\sqrt{n})\) time algorithm for any problem. This can also be extended to a \(\exp(n^{\poly(\epsilon)})\) time algorithm for the \(c\) vs \(\epsilon\) \(d\)-to-\(1\) games problem for every absolute constans \(c>0,d\in\N\) that are independent of \(\epsilon\).
• The known hard instances that match the UGC’s predictions are integrality gaps for the Basic SDP program that are CSP instances where for some \(1 \geq c > s \geq 0\), the SDP value is at least \(c\) but it can be proven using various isoperimetric, invariance and concentration results that the true value is at most \(s\). It turns out that the proofs of (sufficiently close variants of) these results can be captured in the constant degree sos proof system, and so there is some constant \(d\) such that the degree \(d\) sos value for these instances is also at most \(s\).
• The unique games conjecture is closely related to a conjecture known as the small set expansion hypothesis (the relation is roughly related to the relation between the Max Cut and Sparsest Cut problems). This latter conjecture is in turn related to the task of finding an (approximately) sparse vector inside a linear subspace, where the sos algorithm does provide non trivial average-case guarantees. It is also related to the Best Separable State problem in quantum information theory, for which the sos algorithm provides non trivial worst case guarantees, though in incomparable parameter regimes. Thus there is a plausible path to refuting the small set expansion hypothesis (and possibly the UGC itself) via sos-based algorithm for finding sparse vectors in a subspace.

Nevertheless, at the moment the status of the UGC is wide open, and indeed some of the efforts to prove the UGC (Khot and Moshkovitz 2016),(Khot, Minzer, and Safra 2016) have also focused on coming up with integrality gaps for the sos program and/or using tools such as the short code (Barak et al. 2012) that have been developed in this context. So, regardless of the final outcome, the sos algorithm and the unique games conjecture will shed a lot of light on one another.