Notation
Hiding constants
Unless explicitly stated otherwise, \(O(\cdot)\)-notation hides absolute multiplicative constants. Concretely, every occurrence of \(O(x)\) is a placeholder for some function \(f(x)\) that satisfies \(\forall x\in \R.\, \abs{f(x)}\le C\abs{x}\) for some absolute constant \(C>0\). Similarly, \(\Omega(x)\) is a placeholder for a function \(g(x)\) that satisfies \(\forall x\in \R.\, \abs{g(x)} \ge \abs{x}/C\) for some absolute constant \(C>0\).
Vectors
All vectors are column vectors unless specified otherwise. In particular, the notation \((a,b,c)\) is short hand for a column vector with entries \(a,b,c\in \R\),. We denote the coordinate basis of \(\R^n\) by \(\set{e_i}_{i\in [n]}\). For a vector \(v\in\R^n\), we let \(\transpose v\) be the corresponding row vector.
Inner products and norms
For vectors \(u,v\in \R^n\) with \(u=(u_1,\ldots,u_n)\) and \(v=(v_1,\ldots,v_n)\), we define the inner product of \(u\) and \(v\), unless specified otherwise, \[ \iprod{u,v}=\transpose u v=\sum_{i=1}^n u_i \cdot v_i\,. \] The (Euclidean) norm of a vector \(v\) is \(\norm{v}=\iprod{v,v}^{1/2}\). For \(p\ge 1\), we define the \(\ell^p\)-norm of \(v\), \[ \norm{v}_p = \Paren{\sum_{i=1}^n \abs{v_i}^p}^{1/p}\,. \] For \(p=\infty\), we take the limit, so that \[ \norm{v}_\infty =\max_{i\in [n]} \abs{v_i}\,. \]
Kronecker product
For two matrices \(A\) and \(B\), their Kronecker product is the matrix A\(\otimes B\) with entries \((A\otimes B)_{ii',jj'} = A_{i,j} B_{i',j'}\). This operation also applies to row and column vectors (viewed as matrices with only one column or one row). We use the notation \(A^{\otimes k}=A\otimes \cdots \otimes A\) (\(k\)-times) for the \(k\)-fold tensor power of a matrix \(A\).
Matrices
For matrices with more than two indices, we separate row and column indices by a comma. For example if \(A\) is a linear combination of matrices of the form \(e_i \transpose{(e_j \otimes e_k)}\), we denote the entries of \(A\) by \(A_{i,jk}\). (Note that this convention is consistent with the above notation for Kronecker products.)
Traces
The trace is cyclic, that is, for all matrices \(A\in \R^{m\times n}\) and \(B\in \R^{n\times m}\), \[ \Tr AB = \Tr BA \,. \] A consequence of this property is that for \(x,y\in \R^{n}\) and \(A\in \R^{n\times n}\), \[ \Tr A x \transpose y = \Tr \transpose y A x = \iprod{y, A x}\,. \]
Polynomials
Let \(\R[x]\) be the set of polynomials with real coefficients in variables \(x=(x_1,\ldots,x_n)\). For \(d\in \N\), let \(\R[x]_{\le d}\) be the set of polynomials of degree at most \(d\).