Notations#
In this document, matrices are typically represented by uppercase letters, such as \(M\), while vectors and scalars are denoted by lowercase letters, like \(x\). Scalar values are often indicated using lowercase Greek \(\lambda\). The transpose of a matrix or vector is indicated with a superscript \(M^T\). Slicing of tensors is performed as in python. For instance, the entry at row \(i\) and column \(j\) of a matrix \(M\) is expressed h as \(M[i,j]\). To remove the \(j\)th column of a matrix \(M\), we write \(M[:,-j]\). Notation \(\mathbb{R}_+\) denotes the set of nonnegative real numbers, and constraints like \(x \geq 0\) indicate elementwise nonnegativity. The Frobenius norm is denoted as \(\|M\|_F\). The support of a vector \(x\), or the set of indices of its nonzero elements, is denoted by \(\text{Supp}(x)\) [TODO check]. The Kronecker product is symbolized by \(\boxtimes\) when it can be confused with another tensor product, or by \(\otimes\) otherwise. The gradient of a function \(f\) is written as \(\nabla f\), and its Hessian as \(\nabla^2 f\). The (left) Moore-Penrose pseudo-inverse of a matrix \(M\) is represented by \(M^\dagger\). The Kullback-Leibler divergence between two nonnegative reals \(y\) and \(z\) is denoted as \(\KL{y, z}\). Finally, the spark of a matrix \(M\), or the smallest number of linearly dependent columns in \(M\), is written as \(\text{spark}(W)\).
Tensor notations#
The unfolding of a tensor \(T\) along mode \(i\), denoted \(T_{[i]}\), is obtained by row-first vectorization of all modes but \(i\). The Khatri-Rao product of two matrices \(A\) and \(B\) with the same number \(r\) of columns is the columnwise Kronecker product,
The multiway product can be defined for three matrices \(A\), \(B\), \(C\) multiplied on modes one, two and three with a tensor \(G\) as
It can be implemented with tensor contractions, or with matrix-matrix products with unfoldings and reshapes involved.
The CP decomposition is represented using the Kruskal notation \(\llbracket A,B,C\rrbracket\).