Notation

Notation#

General notation#

  • Matrices are typically represented by uppercase letters, such as \(M\), while vectors and scalars are denoted by lowercase letters, like \(x\).

  • Scalar values are often indicated using lowercase Greek \(\lambda\).

  • The transpose of a matrix or vector is indicated with a superscript \(M^T\).

  • Slicing of tensors is performed as in python. For instance, the entry at row \(i\) and column \(j\) of a matrix \(M\) is expressed h as \(M[i,j]\). To remove the \(j\)th column of a matrix \(M\), we write \(M[:,-j]\). To consider all columns and rows up to indices \(i\) and \(j\), we write \(M[:i,:j]\).

  • Notation \(\mathbb{R}_+\) denotes the set of nonnegative real numbers, and constraints like \(x \geq 0\) indicate elementwise nonnegativity.

  • The Frobenius norm is denoted as \(\|M\|_F\).

  • The support of a vector \(x\), or the set of indices of its nonzero elements, is denoted by \(S(x)\), or simply \(S\) when clear from context.

  • The number of elements of a discrete set \(\mathcal{K}\) is denoted by \(\#\mathcal{K}\).

  • The Kronecker product is symbolized by \(\otimes_K\) to avoid confusion with the generic tensor product \(\otimes\).

  • The gradient of a function \(f\) is written as \(\nabla f\), and its Hessian as \(\nabla^2 f\).

  • The (left) Moore-Penrose pseudo-inverse of a matrix \(M\) is represented by \(M^\dagger\).

  • The Kullback-Leibler divergence between two nonnegative reals \(y\) and \(z\) is denoted as \(\KL{y, z}\).

  • The spark of a matrix \(M\), or the smallest number of linearly dependent columns in \(M\), is written as \(\text{spark}(W)\).

  • The column space, or span, of a matrix \(U\) is the set of any vector that can be written as \(Ux\) and is denoted as \(\text{col}(U)\). The positive span of matrix \(U\), obtained when \(x\) is elementwise nonnegative, is denoted by \(\cp{U}\).

  • The set of linear maps acting on vectors from a subspace \(E\) is denoted as \(\mathcal{L}(E)\).

  • The matrix of vector full of ones is written either \(1_{n}\) where \(n\) is its dimension, or sometimes simply \(1\).

Tensor notation#

The unfolding of a tensor \(T\) along mode \(i\), denoted \(T_{[i]}\), is obtained by row-first vectorization of all modes but \(i\). The Khatri-Rao product of two matrices \(A\) and \(B\) with the same number \(r\) of columns is the columnwise Kronecker product,

\[ A\odot B = \left[A[:,1]\otimes_K B[:,1], \ldots, A[:,r]\otimes_K B[:,r] \right]. \]

The multiway product can be defined for three matrices \(A\), \(B\), \(C\) multiplied on modes one, two and three with a tensor \(G\) as

\[\left(A\times_1 B\times_2 C\times_3 G\right)[i,j,k] = \sum_{r_1,r_2,r_3}A[i,r_1]B[j,r_2]C[k,r_3]G[r_1,r_2,r_3].\]

It can be implemented with tensor contractions, or with matrix-matrix products with unfoldings and reshapes involved.

The CP decomposition is represented using the Kruskal notation \(\llbracket A,B,C\rrbracket\).