Tensor Decompositions

A tensor of order N is a data structure that generalizes vectors (order one) and matrices (order two). A color picture is an example of a tensor of order 3 and a video sequence is a tensor of order 4. In neural networks, in particular in convolution networks, there exist tensors of even higher orders. The amount of data in such tensors is enormous and they can easily become too large to work with. In such situations, tensor decompositions can help. By approximating a tensor of a high order with a "product" (contraction) of tensors of lower orders, one can dramatically reduce the requirements on memory and speed at the cost of a small and controllable loss of precision. In dynamical programming, which is used in optimization and in optimal control of technological processes, tensor decompositions are used for modeling complicated functions depending on a large number of variables.