Einsum Is All You Need – Einstein Summation in Deep Learning

For instance, calculating the dot product of two vectors can be written as \[ {\color{green}c} = \sum_i = \] A problem that I encounter often in deep learning is applying a transformation to vectors in a higher-order tensor. The desired computation can be expressed using einsum \[ {\color{green}C_{ntq}} = \sum_k = \] As a final example, say you are given an order-4 tensor and you are supposed to project vectors in the 3rd dimension to \(Q\) using the projection matrix from before. \[ {\color{green}C_{mqn}} = \sum_t\sum_k = \] Note that transposing the result of the tensor contraction is achieved by swapping \(n\) with \(m\) (\({\color{green}C_{mqn}}\) instead of \({\color{green}C_{nqm}}\)).

Source: rockt.github.io