## Three interpretations of matrix products

This article explores three different interpretations of matrix products with the help of animations. Each of them provides a different perspective on the matrix product and can be useful in different contexts.

0. Prerequisites and notations

A row vector is shown as a horizontal rectangle and a column vector as a vertical rectangle. The scalar numbers are shown as circles.

A circle next to a rectangle represents the scalar multiplication of the number with the vector. The color of the circle and the rectangle are the same to indicate that they are related.

Finally, the outer product of a column vector and a row vector is shown as a vertical rectangle joined with a horizontal rectangle on the top left corner. This visual aid should make sense if you are familiar with the concept of outer products.

An outer product of a column vector and a row vector

That's it! On to the visualizations now.

1. Interpretation #1: Weighted sum of columns or rows

Scaling a vector by a scalar quantity defined as multiplying each of the vector's components by the scalar. Adding two vectors is defined as adding their corresponding components. This interpretation is quite useful when using matrices in geometry or any visual application. Eg taking a convex sum of n points in a 2D plane (or any higher-dimensional space).

1.1 Matrix vector product as a weighted sum of columns

A matrix vector product can be seen as scaling the columns and then adding them up

1.2 Vector matrix product as a weighted sum of rows

A vector matrix product can be seen as scaling the rows and then adding them up

Note the duality between the two interpretations. The first one is useful when you want to think of the matrix as a collection of columns and the second one is useful when you want to think of the matrix as a collection of rows.

This is covered in more detail in Geometry of weighted sums.

2. Interpretation #2: Dot products of columns or rows with input vector

A dot product is a function that takes two vectors of equal length and returns a scalar quantity. It is defined as the sum of the products of the corresponding components of the two vectors.

2.1 Matrix vector product as a dot product of rows with input vector

A matrix vector product can be seen as list of dot products of each row with the input vector.

2.2 Vector matrix product as a dot product of columns with input vector

A vector matrix product can be seen as a list of dot products of each column with the input vector.

One needs to have some understanding of the dot product to appreciate this interpretation. It is covered in more detail in The Dot Product of Two Vectors (paywalled).

3. Interpretation #3: Matrix-matrix product as the sum of outer products

This is a less common interpretation but is quite useful. An outer product is a function that takes two vectors of arbitrary lengths and returns a matrix. There are two ways to look at this matrix:

1. Each column of the output matrix is the first vector scaled by the corresponding component of the second vector.

2. Each row of the output matrix is the second vector scaled by the corresponding component of the first vector.

A matrix-matrix product can be seen as a sum of outer products of the columns of the first matrix with the rows of the second matrix. This interpretation is useful in studying the spectral properties of symmetric matrices. Eg the product of a symmetric matrix with a vector can be seen as a sum of projections of the vector onto the eigenvectors of the matrix (scaled by the corresponding eigenvalues).

A matrix vector product can be seen as scaling the columns and then adding them up

This is covered in more detail in Symmetric Mv Product (paywalled).

I find this interpretation quite beautiful. Note that when the number of columns in the second matrix is one, it reduces to the weighted sum of columns interpretation. Similarly when the number of rows in the first matrix is one, it reduces to the weighted sum of rows interpretation.

##### Read the e-book 📖

to learn linear algebra from scratch through a series of visual essays.