The Importance of Eigenvectors

An Intuitive Explanation of Eigenstuff

This is part 6 of my linear algebra series! Here are all the previous posts to catch you up:

1. 3 Ways To Think About Vectors

2. Why Is It Called LINEAR Algebra?

3. Basis Vectors And Pokemon

4. What Is A Matrix?

5. Determinants and The Rank-Nullity Theorem

By far the most confusing part of my linear algebra class was learning about eigenvectors and eigenvalues. While I was able to memorize how to calculate them, I could never really wrap my head around what they really represented, and why they were interesting or useful, until only recently. So here’s my explanation!

In this post, we’ll cover:

1. The Technical Definition

2. Visual Examples

3. Similar Matrices

4. WHY are Eigenvectors Important?

---

1. The Technical Definition

First we have to get the official definition out of the way, and then I’ll show you some examples!

Let A be a nxn matrix, and v a non-zero vector. Then v is called an eigenvector of A if:

\(A\vec{v} = \lambda \vec{v}\)

That is, matrix multiplication of A times v is the same as multiplying v by some scalar multiple λ, which we call the corresponding eigenvalue of v.

2. Visual Examples

Example 1

\(\text{Let } A = \begin{bmatrix} 5 & 0 \\ 0 &1 \end{bmatrix}\)

Watch the linear transformation represented by matrix A:

This is the horizontal stretch transformation, which stretches the grid by multiplying the x-coordinate by a factor of 5 and doing nothing to the y-coordinate.

I’ve plotted a few random vectors on the grid so we can see how matrix A transforms them all.

Notice how the green vector (1,0) gets mapped to (5,0), and the red vector (0,1) gets mapped to (0,1). See how neither of them change direction?

They almost represent a centre of gravity for the other vectors to rotate towards or away from. Thus the vectors (1,0) and (0,1) are called the eigenvectors of the matrix, and their scaling factors (5 and 1) are called the eigenvalues. We will see that these vectors and scaling factors represent a fundamental aspect of the matrix.

Example 2

\(\text{Let } B = \begin{bmatrix} 4 & 1 \\ 3 & 2 \end{bmatrix}\)

Here we have matrix B which transforms the grid by stretching sort of diagonally up and to the right.

The eigenvectors of matrix B are here in purple (1,1) and orange (-1,3).

They get stretched by a factor of 5 and 1, so these are the corresponding eigenvalues.

Note: the vectors (-1,-1) and (1, -3) also don’t change directions, but these are just scalar multiples of (1,1) and (-1,3), so we forget about them and we’re only concerned with linearly independent eigenvectors.

3. Similar Matrices

Do you notice how Example 1 and 2 seem a little similar? We looked at two separate matrices A and B, and they both represented stretching in one direction by a factor of 5, and stretching the other direction by a factor of 1?

The transformation by matrix B almost looks like a horizontal stretch transformation just rotated a little bit.

Indeed, these two matrices do represent the same linear transformation, just under a different choice of basis!

This is made a little more clear if we hide the x and y axis, and redraw the grid using (1,1) and (-1,3) as our new basis.

We say that A and B are similar matrices, since they represent the same linear transformation, T:

\(T: \mathbb{R}^2 \rightarrow \mathbb{R}^2\)

Matrix A represents T for the basis {(1,0), (0,1)} and matrix B represents T for the basis {(1,1), (-1,3)}. These are the eigenvectors!

One reason why eigenvectors are so important to mathematicians is because they provide a way of finding similar matrices that represent the same linear transformation.

A and B are similar matrices, but A is by far the superior one. It even has the eigenvalues of B as the entries on its diagonal:

\(A = \begin{bmatrix} 5&0\\0&1 \end{bmatrix}\)

If you know the eigenvectors of B, you can perform a change of basis and transform matrix B into matrix A as follows:

\(B = \begin{bmatrix} 4&1\\3&2\end{bmatrix} \text{ has eigenvectors } \begin{bmatrix} 1\\1\end{bmatrix} , \begin{bmatrix} -1\\3\end{bmatrix}. \text{ Let } Q = \begin{bmatrix} 1&-1\\1&3\end{bmatrix}\)

\(Q^{-1}BQ = A\)

\(\begin{bmatrix} 1&-1\\1&3\end{bmatrix}^{-1} \begin{bmatrix} 4&1\\3&2\end{bmatrix}\begin{bmatrix} 1&-1\\1&3\end{bmatrix} = \begin{bmatrix} 5&0\\0&1\end{bmatrix}\)

Note: the exponent of -1 here represents the matrix inverse.

By the same process, we can decompose matrix C into matrix D using eigenvectors:

\(C = \begin{bmatrix} 51&-40&10\\78&-62&16\\72&-60&17\end{bmatrix} \hspace{2cm} D = \begin{bmatrix} 1&0&0\\0&2&0\\0&0&3\end{bmatrix}\)

Here, matrices C and D are similar, and represent the same linear transformation, but D is a far superior matrix.

For starters, it’s easier to understand what’s going on when looking at D: it just stretches the y-coordinate by 2 and the z-coordinate by 3.

From a practical perspective, it’s much easier to multiply by D than it is to multiply by C. Lots of math problems involve multiplying by a matrix many thousands of times, and sometimes that matrix has 100,000x100,000 entries! If we were to multiply matrix C by itself many times, it would be very computationally slow. Just try it yourself! On the other hand, multiplying D by itself is easy. Raising D to the nth power looks like this:

\(D^n = \begin{bmatrix} 1^n&0&0\\0&2^n&0\\0&0&3^n\end{bmatrix}\)

And it should come at no surprise that these diagonal entries 1, 2 and 3 are the eigenvalues of C!

D is called a diagonal matrix. So is A.

4. Why Eigenvectors Are Important

Eigenvectors and eigenvalues are so important because they help us understand and simplify matrices. They provide a way of identifying when two different matrices actually represent the same linear transformation, just under different bases. And they also provide a way of choosing a basis for a vector space so that a linear transformation is represented by a diagonal matrix, which has a lot of nice properties. In particular, diagonal matrices make computations much faster, so using eigenvectors to find these similar diagonal matrices is crucial for making computer programs run more efficiently.

In the next post, we’ll wrap up everything we’ve learned about linear algebra (systems of equations, linear transformations, determinants, kernels, images, and eigenvectors) in one grand theorem that links all these concepts together! See you then!