All About Inverses

A Concise Primer On Linear Algebra

Onkur Sen

Basics

What is a matrix?

\( A \in \mathcal{R}^{m \times n}, A = a_{ij} \) \(m\) = number of rows \(n\) = number of columns \(m=n \Rightarrow \) "square matrix"

Addition

If \(A\) and \(B\) are the same size, then add component by component. \( A, B \in \mathcal{R}^{m \times n}\) \(\Rightarrow (A+B)_{ij} = A_{ij} + B_{ij} \)

Properties of Addition

It is associative. \( (A+B)+C = A+(B+C) \) It is commutative. \( A+B = B+A \) The identity for addition is the matrix of all zeros (written as \(0\)). \( 0_{ij} = 0 \; \forall i, j \) \( A+0 = 0+A = A \)

Multiplication

If \(A\) has the same number of columns as \(B\) has rows, then we can multiply them. \( A\in \mathcal{R}^{m \times n}, B\in \mathcal{R}^{n \times p} \Rightarrow AB\in \mathcal{R}^{m \times p}\) \( (AB)_{ij} = \sum_{k=1}^n a_{ik} b_{kj} \) This is the dot product of the \(i\)th row of \(A\) with the \(j\)th column of \(B\).

Properties of Multiplication

It is associative. \((AB)C\) = \(A(BC)\) It is not necessarily commutative. \(AB \stackrel{?}{=} BA\) The identity for multiplication is the matrix with \(1\)s on the diagonals and zero otherwise. It is written as \(I\). \( I_{ij} = 1 \mbox{ if } i=j, 0 \mbox{ otherwise} \) \( IA = AI = A \)

Transpose

The transpose of a matrix \(A\) is another matrix with the rows and columns of \(A\) inverted. It is written as \( A^T \). \( A^T_{ij} = A_{ji} \)Note: \( (A^T)^T = A \)

Inverse of a Matrix

What is it?

The inverse of a square matrix \(A\) is a matrix that multiplies with \(A\) to make the identity matrix. It is written as \(A^{-1}\). \( AA^{-1} = A^{-1}A = I \)BUT it does not exist for all \(A\).

Why is it important?

Invertibility comes up again and again in linear algebra. We'll examine four different use cases.

Row Operations

(and inverses)

Elementary Row Operations

There are only three!

Switch two rows: \( r_i, r_j \rightarrow r_j, r_i\) Multiply a row by a constant: \( r_i \rightarrow Cr_i\) Add two rows together: \( r_i \rightarrow r_i + r_j\)

You can also do 2 and 3 at the same time: \( r_i \rightarrow C_1 r_i + C_2 r_j\)

Can you invert a matrix? (Part 1)

\(A\) is invertible if it can be reduced to \(I\) using elementary row operations. \(A^{-1}\) is the result of the same operations applied to \(I\). The method to do this on the augmented matrix \( [A \; \vert \; I]\) is called Gaussian elimination.

Intuition for inverting Part 1

By going from \(A\) to \(I\) with row operations, we basically multiply by \(A^{-1}\) (since \(AA^{-1}=I\)). Furthermore, since \(IA^{-1} = A^{-1}\), we can do the same operations on \(I\) to get \(A^{-1}\) explicitly.

Linear independence

(and inverses)

Vectors

A vector \(v\) is a matrix with one column. \( v \in R^n \)

Linear independence

A set of vectors \(\{v_i\}_{i=1}^n\) is linearly independent if the only linear combination equal to zero has all coefficients equal to zero. \( \{v_i\}_{i=1}^n \) linearly independent \( \Longrightarrow \left( \sum_{i=1}^n c_i v_i = 0 \Leftrightarrow c_i = 0 \; \forall i \right) \) Alternately, \(\{v_i\}_{i=1}^n\) is linearly independent if no vector \(v_j\) can be created from a linear combination of the other vectors.

Can you invert a matrix? (Part 2)

\(A\) is invertible if its rows are linearly independent. \(A\) is also invertible if its columns are linearly independent.

Determinants

(and inverses)

What is it?

The determinant is a value associated with square matrices. It is written as \( \det(A) \).

Can you invert a matrix? (Part 3)

A square matrix \(A\) is invertible if \( \det(A) \neq 0 \).

How do you calculate it?

It's best shown with examples.

2 x 2 Determinant

\[ A = \left(\begin{array}{cc} a & b\\ c & d\\ \end{array}\right) \] \( \Rightarrow \det(A) = ad-bc \)

3 x 3 Determinant

\[ A = \left( \begin{array}{ccc} a_1 & a_2 & a_3\\ b_1 & b_2 & b_3\\ c_1 & c_2 & c_3\\ \end{array} \right) \] \( \Rightarrow \det(A) = a_1(b_2c_3-b_3c_2) - \) \( a_2(b_1c_3-b_3c_1) + \) \( a_3(b_1c_2-b_2c_1)\)

n x n Determinant

For any row \(A_i\), \[ \det(A) = \sum_{j=1}^n a_{ij} \cdot (-1)^{i+j} \cdot M_{ij}, \] where \( M_{ij} \) is the (i, j) minor matrix.

Eigenvalues,Eigenvectors

(and inverses)

What are they?

For a square matrix \(A\), we often examine special vectors \(v\) that do not rotate when \(A\) is applied. That is, for a certain scalar \(\lambda\), \(Av = \lambda v\). In this case, we call \(\lambda\) and \(v\) an eigenvalue and eigenvector of \(A\).

Why are they important?

After doing a bit of algebra, we can notice: \( Av = \lambda v \) \( \Rightarrow Av - \lambda Iv = 0 \) \( \Rightarrow (A - \lambda I)v = 0. \) Now we can make an assertion...

Can you invert a matrix? (Part 4)

Assertion

If \(v \neq 0\), then \((A-\lambda I)\) is not invertible. Thus, \(\det{(A-\lambda I)} = 0\).

Proof

Suppose \((A-\lambda I)^{-1}\) exists. \( (A-\lambda I)v = 0 \) \( \Rightarrow (A-\lambda I)^{-1}(A-\lambda I)v = 0 \) \( \Rightarrow Iv = 0\) \( \Rightarrow v=0. \) This contradicts our original assumption that \(v \neq 0\).QED

How do I find eigenvalues?

The \(v=0\) case is boring. \( A\times 0 = 0 = \lambda \times 0 \Rightarrow \lambda\) can be anything. So, let's focus on \(v \neq 0\), so that \(\det{(A-\lambda I)} = 0\). The left-hand side is an \(n^{th}\)-order polynomial in \(\lambda \). Thus, by the Fundamental Theorem of Algebra, every square matrix has \(n\) eigenvalues. However, the eigenvalues are not necessarily distinct.

How do I find eigenvectors?

Suppose the eigenvalues \( \{\lambda_i\}_{i=1}^n \) are all distinct. To get the eigenvectors \( \{v_i\}_{i=1}^n \) , solve the original equations: \( (A - \lambda_i)v_i = 0, i = 1,\ldots,n \). This produces \(n\) simultaneous linear equations. Then, write each \(v_i\) in terms of one common component. Finally, factor the common component out. For example, if at first we obtain \( (v_1, 3v_1, 2v_1) \), then the actual eigenvector is simply \( (1, 3, 2) \).

Thanks for reading!

Resources

Matrices and Linear Algebra: accessible introductory pamphletCourse 18.06SC: MIT's famous linear algebra courseLinear Algebra Done Right: a canonical textbook

Like this? Questions? Comments?

Check out my website.Email me.Tweet @onkursen.

All About Inverses – A Concise Primer On Linear Algebra – Onkur Sen

onkursen

All About Inverses – A Concise Primer On Linear Algebra – Onkur Sen

0 1 (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/platform.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();