Inverse Matrix of a Linear System

From the introductory lesson on linear systems, we have learned that a linear system is of the form

Such a system does not have to have only one solution. In fact, it can have infinitely many solutions, but also no solution at all. It could also have exactly one.

For now we concern ourselves with the case in which a linear system has exactly one solution. One way this can happen, for example, is when the system has a non-singular (or invertible) matrix.

This terminology might seem counter-intuitive, since a non-singular matrix is found in a linear system which has exactly one solution. Keep this in mind to avoid confusing the terminology.

A matrix in a linear system that is non-singular is invertible. To understand what this means, I will use an example from scalar algebra, which is more familiar than matrix algebra.

Parallel Example From Scalar Algebra

Take the equation

where $a$, $x$, and $b$ are all scalars. Say that we want to find the unknown $x$ and are given values for $a$ and $b$. To solve the equation, we could divide both sides by $a$:

What we did was divide by $a$, which is the same as multiplying by $\frac{1}{a}$, which is the same as multiplying by $a^{-1}$.

$a^{-1}$ is said to be the multiplicative inverse of $a$, because

What we did was basically

In the context of scalars, $1$ is the multiplicative identity, which is the equivalent of an identity matrix in linear algebra.

How This Applies to Matrices

The same can be said about matrices. A matrix $A$ is said to be invertible when there exists some matrix $A^{-1}$ such that

where $I$ is the identity matrix of the appropriate size.

Not all matrices are invertible. For some matrices $A$ it is possible to find $A^{-1}$, for others, this is not the case. Whether a matrix $A$ in a linear system is invertible or not tells us a lot about the potential solutions to the system. When the matrix $A$ is invertible, we know that there will be exactly one solution which is given by

I can already tell you that only square matrices are invertible, whereas rectangular matrices are never invertible in the usual sense.

This is similar to what we saw in the example from scalar algebra that I used above. You just multiply both sides of the equation $A \vec{x} = \vec{b}$ by $A^{-1}$. On the left side, you are left with $I \vec{x}$ which is the same as $\vec{x}$. On the right side, you get $A^{-1} \vec{b}$.

Next Steps

We now covered the case for which a linear system has exactly one solution, which is what we would expect and what is most intuitive. But naturally, since I said that in certain cases, $A^{-1}$ does not exist, what happens in those cases? This is the next question we must ask which I will cover in subsequent lessons.