Solutions to a Linear System

We have talked a lot about linear systems, how many solutions they can have, what their matrices represent, etc., but we never talked about how to systematically find the solution to a linear system of equations.

In case it wasn’t clear - because I already did this in the lesson about linear dependence - a linear system with an $m \times n$ matrix can be rewritten as a linear system of equations with $m$ equations and $n$ variables.

Take the linear system

\begin{align*} A \vec{x} &= \vec{b} \\ \begin{bmatrix} 1 & 2 & 1 \\ 2 & -1 & 3 \\ 3 & 1 & 2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} &= \begin{bmatrix} 4 \\ 7 \\ 5 \end{bmatrix}. \end{align*}

This can be rewritten as

\begin{cases} x_1 + 2 x_2 + x_3 = 4 \\ 2 x_1 - x_2 + 3 x_3 = 7 \\ 3 x_1 + x_2 + 2 x_3 = 5. \end{cases}

Conversely, you can rewrite any linear system of equations in the form $A \vec{x} = \vec{b}$ .

More formally, any linear system of equations in the form

\begin{cases} a_{11} x_1 + a_{12} x_2 + \cdots + a_{1n} x_n = b_1 \\ a_{21} x_1 + a_{22} x_2 + \cdots + a_{2n} x_n = b_2 \\ \vdots \\ a_{m1} x_1 + a_{m2} x_2 + \cdots + a_{mn} x_n = b_m \\ \end{cases},

can be rewritten in the form

A \vec{x} = \vec{b},

where

A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}, \quad \vec{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \quad \vec{b} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix},

and vice-versa.

Visual Intuition

You should already be familiar with some method to solve a linear system of equations from earlier in your studies. We will learn more advanced and mechanical methods later in the course, but for now we will stick to what we already know.

I already mentioned the visual intuition for a linear system of equations in my lesson about linear dependence.

There, I talked about the possibility of thinking of a linear system with a $2 \times 2$ matrix as two lines in a 2D-space. Their intersection (if it exists) is the solution to the linear system. There are two ways of thinking of the solution to a linear system. This is one of them, and it’s called the row picture. There’s also the so-called column picture, and we’ll look into both versions now.

We take the linear system

\begin{bmatrix} 1 & 2 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \end{bmatrix}

as an example for the explanation.

Row Picture

You can perform the matrix-by-vector product between $A$ and $\vec{x}$ , and then separate the entire equation into rows. This is what we did above. You basically rewrite the system into a system of equations.

So we multiply $A$ by $\vec{x}$ , and our linear system becomes

\begin{align*} \begin{bmatrix} 1 & 2 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \end{bmatrix} \\ \begin{bmatrix} x_1 + 2x_2 \\ -x_1 + x_2 \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \end{bmatrix}. \end{align*}

If we split everything up into rows we are left with two equations:

\begin{cases} x_1 + 2x_2 = 4 \\ -x_1 + x_2 = -1. \end{cases}

If we plot $x_2$ on the $y$ -axis and $x_1$ on the $x$ -axis, we get two lines, which intersect at some point, since this system is non-singular.

As you can see, the lines cross at the point $(2,1)$ . In fact, this is also the solution to our system. Look, we will try $x_1 = 2$ and $x_2 = 1$ :

\begin{align*} &\begin{cases} x_1 + 2x_2 = 4 \\ -x_1 + x_2 = -1. \end{cases} \\ &\begin{cases} 2 + 2 \cdot 1 = 4 \\ -2 + 1 = -1 \end{cases} \\ &\begin{cases} 4 = 4 \\ -1 = -1. \end{cases} \end{align*}

A similar idea holds for 3D-Systems as well. In those cases, instead of lines, we’re dealing with planes intersecting each other. Two planes intersecting form a line, and if you add a third plane, you get a point.

So we saw that one way to think of the solution of a linear system is by splitting it up into rows and plotting the individual equations.

Column Picture

Another way of thinking about the solution to a linear system is by rewriting it as a linear combination of the columns. This remains closer to the matrix-by-vector product between $A$ and $\vec{x}$ , where we think of the left side of $A\vec{x} = \vec{b}$ as a linear combination of the columns of $A$ .

We use the same linear system we had above:

\begin{bmatrix} 1 & 2 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \end{bmatrix}.

The multiplication between $A$ and $\vec{x}$ can be rewritten as

x_1 \begin{bmatrix} 1 \\ -1 \end{bmatrix} + x_2 \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \end{bmatrix}.

If we now try the solution we found before ( $x_1 = 2$ , $x_2 = 1$ ), it will still work:

\begin{align*} 2 \begin{bmatrix} 1 \\ -1 \end{bmatrix} + 1 \begin{bmatrix} 2 \\ 1 \end{bmatrix} &= \begin{bmatrix} 4 \\ -1 \end{bmatrix} \\ \begin{bmatrix} 2 \\ -2 \end{bmatrix} + \begin{bmatrix} 2 \\ 1 \end{bmatrix} &= \begin{bmatrix} 4 \\ -1 \end{bmatrix} \\ \begin{bmatrix} 4 \\ -1 \end{bmatrix} &= \begin{bmatrix} 4 \\ -1 \end{bmatrix}. \end{align*}

What we’re asking now is “what linear combination of the columns of $A$ gets us to the vector $\vec{b}$ ?”. This can be represented like this:

Where $\vec{a}_1$ and $\vec{a}_2$ are the first and second columns of $A$ respectively. To reach $\vec{b}$ , we had to add $2\vec{a}_1$ and $\vec{a}_2$ .

Conclusion

So in conclusion, the row picture focuses on the equations themselves. We want to find a solution $x_1, x_2, \cdots, x_n$ to solve all equations simultaneously.

The column picture, on the other hand, focuses on the columns of $A$ . We see the solution as trying to find the coefficients $x_1, x_2, \cdots, x_n$ such that the linear combination $x_1 \vec{a}_1 + x_2 \vec{a}_2 + \cdots + x_n \vec{a}_n$ results in $\vec{b}$ .

Both viewpoints really describe the same problem, but from different perspectives.