The Geometry of Linear Equations

0. Every word, from zero

Read this first. Every word used in this lecture is explained from the very beginning, with a tiny example and an everyday picture. Each section below repeats the words it uses, so you never have to scroll back.

Word	Plain meaning	Tiny example	Everyday picture
Unknown	a number we don't know yet and want to find; we give it a letter	$x$ in $x + 1 = 3$ (it is $2$)	the price tag hidden under your thumb
Equation	a sentence "left side = right side" that must be true	$x + 1 = 3$	a balance scale that must stay level
Coefficient	the number that multiplies an unknown	in $2x$, the coefficient is $2$	"2 packets of" something
Linear equation	each unknown is only multiplied by a number, then the pieces are added	$2x - y = 0$ (linear); $x^2 + y = 1$ (not)	a straight-line rule, no curves
System of equations	several equations that must all be true at the same time	$2x - y = 0$ and $-x + 2y = 3$	several rules a recipe must obey at once
Solution	values of the unknowns that make every equation true	$x = 1, y = 2$ for the pair above	the one recipe that passes every rule
Substitution	find one unknown in terms of another, then put that into a different equation	$y = 2x$, so $-x + 2y$ becomes $-x + 4x$	swapping a word for its meaning
Vector	an ordered list of numbers, written as a column	$\begin{bmatrix} 2 \\ 1 \end{bmatrix}$	an arrow: "2 steps right, 1 step up"
Component	one number inside a vector	$(2, 1)$ has components $2$ and $1$	one item on a shopping list
Real numbers $\mathbb{R}$	every number on the number line	$0,\ -3,\ 2.5,\ \pi$	every mark on a ruler, including between the lines
$\mathbb{R}^2$, $\mathbb{R}^3$, $\mathbb{R}^n$	all vectors with 2, 3, or $n$ components	$(2, 1)$ is in $\mathbb{R}^2$	a flat sheet (2), a room (3), too many directions to draw ($n$)
Origin	the zero point, where every arrow starts	$(0, 0)$	"home" on a map
Scalar	a single ordinary number (not a list)	$2$, $-1$	a zoom factor
Scalar multiplication	multiply every component of a vector by one number	$2(2, 1) = (4, 2)$	walking the same way, twice as far
Vector addition	add component by component	$(2, 1) + (-1, 2) = (1, 3)$	walk one arrow, then the next from where you stopped
Linear combination	scale some vectors, then add them	$1(2, -1) + 2(-1, 2) = (0, 3)$	a recipe: 1 cup of this + 2 cups of that
Weights	the numbers used in a linear combination	the $1$ and $2$ above	how many cups of each ingredient
Matrix	a rectangular table of numbers	$\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}$	a spreadsheet
Row / column	one line read left→right / one line read top→bottom	row 1 $= (2, -1)$; column 2 $= (-1, 2)$	row = one scenario; column = one product
$m \times n$	a matrix with $m$ rows and $n$ columns	$\begin{bmatrix} 1 & 2 & 3 \end{bmatrix}$ is $1 \times 3$	"rows by columns", like 3 shelves by 4 boxes
Coefficient matrix $A$	the table of all coefficients: row $i$ = equation $i$, column $j$ = unknown $j$	for $2x - y = 0,\ -x + 2y = 3$: $A = \begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}$	the menu card: what each ingredient adds
Right-hand side $b$	the vector of numbers the equations must equal	$b = (0, 3)$	the target, the order the client placed
$Ax = b$	the whole system in one line: matrix times unknowns equals target	$\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}\begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix}$	"which recipe makes this dish?"
Row picture	draw each equation as a line (or plane); the solution is where they all meet	two lines crossing at $(1, 2)$	two roads crossing at one junction
Column picture	see $Ax = b$ as "which mix of the column arrows lands on $b$?"	$1\,(\text{col }1) + 2\,(\text{col }2) = b$	walking along given arrows to reach a treasure
Plane	a flat surface that goes on forever; a linear equation in 3 unknowns draws one	$2x - y = 0$ in 3D	an endless sheet of glass
Hyperplane	the "flat" shape one linear equation makes in 4 or more dimensions	$x_1 + x_2 + x_3 + x_4 = 1$	a plane you can't draw
Dimension	how many separate directions a space has	a line: 1; a sheet: 2; a room: 3	how many numbers you need to say "where"
Span	every point you can reach with all combinations of some vectors	span of $(1, 0)$ and $(0, 1)$ = whole plane	every spot you can walk to using only those arrows
Column space	the span of the columns of a matrix	for $A$ above: all of $\mathbb{R}^2$	every payoff your products can build
Parallel	lines (or planes) with the same direction that never meet	$x + y = 1$ and $x + y = 3$	two railway tracks
Singular	a square matrix whose columns fill less than the whole space; some $b$ can't be reached	$\begin{bmatrix} 1 & 1 \\ 2 & 2 \end{bmatrix}$	two arrows pointing the same way: you're stuck on one road
Non-singular / invertible	a square matrix whose columns fill the whole space: exactly one solution for every $b$	$\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}$	arrows pointing different ways: you can go anywhere
Dot product	multiply matching entries of two lists, then add	$(2, 5) \cdot (1, 2) = 2 + 10 = 12$	price list · quantity list = bill
Payoff	the money a product pays at expiry, in each possible scenario	20,000 CE pays $\max(S - 20{,}000, 0)$	what the ticket is worth when the match ends
CE / PE	call option (pays if Nifty ends above the strike) / put option (pays if it ends below)	19,800 CE at 20,000 pays $200$ points	a bet on "up" / a bet on "down"
Strike	the fixed level written into an option	the "20,000" in 20,000 CE	the line the bet is measured from
Lot	one unit of a traded product (here we just count lots)	"1 lot of A"	one packet
Bond	pays the same fixed amount whatever happens	pays $1$ if up, $1$ if down	money in a locker
Replication	building a wanted payoff out of products you can trade	2 bonds + 1 stock pays $(5, 3)$	cooking a restaurant dish at home from basic ingredients
Complete market	every payoff can be replicated (the payoff columns fill the whole space)	bond + stock in a two-state market	a kitchen where any dish can be made

1. The problem: solving $Ax = b$

View	You look at…	The question becomes…
Row picture	one equation (row) at a time	Where do all the lines or planes meet?
Column picture ★	one column at a time	What combination of the columns makes $b$?
Matrix form	the whole system at once	Find $x$ with $Ax = b$. This is the compact algebra behind both pictures.

2. Vectors: the building blocks

What are $\mathbb{R}$, $\mathbb{R}^2$, $\mathbb{R}^3$ and $\mathbb{R}^n$?

$\mathbb{R}$ stands for the real numbers: every number on the number line, such as $0$, $-3$, $2.5$, $\sqrt{2}$ and $\pi$. The small raised number says how many components each vector has.

Symbol	What it contains	Picture	Example
$\mathbb{R}$	single real numbers	a line (1D)	$5$
$\mathbb{R}^2$	all vectors with 2 components $(x, y)$	the flat plane (2D), like a sheet of paper	$(2, 1)$
$\mathbb{R}^3$	all vectors with 3 components $(x, y, z)$	ordinary 3D space, like a room	$(1, -2, 4)$
$\mathbb{R}^n$	all vectors with $n$ components	$n$-dimensional space (cannot be drawn when $n > 3$)	$(v_1, v_2, \dots, v_n)$

Read $\mathbb{R}^2$ as "R-two". So "$v$ is in $\mathbb{R}^2$" simply means "$v$ is a list of 2 real numbers".

A vector as an arrow

Every vector can be drawn as an arrow that starts at the origin $(0, 0)$. Its components say how far to move along each axis:

• first component $2$: move 2 to the right (along the $x$-axis);
• second component $1$: move 1 up (along the $y$-axis).

The arrow ends at the point $(2, 1)$. So the same two numbers can be seen as a point or as an arrow to that point. In linear algebra we usually think of the arrow.

In $\mathbb{R}^3$ it works the same way with a third direction: $(1, -2, 4)$ means 1 along $x$, 2 backwards along $y$, and 4 up along $z$.

We only ever need two operations on vectors:

1. What we do Scalar multiplication: multiply $v = (2, 1)$ by the number $2$.

   Why Multiplying by a number stretches, shrinks or flips the arrow. We want to see "twice as far, same direction".

   How Multiply each component on its own. Top: $2 \times 2 = 4$. Bottom: $2 \times 1 = 2$.

   $$ 2\begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 \times 2 \\ 2 \times 1 \end{bmatrix} = \begin{bmatrix} 4 \\ 2 \end{bmatrix} $$

   What we have now $2v = (4, 2)$: same direction as $v$, twice as long.

2. What we do Multiply $v = (2, 1)$ by $-1$.

   Why A minus number flips the arrow to point the opposite way.

   How Top: $-1 \times 2 = -2$. Bottom: $-1 \times 1 = -1$.

   $$ -1\begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} -2 \\ -1 \end{bmatrix} $$

   What we have now $-v = (-2, -1)$: same length, pointing backwards (2 left, 1 down).

3. What we do Vector addition: add $v = (2, 1)$ and $w = (-1, 2)$.

   Why Adding means "walk along $v$, then walk along $w$ starting where $v$ ended" (head-to-tail). The sum is where you finish.

   How Add the top numbers: $2 + (-1) = 2 - 1 = 1$. Add the bottom numbers: $1 + 2 = 3$.

   $$ \begin{bmatrix} 2 \\ 1 \end{bmatrix} + \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 + (-1) \\ 1 + 2 \end{bmatrix} = \begin{bmatrix} 1 \\ 3 \end{bmatrix} $$

   What we have now $v + w = (1, 3)$: 1 right and 3 up from the origin.

3. From equations to matrix form

Our first example has two equations and two unknowns:

$$ \begin{aligned} 2x - y &= 0 \\ -x + 2y &= 3 \end{aligned} $$

Collect the numbers into three objects: the coefficient matrix $A$, the vector of unknowns $x$, and the right-hand side $b$.

$$ \underbrace{\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}}_{A} \underbrace{\begin{bmatrix} x \\ y \end{bmatrix}}_{x} = \underbrace{\begin{bmatrix} 0 \\ 3 \end{bmatrix}}_{b} $$

How to read the matrix:

Part of $A$	Corresponds to	In our example
Row $i$	equation $i$	Row 1 $= (2, -1)$ holds the coefficients of $2x - y = 0$
Column $j$	unknown $j$	Column 1 $= (2, -1)$ holds every coefficient of $x$; column 2 $= (-1, 2)$ holds every coefficient of $y$

4. The row picture

Idea: take one equation at a time and draw all the points $(x, y)$ that satisfy it. Each equation gives a line. A solution of the system must satisfy every equation, so it must lie on every line: it is where the lines meet.

How to draw each line

A line is fixed by any two of its points. An easy method is to set one unknown to $0$ and solve for the other, then pick one more convenient value.

	Equation 1: $2x - y = 0$	Equation 2: $-x + 2y = 3$
Through origin?	Yes: $(0,0)$ gives $0 = 0$ ✓	No: $(0,0)$ gives $0 \ne 3$
Point A	$(0, 0)$	$y = 0 \Rightarrow x = -3$: $(-3, 0)$
Point B	$x = 1 \Rightarrow y = 2$: $(1, 2)$	$x = -1 \Rightarrow y = 1$: $(-1, 1)$

The same table, one tiny move at a time:

1. What we do Test whether line 1, $2x - y = 0$, goes through the origin.

   Why If $(0, 0)$ works, we get a first point for free.

   How Put $x = 0$, $y = 0$: $2 \times 0 - 0 = 0 - 0 = 0$. The right side is $0$. Equal ✓.

   What we have now Point A of line 1 is $(0, 0)$.

2. What we do Find a second point on line 1 by choosing $x = 1$.

   Why Two points fix a line. $x = 1$ is an easy number.

   How $2 \times 1 - y = 0$, so $2 - y = 0$. "2 minus what is 0?" Add $y$ to both sides: $2 = y$.

   What we have now Point B of line 1 is $(1, 2)$.

3. What we do Test line 2, $-x + 2y = 3$, at the origin.

   How $-0 + 2 \times 0 = 0$, but the right side is $3$, and $0 \ne 3$.

   What we have now Line 2 misses the origin. We need two other points.

4. What we do On line 2, choose $y = 0$.

   How $-x + 2 \times 0 = 3$, so $-x + 0 = 3$, so $-x = 3$. Flip both signs: $x = -3$.

   What we have now Point A of line 2 is $(-3, 0)$.

5. What we do On line 2, choose $x = -1$.

   How $-(-1) + 2y = 3$. Minus a minus is a plus, so $1 + 2y = 3$. Take 1 from both sides: $2y = 3 - 1 = 2$. Halve both sides: $y = 2 \div 2 = 1$.

   What we have now Point B of line 2 is $(-1, 1)$. Both lines can now be drawn.

Check the solution in both original equations:

1. What we do Put $x = 1$, $y = 2$ into equation 1, $2x - y = 0$.

   Why The crossing point must make every equation true. Reading it off a picture is not proof; the numbers are.

   How $2 \times 1 = 2$. Then $2 - 2 = 0$. The right side is $0$ ✓.

2. What we do Put $x = 1$, $y = 2$ into equation 2, $-x + 2y = 3$.

   How $-x = -1$. $2y = 2 \times 2 = 4$. Then $-1 + 4 = 3$. The right side is $3$ ✓.

   What we have now Both equations hold, so $x = 1,\ y = 2$.

This is the picture you have seen in school. It works well for 2 × 2, but as we will see in section 8 it becomes very hard to use once there are more unknowns.

5. The column picture ★

Now read the same two equations a column at a time. Nothing about the system changes. We only regroup it, pulling out everything that multiplies $x$ and everything that multiplies $y$:

$$ \begin{bmatrix} 2x - y \\ -x + 2y \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} \quad\Longleftrightarrow\quad x \underbrace{\begin{bmatrix} 2 \\ -1 \end{bmatrix}}_{\text{column 1}} + \; y \underbrace{\begin{bmatrix} -1 \\ 2 \end{bmatrix}}_{\text{column 2}} = \underbrace{\begin{bmatrix} 0 \\ 3 \end{bmatrix}}_{b} $$

This is one vector equation instead of two separate equations, and the question changes completely:

Why $x = 1$ and $y = 2$?

These numbers are not picked. The equations force them.

Step 1: Solve the equations.

$$ \begin{aligned} 2x - y &= 0 &&\text{(1)} \\ -x + 2y &= 3 &&\text{(2)} \end{aligned} $$

The plan: use one equation to write $y$ in terms of $x$, put that into the other equation to find $x$, then go back and find $y$.

1. What we do 1a. Use equation (1), $2x - y = 0$, to say what $y$ is in terms of $x$.

   Why Equation (1) has a $0$ on the right and only a plain $-y$, so $y$ is easy to get alone.

   How Add $y$ to both sides. Left: $2x - y + y = 2x$. Right: $0 + y = y$. So $2x = y$, which we read backwards as $y = 2x$.

   $$ \begin{aligned} 2x - y &= 0 \\ 2x &= y &&\text{add } y \text{ to both sides} \\ y &= 2x \end{aligned} $$

   What we have now In any solution, $y$ must be exactly twice $x$.

2. What we do 1b. Put $y = 2x$ into equation (2), $-x + 2y = 3$.

   Why Then equation (2) contains only one unknown, $x$, and one unknown can be solved straight away.

   How, move 1 Replace $y$ by $2x$: $-x + 2(2x) = 3$.

   How, move 2 $2(2x)$ means "2 lots of $2x$" $= 4x$. So $-x + 4x = 3$.

   How, move 3 Minus one $x$ plus four $x$'s leaves three $x$'s: $-1 + 4 = 3$, so $3x = 3$.

   How, move 4 "3 times what is 3?" Divide both sides by 3: $x = 3 \div 3 = 1$.

   $$ \begin{aligned} -x + 2y &= 3 \\ -x + 2(2x) &= 3 &&\text{put } y = 2x \\ -x + 4x &= 3 &&2(2x) = 4x \\ 3x &= 3 &&-x + 4x = 3x \\ x &= 1 &&\text{divide both sides by } 3 \end{aligned} $$

   What we have now $x = 1$.

3. What we do 1c. Find $y$ by putting $x = 1$ back into $y = 2x$.

   Why $y = 2x$ is already "$y$ alone", so this is the quickest way.

   How $y = 2 \times 1 = 2$.

   $$ \begin{aligned} y &= 2x \\ y &= 2(1) &&\text{put } x = 1 \\ y &= 2 \end{aligned} $$

   Another way Use equation (2) directly: $-1 + 2y = 3$. Add 1 to both sides: $2y = 3 + 1 = 4$. Halve both sides: $y = 4 \div 2 = 2$. Same answer.

   $$ \begin{aligned} -x + 2y &= 3 \\ -1 + 2y &= 3 &&\text{put } x = 1 \\ 2y &= 4 &&\text{add } 1 \text{ to both sides} \\ y &= 2 &&\text{divide both sides by } 2 \end{aligned} $$

   What we have now $x = 1$, $y = 2$.

4. What we do 1d. Check both original equations with $x = 1$, $y = 2$.

   Why A small slip in any move would show up here.

   Check (1) $2x - y$: $2 \times 1 = 2$, then $2 - 2 = 0$. Should be $0$ ✓.

   Check (2) $-x + 2y$: $-1$, and $2 \times 2 = 4$, then $-1 + 4 = 3$. Should be $3$ ✓.

   $$ \begin{aligned} \text{(1)}:&\quad 2(1) - 2 = 2 - 2 = 0 \;✓ \\ \text{(2)}:&\quad -1 + 2(2) = -1 + 4 = 3 \;✓ \end{aligned} $$

   What we have now $x = 1$, $y = 2$ is the only pair that satisfies both equations.

Step 2: Put those numbers into the column picture. The column picture says $x\,(\text{column 1}) + y\,(\text{column 2}) = b$:

$$ x \begin{bmatrix} 2 \\ -1 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} $$

1. What we do Scale column 1 by $x = 1$.

   How Top: $1 \times 2 = 2$. Bottom: $1 \times (-1) = -1$.

   What we have now $(2, -1)$: one step along column 1.

2. What we do Scale column 2 by $y = 2$.

   How Top: $2 \times (-1) = -2$. Bottom: $2 \times 2 = 4$.

   What we have now $(-2, 4)$: two steps along column 2.

3. What we do Add the two scaled columns, box by box.

   Why In the column picture the answer is "this much of column 1 plus this much of column 2".

   How Top: $2 + (-2) = 0$. Bottom: $-1 + 4 = 3$.

   $$ 1 \begin{bmatrix} 2 \\ -1 \end{bmatrix} + 2 \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 \\ -1 \end{bmatrix} + \begin{bmatrix} -2 \\ 4 \end{bmatrix} = \begin{bmatrix} 2 + (-2) \\ -1 + 4 \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} = b \;✓ $$

   What we have now $(0, 3)$, which is exactly $b$ ✓.

It lands exactly on $b = (0, 3)$. The same two numbers that satisfy the equations are the amounts that make the columns add up to $b$.

Step 3: What $x$ and $y$ mean in each picture. They are the same two numbers, but they play a different role:

	What $x$ and $y$ are	The question being asked
Row picture	coordinates of a point	Which point lies on both lines?
Column picture	amounts that scale each column vector	How much of each column do I need to reach $b$?

In the column picture the columns are fixed arrows. You are not looking for a location. You are looking for how far to travel along each arrow. As a walk from the origin:

Move	Now at
start	$(0, 0)$
go $1 \times (2, -1)$	$(2, -1)$
go $2 \times (-1, 2)$	$(0, 3) = b$ ✓

That is why $x$ and $y$ are called weights rather than coordinates: $x = 1$ means "one unit of column 1", and $y = 2$ means "two units of column 2".

6. Row vs column: side by side

Both pictures solve the same system and give the same answer $(x, y) = (1, 2)$. They just draw it differently.

	Row picture	Column picture
What is drawn	One line per equation	One arrow per unknown (column), plus $b$
What $(x, y)$ means	A point in the plane	Weights in a combination
Where the answer is	The point where lines meet	The weights that make the arrows reach $b$
If $b$ changes	The lines move	The arrows stay, only $b$ moves
In $n$ dimensions	$n$ hyperplanes meeting: impossible to picture	$n$ vectors combining to $b$: still a clear idea

7. All combinations: which $b$ can we reach?

The question

So far the right-hand side was fixed at $b = (0, 3)$, and we found the one combination that reaches it. Now change the question:

Try some combinations

Column 1 $= (2, -1)$ and column 2 $= (-1, 2)$. Pick a few values of $x$ and $y$ and see where we land:

$x$	$y$	$x(2, -1) + y(-1, 2)$	Lands on
$1$	$0$	$(2, -1) + (0, 0)$	$(2, -1)$
$0$	$1$	$(0, 0) + (-1, 2)$	$(-1, 2)$
$1$	$1$	$(2, -1) + (-1, 2)$	$(1, 1)$
$1$	$2$	$(2, -1) + (-2, 4)$	$(0, 3)$ ← our $b$
$2$	$1$	$(4, -2) + (-1, 2)$	$(3, 0)$
$-1$	$-1$	$(-2, 1) + (1, -2)$	$(-1, -1)$

One row of the table, slowly ($x = 2$, $y = 1$):

1. What we do Scale column 1 by $x = 2$.

   How Top: $2 \times 2 = 4$. Bottom: $2 \times (-1) = -2$. Result $(4, -2)$.

2. What we do Scale column 2 by $y = 1$.

   How Top: $1 \times (-1) = -1$. Bottom: $1 \times 2 = 2$. Result $(-1, 2)$.

3. What we do Add them box by box.

   How Top: $4 + (-1) = 3$. Bottom: $-2 + 2 = 0$.

   What we have now We land on $(3, 0)$, as the table says. Every other row is done the same way.

The results go up, down, left and right. Drawing every whole-number combination gives a slanted grid that keeps going in all directions:

Proof that every $b$ can be reached

Take any target $b = (b_1, b_2)$, where $b_1$ and $b_2$ are any numbers. We need $x$ and $y$ with

$$ x \begin{bmatrix} 2 \\ -1 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} $$

Comparing the top and bottom components gives two equations:

$$ \begin{aligned} 2x - y &= b_1 &&\text{(top)} \\ -x + 2y &= b_2 &&\text{(bottom)} \end{aligned} $$

Solve exactly as in section 5, only with $b_1, b_2$ instead of $0, 3$.

1. What we do Get $y$ alone from the top equation, $2x - y = b_1$.

   Why Same plan as section 5: one unknown in terms of the other, then substitute.

   How, move 1 Add $y$ to both sides: $2x = b_1 + y$.

   How, move 2 Take $b_1$ away from both sides: $2x - b_1 = y$, i.e. $y = 2x - b_1$.

   $$ \begin{aligned} 2x - y &= b_1 \\ 2x &= b_1 + y &&\text{add } y \text{ to both sides} \\ y &= 2x - b_1 &&\text{subtract } b_1 \text{ from both sides} \end{aligned} $$

   What we have now $y = 2x - b_1$. (Check with $b_1 = 0$: $y = 2x$, as in section 5.)

2. What we do Put $y = 2x - b_1$ into the bottom equation, $-x + 2y = b_2$.

   Why Then only $x$ is unknown.

   How, move 1 Replace $y$: $-x + 2(2x - b_1) = b_2$.

   How, move 2 Open the bracket: $2 \times 2x = 4x$ and $2 \times (-b_1) = -2b_1$. So $-x + 4x - 2b_1 = b_2$.

   How, move 3 Collect the $x$'s: $-1 + 4 = 3$, so $3x - 2b_1 = b_2$.

   How, move 4 Add $2b_1$ to both sides: $3x = 2b_1 + b_2$.

   How, move 5 Divide both sides by 3: $x = \frac{2b_1 + b_2}{3}$.

   $$ \begin{aligned} -x + 2y &= b_2 \\ -x + 2(2x - b_1) &= b_2 &&\text{put } y = 2x - b_1 \\ -x + 4x - 2b_1 &= b_2 &&\text{multiply out the bracket} \\ 3x - 2b_1 &= b_2 &&-x + 4x = 3x \\ 3x &= 2b_1 + b_2 &&\text{add } 2b_1 \text{ to both sides} \\ x &= \frac{2b_1 + b_2}{3} &&\text{divide by } 3 \end{aligned} $$

   What we have now A formula for $x$ that works for any target.

3. What we do Put that $x$ back into $y = 2x - b_1$.

   How, move 1 $2 \times \frac{2b_1 + b_2}{3} = \frac{4b_1 + 2b_2}{3}$ (double the top of the fraction).

   How, move 2 To take away $b_1$ from a fraction with bottom 3, write $b_1$ as $\frac{3b_1}{3}$ (three thirds of $b_1$ is $b_1$).

   How, move 3 Subtract the tops: $4b_1 + 2b_2 - 3b_1 = b_1 + 2b_2$. So $y = \frac{b_1 + 2b_2}{3}$.

   $$ \begin{aligned} y &= 2\cdot\frac{2b_1 + b_2}{3} - b_1 \\ y &= \frac{4b_1 + 2b_2}{3} - \frac{3b_1}{3} &&\text{write } b_1 \text{ as } \tfrac{3b_1}{3} \\ y &= \frac{b_1 + 2b_2}{3} &&4b_1 - 3b_1 = b_1 \end{aligned} $$

   What we have now Formulas for both weights, for any $b$.

Test the formula on three targets:

Target $b$	$x = \frac{2b_1 + b_2}{3}$	$y = \frac{b_1 + 2b_2}{3}$	Check $x(2,-1) + y(-1,2)$
$(0, 3)$	$\frac{0 + 3}{3} = 1$	$\frac{0 + 6}{3} = 2$	$(2 - 2,\ -1 + 4) = (0, 3)$ ✓
$(3, 0)$	$\frac{6 + 0}{3} = 2$	$\frac{3 + 0}{3} = 1$	$(4 - 1,\ -2 + 2) = (3, 0)$ ✓
$(5, 7)$	$\frac{10 + 7}{3} = \frac{17}{3}$	$\frac{5 + 14}{3} = \frac{19}{3}$	$(\frac{34 - 19}{3},\ \frac{-17 + 38}{3}) = (5, 7)$ ✓

The hardest row, $b = (5, 7)$, slowly:

1. What we do Work out $x = \frac{2b_1 + b_2}{3}$ with $b_1 = 5$, $b_2 = 7$.

   How $2 \times 5 = 10$. Then $10 + 7 = 17$. So $x = \frac{17}{3}$.

2. What we do Work out $y = \frac{b_1 + 2b_2}{3}$.

   How $2 \times 7 = 14$. Then $5 + 14 = 19$. So $y = \frac{19}{3}$.

3. What we do Check the top component of $x(2, -1) + y(-1, 2)$.

   How $2 \times \frac{17}{3} = \frac{34}{3}$. $-1 \times \frac{19}{3} = -\frac{19}{3}$. Add: $\frac{34 - 19}{3} = \frac{15}{3} = 5$. Should be $b_1 = 5$ ✓.

4. What we do Check the bottom component.

   How $-1 \times \frac{17}{3} = -\frac{17}{3}$. $2 \times \frac{19}{3} = \frac{38}{3}$. Add: $\frac{-17 + 38}{3} = \frac{21}{3} = 7$. Should be $b_2 = 7$ ✓.

   What we have now Even a messy target is reached exactly.

Why it works: the columns point in different directions

Column 1 and column 2 are not on the same line. Two arrows in different directions work like "east" and "north" on a map: by walking some amount along each, you can reach any spot.

Name for it: span

From now on, every system comes with two questions:

Question	In symbols
1. Which combination gives this particular $b$?	Solve $Ax = b$.
2. Which $b$ can be reached at all?	What is the span of the columns?

8. Three equations, three unknowns

$$ \begin{aligned} 2x - y \phantom{{}+2y-z} &= 0 &&\text{(1)} \\ -x + 2y - z &= -1 &&\text{(2)} \\ -3y + 4z &= 4 &&\text{(3)} \end{aligned} \qquad\Longleftrightarrow\qquad \underbrace{\begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -3 & 4 \end{bmatrix}}_{A} \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \underbrace{\begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}}_{b} $$

Now $A$ is $3 \times 3$: three equations (rows) and three unknowns (columns).

8.1 Solve it step by step

The plan: equation (1) has only $x$ and $y$, so it gives $y$ in terms of $x$. Then (2) gives $z$ in terms of $x$. Then (3) has only one unknown left, $x$.

1. What we do Step a. Get $y$ alone from equation (1), $2x - y = 0$.

   Why Equation (1) has only two unknowns and a $0$ on the right: the easiest one to start with.

   How Add $y$ to both sides: $2x - y + y = 0 + y$, so $2x = y$.

   $$ \begin{aligned} 2x - y &= 0 \\ 2x &= y &&\text{add } y \text{ to both sides} \\ y &= 2x \end{aligned} $$

   What we have now $y = 2x$.

2. What we do Step b. Put $y = 2x$ into equation (2), $-x + 2y - z = -1$, and get $z$ alone.

   Why After replacing $y$, equation (2) only has $x$ and $z$, so it tells us $z$ in terms of $x$.

   How, move 1 Replace $y$: $-x + 2(2x) - z = -1$.

   How, move 2 $2(2x) = 4x$: $-x + 4x - z = -1$.

   How, move 3 $-1 + 4 = 3$ lots of $x$: $3x - z = -1$.

   How, move 4 Add $z$ to both sides: $3x = -1 + z$.

   How, move 5 Add $1$ to both sides: $3x + 1 = z$.

   $$ \begin{aligned} -x + 2y - z &= -1 \\ -x + 2(2x) - z &= -1 &&\text{put } y = 2x \\ -x + 4x - z &= -1 &&2(2x) = 4x \\ 3x - z &= -1 &&-x + 4x = 3x \\ 3x + 1 &= z &&\text{add } z \text{ and } 1 \text{ to both sides} \\ z &= 3x + 1 \end{aligned} $$

   What we have now $y = 2x$ and $z = 3x + 1$. Everything depends only on $x$.

3. What we do Step c. Put both into equation (3), $-3y + 4z = 4$.

   Why Then equation (3) has just one unknown, $x$.

   How, move 1 Replace: $-3(2x) + 4(3x + 1) = 4$.

   How, move 2 Open the brackets: $-3 \times 2x = -6x$; $4 \times 3x = 12x$; $4 \times 1 = 4$. So $-6x + 12x + 4 = 4$.

   How, move 3 $-6 + 12 = 6$: $6x + 4 = 4$.

   How, move 4 Take 4 from both sides: $6x = 4 - 4 = 0$.

   How, move 5 "6 times what is 0?" Only $0$: $x = 0 \div 6 = 0$.

   $$ \begin{aligned} -3y + 4z &= 4 \\ -3(2x) + 4(3x + 1) &= 4 &&\text{put } y = 2x,\ z = 3x + 1 \\ -6x + 12x + 4 &= 4 &&\text{multiply out the brackets} \\ 6x + 4 &= 4 &&-6x + 12x = 6x \\ 6x &= 0 &&\text{subtract } 4 \text{ from both sides} \\ x &= 0 &&\text{divide by } 6 \end{aligned} $$

   What we have now $x = 0$.

4. What we do Step d. Put $x = 0$ back into $y = 2x$ and $z = 3x + 1$.

   How $y = 2 \times 0 = 0$. $z = 3 \times 0 + 1 = 0 + 1 = 1$.

   $$ \begin{aligned} y &= 2x = 2(0) = 0 \\ z &= 3x + 1 = 3(0) + 1 = 1 \end{aligned} $$

   What we have now $(x, y, z) = (0, 0, 1)$.

5. What we do Step e. Check all three original equations with $(0, 0, 1)$.

   Check (1) $2x - y = 2 \times 0 - 0 = 0 - 0 = 0$. Should be $0$ ✓.

   Check (2) $-x + 2y - z = -0 + 2 \times 0 - 1 = 0 + 0 - 1 = -1$. Should be $-1$ ✓.

   Check (3) $-3y + 4z = -3 \times 0 + 4 \times 1 = 0 + 4 = 4$. Should be $4$ ✓.

   $$ \begin{aligned} \text{(1)}:&\quad 2(0) - 0 = 0 \;✓ \\ \text{(2)}:&\quad -0 + 2(0) - 1 = -1 \;✓ \\ \text{(3)}:&\quad -3(0) + 4(1) = 4 \;✓ \end{aligned} $$

8.2 Row picture: three planes

With three unknowns, each equation is a flat plane in 3D space.

Plane	Equation	Through the origin?	Shape
1	$2x - y = 0$	Yes (right side is $0$)	No $z$ in it, so $z$ can be anything. The plane is "vertical" and contains the whole $z$-axis.
2	$-x + 2y - z = -1$	No ($0 \ne -1$)	A tilted plane. Three of its points are $(1,0,0)$, $(0,0,1)$ and $(0,-\tfrac12,0)$.
3	$-3y + 4z = 4$	No ($0 \ne 4$)	No $x$ in it, so $x$ can be anything. The plane runs parallel to the $x$-axis.

The steps in 8.1 have a picture:

1. Steps a and b = planes 1 and 2 meet in a line. Points on both planes satisfy $y = 2x$ and $z = 3x + 1$. So their common line is all points $(x,\ 2x,\ 3x + 1)$, for example $(0, 0, 1)$ and $(1, 2, 4)$.
2. Step c = plane 3 cuts that line in one point. Only $x = 0$ puts the line on plane 3.
3. That point is the solution $(0, 0, 1)$: the only point on all three planes.

8.3 Column picture: three vectors in 3D

Group the same system by columns:

$$ x \underbrace{\begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix}}_{\text{col 1}} + y \underbrace{\begin{bmatrix} -1 \\ 2 \\ -3 \end{bmatrix}}_{\text{col 2}} + z \underbrace{\begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}}_{\text{col 3}} = \underbrace{\begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}}_{b} $$

The question: how much of each column do we need to build $b$?

Compare $b$ with the columns: $b = (0, -1, 4)$ is exactly column 3. So we need none of column 1, none of column 2 and one of column 3:

1. What we do Compare $b$ with each column, number by number.

   Why If $b$ equals one column, the recipe is "1 of that column, 0 of the rest", with no solving needed.

   How Column 3 is $(0, -1, 4)$ and $b$ is $(0, -1, 4)$: top $0 = 0$, middle $-1 = -1$, bottom $4 = 4$.

   What we have now Guess: $x = 0$, $y = 0$, $z = 1$.

2. What we do Build $0(\text{col 1}) + 0(\text{col 2}) + 1(\text{col 3})$, row by row.

   How Top: $0 \times 2 + 0 \times (-1) + 1 \times 0 = 0 + 0 + 0 = 0$. Middle: $0 \times (-1) + 0 \times 2 + 1 \times (-1) = 0 + 0 - 1 = -1$. Bottom: $0 \times 0 + 0 \times (-3) + 1 \times 4 = 0 + 0 + 4 = 4$.

   $$ 0 \begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix} + 0 \begin{bmatrix} -1 \\ 2 \\ -3 \end{bmatrix} + 1 \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix} = \begin{bmatrix} 0 + 0 + 0 \\ 0 + 0 - 1 \\ 0 + 0 + 4 \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix} = b \;✓ $$

   What we have now $(0, -1, 4) = b$ ✓.

So $(x, y, z) = (0, 0, 1)$: the same answer as 8.1, found in one look.

8.4 Change only the right-hand side

Keep the same $A$, but choose a new $b$ equal to column 1 + column 2:

1. What we do Build the new $b$ = column 1 + column 2.

   Why If we build $b$ ourselves from the columns, we already know the recipe.

   How Top: $2 + (-1) = 2 - 1 = 1$. Middle: $-1 + 2 = 1$. Bottom: $0 + (-3) = 0 - 3 = -3$.

   $$ b = \begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix} + \begin{bmatrix} -1 \\ 2 \\ -3 \end{bmatrix} = \begin{bmatrix} 2 - 1 \\ -1 + 2 \\ 0 - 3 \end{bmatrix} = \begin{bmatrix} 1 \\ 1 \\ -3 \end{bmatrix} $$

   What we have now $b = (1, 1, -3)$, made of one column 1 and one column 2, so the weights are $x = 1$, $y = 1$, $z = 0$.

2. What we do Check in the three original equations (now with right-hand sides $1, 1, -3$).

   Check (1) $2x - y = 2 \times 1 - 1 = 2 - 1 = 1$. Should be $1$ ✓.

   Check (2) $-x + 2y - z = -1 + 2 \times 1 - 0 = -1 + 2 - 0 = 1$. Should be $1$ ✓.

   Check (3) $-3y + 4z = -3 \times 1 + 4 \times 0 = -3 + 0 = -3$. Should be $-3$ ✓.

   $$ \begin{aligned} \text{(1)}:&\quad 2(1) - 1 = 1 \;✓ \\ \text{(2)}:&\quad -1 + 2(1) - 0 = 1 \;✓ \\ \text{(3)}:&\quad -3(1) + 4(0) = -3 \;✓ \end{aligned} $$

	What changes when $b$ changes
Row picture	All three planes move (their right-hand sides changed) and now meet at a new point $(1, 1, 0)$.
Column picture	The three columns do not move at all. Only the mix changes, from $(0, 0, 1)$ to $(1, 1, 0)$.

Here $b$ was chosen so the answer could be spotted. In general it cannot be, and substitution gets messy with many unknowns. The systematic method is elimination, covered in Lecture 03.

9. When it goes wrong: singular matrices

For the 3 × 3 matrix above, the answer is yes. Strang calls it a good matrix: non-singular, or invertible.

What could go wrong?

Suppose the three columns all lie in the same plane. Stretching and adding vectors that lie in a plane can never leave that plane. So every combination stays in it, and any $b$ outside the plane is unreachable.

For example, if column 3 = column 1 + column 2, then column 3 gives nothing new. It points in a direction we could already reach, so three columns do the job of only two.

The three possible outcomes (2 × 2)

10. Thinking in $n$ dimensions

Now imagine 9 equations in 9 unknowns. There are nine columns, and each is a vector with 9 components, so it lives in 9-dimensional space $\mathbb{R}^9$. Nobody can picture that, but the column question is exactly the same: which combination of the nine columns gives $b$, and do their combinations fill all of $\mathbb{R}^9$?

• A random 9 × 9 matrix is almost certainly non-singular. Its nine columns point in "genuinely different" directions and fill the whole space.
• If the 9th column equals the 8th, it adds nothing new. The combinations then fill only an 8-dimensional flat piece (a "plane") inside 9-dimensional space, and every $b$ outside it is unreachable.

11. Matrix × vector, two ways

The matrix form $Ax = b$ contains a multiplication: a matrix times a vector. Take

$$ A = \begin{bmatrix} 2 & 5 \\ 1 & 3 \end{bmatrix}, \qquad x = \begin{bmatrix} 1 \\ 2 \end{bmatrix}. $$

Way 1: by columns (Strang's favourite)

$Ax$ is a combination of the columns of $A$, with the entries of $x$ as the weights:

1. What we do Take $x_1 = 1$ copy of column 1, $(2, 1)$.

   Why The first entry of $x$ says how much of the first column to use.

   How Top: $1 \times 2 = 2$. Bottom: $1 \times 1 = 1$.

   What we have now $(2, 1)$.

2. What we do Take $x_2 = 2$ copies of column 2, $(5, 3)$.

   How Top: $2 \times 5 = 10$. Bottom: $2 \times 3 = 6$.

   What we have now $(10, 6)$.

3. What we do Add the two pieces, box by box.

   How Top: $2 + 10 = 12$. Bottom: $1 + 6 = 7$.

   $$ Ax = 1 \begin{bmatrix} 2 \\ 1 \end{bmatrix} + 2 \begin{bmatrix} 5 \\ 3 \end{bmatrix} = \begin{bmatrix} 2 + 10 \\ 1 + 6 \end{bmatrix} = \begin{bmatrix} 12 \\ 7 \end{bmatrix} $$

   What we have now $Ax = (12, 7)$.

Way 2: by rows (dot products)

Each entry of $Ax$ is one row of $A$ "dotted" with $x$: multiply matching entries and add.

1. What we do Top entry = row 1 $(2, 5)$ dotted with $x = (1, 2)$.

   Why Row 1 is equation 1: it tells what the first answer number is made of.

   How First with first: $2 \times 1 = 2$. Second with second: $5 \times 2 = 10$. Add: $2 + 10 = 12$.

   What we have now Top entry $= 12$.

2. What we do Bottom entry = row 2 $(1, 3)$ dotted with $x = (1, 2)$.

   How $1 \times 1 = 1$. $3 \times 2 = 6$. Add: $1 + 6 = 7$.

   $$ Ax = \begin{bmatrix} \text{row}_1 \cdot x \\ \text{row}_2 \cdot x \end{bmatrix} = \begin{bmatrix} 2(1) + 5(2) \\ 1(1) + 3(2) \end{bmatrix} = \begin{bmatrix} 12 \\ 7 \end{bmatrix} $$

   What we have now $Ax = (12, 7)$, the same as Way 1 ✓.

Look at the middle steps: both ways do exactly the same four multiplications ($2 \cdot 1$, $5 \cdot 2$, $1 \cdot 1$, $3 \cdot 2$). They only group them differently, so they must give the same answer.

	By columns	By rows
Thinks of $Ax$ as	a combination of columns	a list of dot products
Matches	the column picture	the row picture
Best for	understanding: what can $Ax$ be?	computing one entry quickly

12. Vocabulary and summary

Linear equation

Unknowns only multiplied by constants and added. Its graph is a line (2D), plane (3D) or hyperplane.

Coefficient matrix $A$

The numbers multiplying the unknowns. Row $i$ is equation $i$, and column $j$ belongs to unknown $j$.

Vector

An ordered list of numbers, drawn as an arrow from the origin.

Linear combination

$c_1v_1 + \dots + c_nv_n$: scale vectors and add them. The most important operation in the course.

Span

The set of all linear combinations of some vectors.

Row picture

Each equation is a line or plane, and the solution is where they all meet.

Column picture

The solution is the set of weights that combine the columns into $b$.

Non-singular (invertible)

The columns fill the whole space, so there is exactly one solution for every $b$.

Singular

The columns fill less than the whole space, so there is no solution for most $b$ and infinitely many for the rest.

Worked examples

Each example is solved step by step. Try it yourself first, then read the solution.

Practice problems

Answers are hidden. Click a problem to check your work.

1. In the lecture's 2 × 2 example, which line goes through the origin, and why?

$2x - y = 0$, because its right-hand side is $0$, so $(0,0)$ satisfies it. The line $-x + 2y = 3$ does not, since $(0,0)$ gives $0 \ne 3$.

2. For the lecture's 3 × 3 matrix $A$, solve $Ax = (2, -1, 0)$ without any computation.

$(2, -1, 0)$ is column 1, so $x = (1, 0, 0)$.

3. Columns $(1,0,1)$, $(0,1,1)$, $(1,1,2)$: can their combinations reach every $b$ in 3D?

No. Column 3 = column 1 + column 2, so all three columns lie in one plane. The matrix is singular, and only $b$ in that plane can be reached.

4. Compute $\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}\begin{bmatrix} 2 \\ -1 \end{bmatrix}$ by columns.

$2(1, 3) - 1(2, 4) = (2 - 2,\; 6 - 4) = (0, 2)$.

1. What we do Take 2 copies of column 1, $(1, 3)$.

   How $(2 \times 1,\ 2 \times 3) = (2, 6)$.

2. What we do Take $-1$ copy of column 2, $(2, 4)$.

   How $(-1 \times 2,\ -1 \times 4) = (-2, -4)$.

3. What we do Add box by box.

   How Top: $2 + (-2) = 0$. Bottom: $6 + (-4) = 2$.

   Check by rows Row 1: $1 \times 2 + 2 \times (-1) = 2 - 2 = 0$ ✓. Row 2: $3 \times 2 + 4 \times (-1) = 6 - 4 = 2$ ✓.

5. For which $c$ is $\begin{bmatrix} 1 & 2 \\ 3 & c \end{bmatrix}$ singular?

It is singular when column 2 is a multiple of column 1: $(2, c) = 2(1, 3)$, so $c = 6$.

6. Write $3x - y = 5,\; x + 4y = 6$ in column form and solve it.

$x(3, 1) + y(-1, 4) = (5, 6)$. From the second equation $x = 6 - 4y$. Substituting gives $18 - 13y = 5$, so $y = 1$ and $x = 2$. Check: $2(3,1) + 1(-1,4) = (5, 6)$ ✓.

1. What we do Get $x$ alone from $x + 4y = 6$.

   How Take $4y$ from both sides: $x = 6 - 4y$.

2. What we do Put it into $3x - y = 5$.

   How, move 1 $3(6 - 4y) - y = 5$.

   How, move 2 Open the bracket: $3 \times 6 = 18$, $3 \times (-4y) = -12y$. So $18 - 12y - y = 5$.

   How, move 3 $-12y - y = -13y$: $18 - 13y = 5$.

   How, move 4 Take 18 from both sides: $-13y = 5 - 18 = -13$.

   How, move 5 Divide by $-13$: $y = 1$.

3. What we do Put $y = 1$ into $x = 6 - 4y$.

   How $4 \times 1 = 4$, then $6 - 4 = 2$. So $x = 2$.

4. Check $3x - y = 6 - 1 = 5$ ✓. $x + 4y = 2 + 4 = 6$ ✓.