The document aims to provide a solid theoretical foundation in vector spaces, which are crucial for understanding many areas of mathematics, physics, engineering, and computer science. It emphasizes not only the definitions and theorems but also provides numerous examples to illustrate how these abstract concepts apply to concrete mathematical objects like vectors in $\mathbb{R}^n$, polynomials, and matrices. The progression of topics is logical, starting with basic building blocks and gradually introducing more complex interrelationships between different concepts within vector spaces.

The document begins by defining a vector space as a non-empty set $V$ of objects, called vectors, on which two operations, vector addition and scalar multiplication, are defined. These operations must satisfy ten specific axioms for all vectors $\mathbf{u}, \mathbf{v}, \mathbf{w}$ in $V$ and all scalars $c, d$ in a field (typically real numbers $\mathbb{R}$).

1. Closure under addition: If $\mathbf{u}, \mathbf{v} \in V$, then $\mathbf{u} + \mathbf{v} \in V$.

2. Commutativity of addition: $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$.

3. Associativity of addition: $(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$.

4. Existence of zero vector: There exists a zero vector $\mathbf{0} \in V$ such that $\mathbf{u} + \mathbf{0} = \mathbf{u}$ for all $\mathbf{u} \in V$.

5. Existence of negative vector: For each $\mathbf{u} \in V$, there exists a vector $-\mathbf{u} \in V$ such that $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$.

6. Closure under scalar multiplication: If $\mathbf{u} \in V$ and $c$ is a scalar, then $c\mathbf{u} \in V$.

7. Distributivity over vector addition: $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$.

8. Distributivity over scalar addition: $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$.

9. Associativity of scalar multiplication: $c(d\mathbf{u}) = (cd)\mathbf{u}$.

10. Identity for scalar multiplication: $1\mathbf{u} = \mathbf{u}$.

The document provides examples of vector spaces, such as $\mathbb{R}^n$, the set of all $m \times n$ matrices ($M_{m \times n}$), and the set of all polynomials of degree at most $n$ ($P_n$). It also shows examples of sets that are not vector spaces by demonstrating the failure of one or more axioms.

A subspace $W$ of a vector space $V$ is a subset of $V$ that is itself a vector space under the same operations defined on $V$. To prove that a non-empty subset $W$ is a subspace, one only needs to check three conditions, as the other axioms are inherited from $V$:

1. The zero vector of $V$ is in $W$ ($\mathbf{0} \in W$).

2. $W$ is closed under vector addition: If $\mathbf{u}, \mathbf{v} \in W$, then $\mathbf{u} + \mathbf{v} \in W$.

3. $W$ is closed under scalar multiplication: If $\mathbf{u} \in W$ and $c$ is any scalar, then $c\mathbf{u} \in W$.

Examples include lines through the origin in $\mathbb{R}^2$ or $\mathbb{R}^3$, planes through the origin in $\mathbb{R}^3$, and the set of all polynomials of degree at most $n$ ($P_n$) as a subspace of the set of all polynomials ($P$).

A vector $\mathbf{w}$ is a linear combination of vectors $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$ if it can be expressed in the form:

$\mathbf{w} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k$

where $c_1, c_2, \dots, c_k$ are scalars.

The span of a set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$, denoted as $\text{span}(S)$ or $\text{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\}$, is the set of all possible linear combinations of these vectors. The document states that the span of any non-empty set of vectors in a vector space $V$ is always a subspace of $V$.

A set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $V$ is said to be linearly dependent if there exist scalars $c_1, c_2, \dots, c_k$, not all zero, such that:

$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k = \mathbf{0}$

If the only solution to this equation is $c_1 = c_2 = \dots = c_k = 0$, then the set of vectors is linearly independent.

A set of vectors $B = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_n\}$ is a basis for a vector space $V$ if two conditions are met:

1. $B$ is linearly independent.

2. $B$ spans $V$ (i.e., $\text{span}(B) = V$).

The dimension of a vector space $V$, denoted $\text{dim}(V)$, is the number of vectors in any basis for $V$. If $V$ consists only of the zero vector, its dimension is 0. If a vector space cannot be spanned by a finite set of vectors, it is called infinite-dimensional. The document highlights that all bases for a given vector space have the same number of vectors.

* Standard basis for $\mathbb{R}^n$: $\{\mathbf{e}_1, \mathbf{e}_2, \dots, \mathbf{e}_n\}$ where $\mathbf{e}_i$ has a 1 in the $i$-th position and 0s elsewhere. $\text{dim}(\mathbb{R}^n) = n$.

* Standard basis for $P_n$: $\{1, x, x^2, \dots, x^n\}$. $\text{dim}(P_n) = n+1$.

* Standard basis for $M_{m \times n}$: The set of $m \times n$ matrices with a single 1 and all other entries 0. $\text{dim}(M_{m \times n}) = mn$.

If $B = \{\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_n\}$ is a basis for a vector space $V$, then for every vector $\mathbf{x} \in V$, there exists a unique set of scalars $c_1, c_2, \dots, c_n$ such that:

$\mathbf{x} = c_1\mathbf{b}_1 + c_2\mathbf{b}_2 + \dots + c_n\mathbf{b}_n$

These scalars are called the coordinates of $\mathbf{x}$ relative to the basis $B$, and the coordinate vector of $\mathbf{x}$ relative to $B$ is denoted as:

$[\mathbf{x}]_B = \begin{bmatrix} c_1 \\ c_2 \\ \vdots \\ c_n \end{bmatrix}$

The document also introduces the concept of change of basis. If $B = \{\mathbf{b}_1, \dots, \mathbf{b}_n\}$ and $C = \{\mathbf{c}_1, \dots, \mathbf{c}_n\}$ are two bases for $V$, then there exists a unique $n \times n$ matrix $P_{C \leftarrow B}$, called the change-of-basis matrix from $B$ to $C$, such that:

$[\mathbf{x}]_C = P_{C \leftarrow B} [\mathbf{x}]_B$

The columns of $P_{C \leftarrow B}$ are the coordinate vectors of the basis vectors in $B$ relative to basis $C$:

$P_{C \leftarrow B} = \begin{bmatrix} [\mathbf{b}_1]_C & [\mathbf{b}_2]_C & \dots & [\mathbf{b}_n]_C \end{bmatrix}$

The inverse matrix $P_{B \leftarrow C}$ allows conversion from $C$ to $B$: $P_{B \leftarrow C} = (P_{C \leftarrow B})^{-1}$.

For an $m \times n$ matrix $A$:

* The row space of $A$, denoted $\text{Row}(A)$, is the subspace of $\mathbb{R}^n$ spanned by the row vectors of $A$.

* The column space of $A$, denoted $\text{Col}(A)$, is the subspace of $\mathbb{R}^m$ spanned by the column vectors of $A$. This is equivalent to the set of all vectors $\mathbf{b}$ for which $A\mathbf{x} = \mathbf{b}$ has a solution.

* The null space of $A$, denoted $\text{Null}(A)$, is the set of all solutions to the homogeneous equation $A\mathbf{x} = \mathbf{0}$. It is a subspace of $\mathbb{R}^n$.

* Row Space: The non-zero rows of the row echelon form (or reduced row echelon form) of $A$ form a basis for $\text{Row}(A)$.

* Null Space: Solve $A\mathbf{x} = \mathbf{0}$ and express the solution in parametric vector form. The vectors in this form constitute a basis for $\text{Null}(A)$.

* The rank of a matrix $A$, denoted $\text{rank}(A)$, is the dimension of its column space (which is equal to the dimension of its row space). It is also the number of pivot positions in the row echelon form of $A$.

* The nullity of a matrix $A$, denoted $\text{nullity}(A)$, is the dimension of its null space. It is equal to the number of free variables in the solution to $A\mathbf{x} = \mathbf{0}$.

$\text{rank}(A) + \text{nullity}(A) = n$

* Vector Space: A non-empty set $V$ of objects (vectors) equipped with two operations (vector addition and scalar multiplication) that satisfy ten specific axioms.

* Subspace: A non-empty subset $W$ of a vector space $V$ that is itself a vector space under the same operations as $V$. It must contain the zero vector and be closed under addition and scalar multiplication.

* Linear Combination: A vector $\mathbf{w}$ expressed as a sum of scalar multiples of other vectors: $\mathbf{w} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k$.

* Linearly Dependent Set: A set of vectors where at least one vector can be written as a linear combination of the others, or equivalently, where the homogeneous equation $c_1\mathbf{v}_1 + \dots + c_k\mathbf{v}_k = \mathbf{0}$ has a non-trivial solution (not all $c_i$ are zero).

* Linearly Independent Set: A set of vectors where none of the vectors can be written as a linear combination of the others, or equivalently, where the only solution to $c_1\mathbf{v}_1 + \dots + c_k\mathbf{v}_k = \mathbf{0}$ is the trivial solution ($c_1 = \dots = c_k = 0$).

* Basis (of a Vector Space): A set of vectors in a vector space $V$ that is both linearly independent and spans $V$.

* Coordinate Vector: If $B = \{\mathbf{b}_1, \dots, \mathbf{b}_n\}$ is a basis for $V$ and $\mathbf{x} = c_1\mathbf{b}_1 + \dots + c_n\mathbf{b}_n$, then the coordinate vector of $\mathbf{x}$ relative to $B$ is $[\mathbf{x}]_B = \begin{bmatrix} c_1 \\ \vdots \\ c_n \end{bmatrix}$.

* Change-of-Basis Matrix: A matrix $P_{C \leftarrow B}$ that transforms coordinate vectors from basis $B$ to basis $C$, such that $[\mathbf{x}]_C = P_{C \leftarrow B} [\mathbf{x}]_B$.

* Row Space of a Matrix ($\text{Row}(A)$): The subspace spanned by the row vectors of matrix $A$.

* Column Space of a Matrix ($\text{Col}(A)$): The subspace spanned by the column vectors of matrix $A$.

* Null Space of a Matrix ($\text{Null}(A)$): The set of all solutions to the homogeneous equation $A\mathbf{x} = \mathbf{0}$.

* Rank of a Matrix ($\text{rank}(A)$): The dimension of the column space (or row space) of matrix $A$.

* Nullity of a Matrix ($\text{nullity}(A)$): The dimension of the null space of matrix $A$.

The document often uses $\mathbb{R}^n$ as a primary example to illustrate the vector space axioms. It also presents non-examples, such as the set of all $2 \times 2$ matrices with non-negative entries, demonstrating how it fails closure under scalar multiplication (e.g., multiplying by -1). This helps students understand the necessity of each axiom.

Consider the set $W$ of all vectors in $\mathbb{R}^3$ of the form $(a, b, a+b)$. To show $W$ is a subspace of $\mathbb{R}^3$:

1. Zero vector: $(0, 0, 0+0) = (0,0,0) \in W$.

2. Closure under addition: Let $\mathbf{u} = (a_1, b_1, a_1+b_1)$ and $\mathbf{v} = (a_2, b_2, a_2+b_2)$ be in $W$.

$\mathbf{u} + \mathbf{v} = (a_1+a_2, b_1+b_2, (a_1+b_1)+(a_2+b_2)) = (a_1+a_2, b_1+b_2, (a_1+a_2)+(b_1+b_2))$. This is of the form $(x, y, x+y)$, so $\mathbf{u} + \mathbf{v} \in W$.

3. Closure under scalar multiplication: Let $\mathbf{u} = (a, b, a+b) \in W$ and $c$ be a scalar.

$c\mathbf{u} = (ca, cb, c(a+b)) = (ca, cb, ca+cb)$. This is of the form $(x, y, x+y)$, so $c\mathbf{u} \in W$.

Since all three conditions are met, $W$ is a subspace of $\mathbb{R}^3$.

Given vectors $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$, $\mathbf{v}_2 = \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}$, $\mathbf{v}_3 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$. To check for linear independence, form the equation $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + c_3\mathbf{v}_3 = \mathbf{0}$, which leads to a matrix equation $A\mathbf{c} = \mathbf{0}$:

$\begin{bmatrix} 1 & 4 & 2 \\ 2 & 5 & 1 \\ 3 & 6 & 0 \end{bmatrix} \begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}$

Row reducing the augmented matrix $[A | \mathbf{0}]$ will reveal if there are non-trivial solutions. If the RREF has only trivial solutions, the vectors are linearly independent. If there are free variables, they are linearly dependent.

For a matrix $A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 2 & 4 & 6 & 8 \\ 3 & 6 & 9 & 12 \end{bmatrix}$, find a basis for $\text{Null}(A)$.

First, solve $A\mathbf{x} = \mathbf{0}$ by row reducing $A$:

$\begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

The equation becomes $x_1 + 2x_2 + 3x_3 + 4x_4 = 0$.

Here, $x_1$ is basic, and $x_2, x_3, x_4$ are free variables.

$x_1 = -2x_2 - 3x_3 - 4x_4$.

The general solution is $\mathbf{x} = \begin{bmatrix} -2x_2 - 3x_3 - 4x_4 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix} = x_2 \begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix} + x_3 \begin{bmatrix} -3 \\ 0 \\ 1 \\ 0 \end{bmatrix} + x_4 \begin{bmatrix} -4 \\ 0 \\ 0 \\ 1 \end{bmatrix}$.

A basis for $\text{Null}(A)$ is $\left\{ \begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} -3 \\ 0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -4 \\ 0 \\ 0 \\ 1 \end{bmatrix} \right\}$.

The dimension of $\text{Null}(A)$ (nullity) is 3. The rank of $A$ is 1 (one pivot). $\text{rank}(A) + \text{nullity}(A) = 1 + 3 = 4$, which is the number of columns of $A$, verifying the Rank-Nullity Theorem.

The journey begins with the formal definition of a vector space, which is a set of "vectors" equipped with two operations – vector addition and scalar multiplication – that must satisfy ten specific axioms. These axioms ensure that the operations behave in a predictable and consistent manner, generalizing the familiar properties of vectors in $\mathbb{R}^n$. The document clarifies this by providing examples of sets that are vector spaces (like $\mathbb{R}^n$, polynomial spaces $P_n$, and matrix spaces $M_{m \times n}$) and those that are not, illustrating the importance of each axiom.

Finally, the document applies these abstract concepts to matrices, introducing the three fundamental subspaces associated with an $m \times n$ matrix $A$: the row space (spanned by rows), the column space (spanned by columns), and the null space (solutions to $A\mathbf{x} = \mathbf{0}$). It provides practical methods for finding bases for each of these subspaces, typically involving row reduction of the matrix. The dimensions of these subspaces are defined as the rank (dimension of column/row space) and nullity (dimension of null space) of the matrix. The document concludes with the powerful Rank-Nullity Theorem, which states that for any matrix $A$, the sum of its rank and nullity equals the number of columns, establishing a fundamental relationship between the solution space and the image space of a linear transformation represented by the matrix.