Jump to content

Introductory Linear Algebra/Matrix inverses and determinants

From Wikibooks, open books for an open world

Matrix inverses

[edit | edit source]

Matrix inverses are analogous to the multiplicative inverse (or reciprocal) in the number system.

Definition. (Matrix inverses) An matrix is invertible (or non-singular) if there exists an matrix such that The matrix is the inverse of , and is usually denoted by . A matrix that has no inverse is non-invertible (or singular).

Remark.

  • by the invertible matrix theorem (proof of its complete version is complicated, and so is skipped), if one of and holds, then the other also holds

In the number system, the multiplicative inverse, if it exists, is unique. Indeed, the matrix inverse, if it exists, is also unique similarly, as shown in the following proposition.

Proposition. (Uniqueness of matrix inverse) Matrix inverse, if it exists, is unique.

Proof. Suppose to the contrary, that distinct matrices and are both inverses of matrix . Then, by definition of matrix inverse. If the matrix inverse of exists, we have which causes a contradiction.

Example. (Invertible matrix) The matrix is invertible and its inverse is since (it implies that the matrix product in another order is also by the invertible matrix theorem)

Clipboard

Exercise.

Is invertible?

yes
no



Example. (Non-invertible matrix) The matrix is non-invertible.

Proof. Suppose to the contrary, that the matrix is invertible, i.e. there exists a matrix such that But, this equality is equivalent to which is impossible, causing a contradiction.


Clipboard

Exercise.

1 Choose all correct statements.

if matrices and are invertible, is also invertible
if matrices and are non-invertible, is also non-invertible
if matrices and are invertible, and are also invertible
if matrices and are non-invertible, is also non-invertible

2 Choose all correct statements.

since , the inverse of is
let be a matrix.
if matrix is invertible, for each matrix with the same size as


Proposition. (Properties of matrix inverse) Let and be invertible matrices of the same size, and let be a nonzero scalar. Then,

  • (self-invertibility) is invertible and
  • (scalar multiplicativity) is invertible and
  • ('reverse multiplicativity') is invertible and
  • (interchangibility of inverse and transpose) is invertible

Proof.

  • (self-invertibility) since is invertible, , and thus is invertible, and its inverse is
  • (scalar multiplicativity) , as desired
  • ('reverse multiplicativity') , as desired
  • (interchangibility of inverse and transpose) , as desired

Remark.

  • inductively, we can have general 'reverse multiplicativity': is invertible and

Matrix inverse can be used to solve SLE, as follows:

Proposition. Let be a SLE in which is an invertible matrix. Then, the SLE has a unique solution given by .

Proof.

Then, we will define the elementary matrix, which is closely related to EROs, and is important for the proof of results related to EROs.

Definition. (Elementary matrix) Let be a positive integer. There are three types of elementary matrices. An elementary matrix of type I,II or III is a matrix obtained by a type I,II or III ERO on the identity matrix respectively.

Remark.

  • if a matrix needs to be obtained by performing two or more EROs on the identity matrix, it is not an elementary matrix

Example. The matrix is an elementary matrix of type I, since it can be obtained by performing the ERO on , the matrix is an elementary matrix of type II, since it can be obtained by performing the ERO on , and the matrix is an elementary matrix of type III, since it can be obtained by performing the ERO on .

The matrix is not an elementary matrix, since it needs to be obtained by performing at least two EROs on , e.g. , in this order

Clipboard

Exercise.

Choose correct statement(s)

product of elementary matrices is elementary matrix
if is the inverse of
if and are invertible matrices of the same size, the SLE has an unique solution
sum of elementary matrices is elementary matrix


Proposition. Let be an matrix. If is obtained from by performing an ERO, then there exists an elementary matrix such that , and can be obtained from by performing the same ERO.

Conversely, if is an elementary matrix, then is the matrix obtained from by performing the corresponding ERO.

Proof. Outline: case: e.g.

  • type I ERO:
  • type II ERO:

  • type III ERO:

Remark.

  • illustration of the proposition:

  • inductively, we have:

Example. The following EROs correspond to the matrix multiplication

Proposition. (Invertibility of elementary matrix) Elementary matrices are invertible. The inverse of an elementary matrix is an elementary matrix of the same type.

Proof. The reverse process of each ERO is an ERO of the same type. Let and be the elementary matrices corresponding to these two EROs (an ERO and its reverse process), which are of the same type. Then, , as desired (since can be obtained from by performing an ERO and its reverse process together).

Remark.

  • if is the RREF of , then for some elementary matrices
  • since elementary matrices are invertible, is invertible, and equal to
  • in other words, for some invertible matrix

Example. Since the reverse process of , and are , and , the inverses of the elementary matrices , and are , and respectively.

In particular, the inverse of type I elementary matrix is itself.

Clipboard

Exercise. It is given that matrix is obtained from matrix by performing the EROs , in this order, and .

1 Find .

2 Find .


Then, we will state a simplified version of invertible matrix theorem, in which some results from the complete version of invertible matrix theorem are removed.

Theorem. (Simplified invertible matrix theorem) Let be an matrix. Then, the following are equivalent.

(i) is invertible

(ii) the homogeneous SLE only has the trivial solution

(iii) the RREF of is

(iv) is a product of elementary matrices

Proof. To prove this, we may establish a cycle of implications, i.e. (i) (ii) (iii) (iv) (i), then, when we pick two arbitrary statements form the four statements, they are equivalent to each other, which means that the four statements are equivalent.

(i) (ii): it follows from the proposition about solving SLE, and

(ii) (iii): since the SLE has a unique solution, the RREF of the augmented matrix of the SLE has a leading one in each of the first columns, but not the st column, i.e. it is . It follows that the RREF of is , since after arbitrary EROs, the rightmost zero column is still zero column.

(iii) (iv): since RREF of is , and RREF of equals for some elementary matrices , it follows that . By definition and general 'reverse multiplicativity' of matrix inverse, we have

i.e. is a product of elementary matrices

(iv) (i): since is a product of elementary matrices and an elementary matrix is invertible, it follows that is invertible by general 'reverse multiplicativity' of matrix inverse.

Remark.

  • this theorem provides us multiple ways to prove invertibility of matrix: we can prove it by proving one of the equivalent statements
  • this may make the proof easier
  • later, when we discuss some results about these equivalent statements, they can be linked to this theorem

Example. Consider the matrix . We can find its RREF by Gauss-Jordan algorithm, as follows: Since its RREF is , by the simplified invertible matrix theorem, we also have the following results:

(i) is invertible

(ii) the homogeneous SLE only has trivial solution

(iii) is a product of elementary matrices

Let's verify them one by one.

(i): YesY

(ii): the SLE can be represented by the augmented matrix , and we can find its RREF by Gauss-Jordan algorithm, as follows:

Then, we can directly read from the RREF of augmented matrix that the SLE only has the trivial solution. YesY

(iii): YesY

Clipboard

Exercise. Consider the matrix , and the SLE .

Choose correct statement(s).

is invertible
has unique solution
is not a product of elementary matrices
the RREF of is


The following provides us a convenient and efficient way to find the inverse of a matrix.

Theorem. (Finding matrix inverse using Gauss-Jordan algorithm) Let be an invertible matrix. Then, we can transform the (augmented) matrix to the (augmented) matrix ( is of same size as ), which is RREF of , using a finite number of EROs, and we have .

Proof. Outline: we can write for some elementary matrices , since is RREF of Then, it can be proved that and . It follows that and thus .

Remark.

  • if is not invertible, we are not able to transform to (but RREF of still exists, it is just not in the form of )

Example. Let . After performing EROs as follows: we have .

We have previously proved that is non-invertible. Now, we verify that it is impossible to transform to in which . We perform EROs as follows: The last matrix is in RREF. We can see from the first ERO that, to make the th entry zero, we will also make the th entry zero. Thus, it is impossible to have such transformation.

Clipboard

Exercise. Let be some elementary matrices of same size .

Choose correct statement(s).

we can transform to in which is of same size as
we can transform to in which are of same size as
we can transform to in which are of same size as
we can transform to in which are of same size



Determinants

[edit | edit source]

Then, we will discuss the determinant, which allows characterizing some properties of a square matrix.

Definition. (Determinant) Let be an matrix. The determinant of , denoted by or , is defined recursively as follows:

  • when , we define
  • when , suppose we have defined the determinant of each matrix. Let be the (sub)matrix obtained by deleting the th row and the th column of . We define the -cofactor of by . Then, we define

Remark.

  • minor is determinant of a square submatrix
  • the matrix consisting of all cofactors is called cofactor matrix
  • the definition when is also called the cofactor expansion (or Laplace expansion) along the first row.
  • another notation: , and there are similar notations for matrices with different sizes
  • the signs of the cofactors are alternating. the signs of cofactor at the position of each entry of a matrix is shown below:

which looks like a 'chessboard' pattern.
  • we can observe from the above pattern that the sign of cofactors located at the main diagonal are always positive
  • this is the case since the row number equals column number in main diagonal, and so
  • (some illustrations of deleting the row and column)

Example. (Formulas of determinants of and matrices) and

For the formula of determinants of matrices, we have a useful mnemonic device for it, namely the Rule of Sarrus, as follows:

Proposition. (Rule of Sarrus) We can compute matrix as shown in the following image: , in which red arrows correspond to positive terms, and blue arrows correspond to negative terms. To be more precise, we can compute the matrix in the image by

Proof. It follows from the formula in the above example.

Then, we will given an example about computing the determinant of a matrix, which cannot be computed by the Rule of Sarrus directly.

Example.

Proposition. (Determinant of zero matrix and identity matrix) The determinant of the zero matrix is , and the determinant of the identity matrix is .

Proof.

  • (since the submatrix obtained after removing the 1st row and 1st column of is )
  • so, inductively,

Indeed, we can compute a determinant by the cofactor expansion along an arbitrary row, as in the following theorem.

Theorem. (Cofactor expansion theorem) Let be an matrix with cofactors . Then, and for each positive integer .

Remark.

  • the first formula is cofactor expansion along the th row, and the second formula is cofactor expansion along the th column

Its proof (for the general case) is complicated, and thus is skipped.

Example. (Illustration of cofactor expansion theorem) We use cofactor expansion along the 2nd column here.

Clipboard

Exercise. Let .

1 Calculate .

14
60
104
120
150

2 Calculate .

14
60
104
120
150

3 Choose correct statement(s).

for each matrix
determinant of each matrix has only one possible value
if two matrices have the same determinant, then these two matrices are the same
determinant of submatrices of a matrix must be smaller than the determinant of


Then, we will discuss several properties of determinants that ease its computation.

Proposition. (Effects on determinant when performing EROs) Let be a square matrix.

  • (type I ERO) If we interchange two rows of , the determinant is multiplied by ;
  • (type II ERO) if we multiply a row of by a nonzero constant , the determinant is multiplied by ;
  • (type III ERO) if we add a multiple of a row of to another row, the determinant remains unchanged.

Proof. Outline:

  • (type I ERO) e.g.

  • (type II ERO) e.g.

  • (type III ERO) e.g.

Remark.

  • for the property related to type II ERO, can be zero, and the determinant is multiplied by zero. But, multiplying the row by is not type II ERO if
  • the determinant of matrix with two identical rows is zero because of the following corollary, which is based on the result about type I ERO
  • in view of this proposition, we have some strategies to compute determinant more easily, as follows:
  • apply type II EROs to take out common multiples of a row to reduce the numerical value of entries, so that the computation is easier
  • apply type III EROs to create more zeros in entries
  • apply cofactor expansion along a row or column with many zeros
  • apart from the EROs mentioned in the proposition, we can actually also apply elementary column operations (ECOs)
  • this is because determinant of matrix transpose equals that of the original matrix (it will be mentioned in the proposition about properties of determinants)
  • so, applying elementary column operations is essentially the same as applying EROs, by viewing the operations in different perspectives
  • we have similar notations for elementary column operations, with (stand for row) replaced by (stand for column)

Example. (Vandermonde matrix)

Corollary. The determinant of a square matrix with two identical rows is zero.

Proof. Let be a square matrix with two identical rows. If we interchange the two identical rows in , the matrix is still the same, but its determinant is multiplied by , i.e. Alternatively, it can be proved by definition and induction.

Clipboard

Exercise.

Calculate . (Hint: apply type III EROs or ECOs multiple times, without affecting the value of determinant, to ease computaion)

10
16
80
160
320


Then, we will introduce a convenient way to determine invertibility of a matrix. Before introducing the theorem, we have a lemma.

Lemma. For each elementary matrix and matrix ,

Proof.

  • (type I: ) and (since we are interchanging rows)
  • (type II: ) and (since we are multiplying a row by nonzero constant)
  • (type III: ) and (since we are adding a multiple of a row to another row)

Theorem. (Determining invertibility by determinant) A square matrix is invertible if and only if its determinant is nonzero.

Proof.

  • only if part: by simplified invertible matrix theorem, a matrix is invertible is equivalent to is product of elementary matrices. So, if we denote the elementary matrices by , then

  • if part: Let in which are elementary matrices and is the RREF of . This implies that

Since , so . Thus, has no zero row (its determinant is zero otherwise). Since is in RREF, it follows that (since is square matrix, if not all columns contain leading ones, then there is at least one zero row lying at its bottom, by definition of RREF). By simplified invertible matrix theorem, is invertible.

After introducing this result, we will give some properties of determinants which can ease the computation of determinants.

Proposition. (Properties of determinants) Let and be square matrices of the same size. Then, the following hold.

  • (multiplicativity)
  • (invariance of determinant after transpose)
  • (determinant of matrix inverse is inverse of matrix determinant)

Proof.

  • (multiplicativity) let in which are elementary matrices and is the RREF of . Then,

and

  • then, it remains to prove that
  • if , then
  • if , then the last row of is a zero row, so
  • the last row of is also a zero row, so
  • the result follows
  • (invariance of determinant after transpose) we may prove it by induction and cofactor expansion theorem, e.g. vs.
  • (determinant of matrix inverse is inverse of matrix determinant) using multiplicativity,

( since is invertible)

Example. Let . Since is non-invertible. By simplified invertible matrix theorem, we also have the following results:

  • the homogeneous SLE has not only trivial solution
  • the RREF of is not
  • cannot be expressed as product of elementary matrices
Clipboard

Exercise.

Choose correct statement(s).

if and are non-invertible, then is also non-invertible
if and are invertible, then is also invertible
if and are non-invertible, then is also non-invertible
if and are invertible, then is also invertible
for each matrix
for each matrix
for each matrix and for each integer


Then, we will introduce adjugate of matrix, which has a notable result related to computation of matrix inverse.

Definition. (Adjugate of matrix) Let be an matrix. The adjugate of , denoted by , is the matrix whose th entry is the cofactor .

Remark.

  • it follows that is the transpose of the cofactor matrix of , i.e.
  • it is more common to use this way to compute adjugate

Theorem. (Relationship between adjugate and determinant) Let be an matrix. Then,

Proof. The proof is complicated, and so is skipped.

Corollary. (Formula of matrix inverse) If is invertible, its inverse is given by

Proof.

Example. (Formula of matrix inverse) Let . Then, That is, we can find the inverse of a matrix by interchanging the -th and th entries, multiplying the th and th by (without interchanging them), and multiplying the matrix by the reciprocal of its determinant.

Example. (Adjugate of non-invertible matrix) Let . Then, Also, we have

Clipboard

Exercise.

Using matrix adjugate, solve the SLE . It is given that this SLE has an unique solution.

Its unique solution is:


Then, we will introduce a result that allows us to compute the unique solution of SLE directly, namely Cramer's rule.

Theorem. (Cramer's rule) Let be a SLE in which is an invertible matrix and (we use this notation to denote , a transpose of matrix). Let , and let be the determinant of the matrix obtained by replacing the th column of by the column for each . The unique solution to the SLE is given by

Proof. Since is invertible, the unique solution of the SLE is . Using the formula of matrix inverse, we have Thus, for each , ( are entries at the th row of (and at the th column of the cofactor matrix of ), so multiplying the entries as above gives the -th entry of , namely )

Example. Consider the SLE Since the unique solution to this SLE is

Clipboard

Exercise. Solve the SLE .

Solution.

Since , the matrix is non-invertible. Thus, we cannot use Cramer's rule. Instead, we can transform the augmented matrix representing the SLE to RREF, as follows: Since there is a leading one at the 4th column, the SLE is inconsistent.