“矩阵代数初步”(Introduction to MATRIX ALGEBRA)课程由Prof. A.K.Kaw(University of South Florida)设计并讲授。
PDF格式学习笔记下载（Academia.edu）
第10章课程讲义下载（PDF）
Summary
- Definition
If $[A]$ is a $n imes n$ matrix, then $[X] eq vec0$ is an eigenvector of $[A]$ if $$[A][X] = lambda[X]$$ where $lambda$ is a scalar and $[X] eq0$. The scalar $lambda$ is called the eigenvalue of $[A]$ and $[X]$ is called the eigenvector corresponding to the eigenvalue $lambda$. - Finding eigenvalue and eigenvector
- To find the eigenvalues of a $n imes n$ matrix $[A]$, we have $$AX=lambda X$$ $$Rightarrow AX-lambda X=0$$ $$Rightarrow (A-lambda I)X=0$$ For the above set of equations to have a non-zero solution $$det(A-lambda I) = 0$$ The above equation is called the characteristic equation of $[A]$, which gives $$lambda^n + c_1lambda^{n-1} + cdots + c_n=0$$ Hence this polynomial has $n$ roots.
- For example, finding the eigenvalues of the matrix $$[A] = egin{bmatrix}3& -1.5\ -0.75& 0.75 end{bmatrix}$$ We have $$A-lambda I = egin{bmatrix}3 - lambda & -1.5\ -0.75& 0.75 - lambda end{bmatrix}$$ $$det(A - lambda I) = (3-lambda)(0.75-lambda) - (-0.75)(-1.5)$$ $$Rightarrow lambda^2-3.75lambda + 1.125 =0$$ $$Rightarrow lambda = {3.75pmsqrt{{3.75}^{2} - 4.5}over2} = 3.421165, 0.3288354$$ That is, the eigenvalues are 3.421165 and 0.3288354.
- To find the eigenvectors of the above matrix $[A]$. Let $[X] = egin{bmatrix}x_1 \ x_2end{bmatrix}$ and we already have $lambda_1 = 3.421165$ and $lambda_2 = 0.3288354$. When $lambda = 3.421165$, from the definition we have $$(A-lambda I)X=0$$ $$Rightarrow left(egin{bmatrix}3& -1.5\ -0.75& 0.75 end{bmatrix} - egin{bmatrix}3.421165& 0\ 0& 3.421165 end{bmatrix} ight) egin{bmatrix}x_1 \ x_2end{bmatrix} = 0$$ $$Rightarrow egin{bmatrix}-0.421165& -1.5\ -0.75& -2.671165 end{bmatrix} egin{bmatrix}x_1 \ x_2end{bmatrix} = egin{bmatrix}0 \ 0end{bmatrix}$$ $$Rightarrow -0.421165x_1 -1.5x_2 = 0 Rightarrow x_2 = -0.2807767x_1$$ that is, $$[X] = egin{bmatrix}x_1\ -0.2807767x_1end{bmatrix} = x_1egin{bmatrix}1 \ -0.2807767end{bmatrix}$$ Hence the eigenvector corresponding to $lambda_1 = 3.421165$ is $$egin{bmatrix}1 \ -0.2807767end{bmatrix}$$ Similarly, we have calculate the eigenvector corresponding to $lambda_2 = 0.3288354$: $$ left(egin{bmatrix}3& -1.5\ -0.75& 0.75 end{bmatrix} - egin{bmatrix}0.3288354& 0\ 0& 0.3288354 end{bmatrix} ight) egin{bmatrix}x_1 \ x_2end{bmatrix} = 0$$ $$Rightarrow egin{bmatrix}2.671165& -1.5\ -0.75& 0.4211646 end{bmatrix} egin{bmatrix}x_1 \ x_2end{bmatrix} = egin{bmatrix}0 \ 0end{bmatrix}$$ $$Rightarrow 2.671165x_1 -1.5x_2 = 0 Rightarrow x_2 = 1.780776x_1$$ that is, $$[X] = egin{bmatrix}x_1\ 1.780776x_1end{bmatrix} = x_1egin{bmatrix}1 \ 1.780776end{bmatrix}$$ Hence the eigenvector corresponding to $lambda_1 = 0.3288354$ is $$egin{bmatrix}1 \ 1.780776end{bmatrix}$$
- Some related theorems
- If $[A]$ is a $n imes n$ triangular matrix - upper triangular, lower triangular and diagonal, the eigenvalues of $[A]$ are the diagonal entries of $[A]$.
- $lambda = 0$ is an eigenvalue of $[A]$ if $[A]$ is a singular (non-invertible) matrix.
- $[A]$ and $[A]^{T}$ have the same eigenvalues.
- Eigenvalues of a symmetric matrix are real.
- Eigenvectors of a symmetric matrix are orthogonal, but only for distinct eigenvalues.
- $|det(A)|$ is the product of the absolute values of the eigenvalues of $[A]$.
- Power Method
- One of the most common methods used for finding eigenvalues and eigenvectors is the power method. It is used to find the largest eigenvalue in an absolute sense. Note that if this largest eigenvalues is repeated, this method will not work. Also this eigenvalue needs to be distinct.
- The method is as follows:
- Assume a guess $X^{(0)}$ for the eigenvector in $$AX=lambda X$$ equation. One of the entries of $X^{(0)}$ needs to be unity.
- Find $$Y^{(1)} = AX^{(0)}$$
- Scale $Y^{(1)}$ so that the chosen unity component remains unity. $$Y^{(1)} = lambda^{(1)}X^{(1)}$$
- Repeat steps 2 and 3 with $X=X^{(1)}$ to get $X^{(2)}$.
- Repeat steps 2 and 3 until the value of the eigenvalue converges.
- For example, using the power method, find the largest eigenvalue and the corresponding eigenvectors of $$[A] = egin{bmatrix}1.5& 0& 1\ -0.5& 0.5& -0.5\ -0.5& 0& 0 end{bmatrix}$$ given with the initial guess $egin{bmatrix}1\ 1\ 1 end{bmatrix}$. From the algorithm, we have $$AX^{(0)} = egin{bmatrix}1.5& 0& 1\ -0.5& 0.5& -0.5\ -0.5& 0& 0 end{bmatrix} egin{bmatrix}1\ 1\ 1 end{bmatrix} = egin{bmatrix}2.5\ -0.5\ -0.5 end{bmatrix}$$ $$Rightarrow Y^{(1)} = 2.5egin{bmatrix}1\ -0.2\ -0.2 end{bmatrix} $$ so $lambda^{(1)} = 2.5$ and $X^{(1)} = egin{bmatrix}1\ -0.2\ -0.2 end{bmatrix}$. Note that we choose the first element of $X^{(0)}$ to be unity. Then $$AX^{(1)} = egin{bmatrix}1.5& 0& 1\ -0.5& 0.5& -0.5\ -0.5& 0& 0 end{bmatrix} egin{bmatrix}1\ -0.2\ -0.2 end{bmatrix} = egin{bmatrix}1.3\ -0.5\ -0.5 end{bmatrix}$$ $$Rightarrow Y^{(2)} = 1.3egin{bmatrix}1\ -0.3846\ -0.3846 end{bmatrix}$$ so $lambda^{(2)} = 1.3$ and $X^{(2)} = egin{bmatrix}1\ -0.3846\ -0.3846 end{bmatrix}$. Thus far, the absolute relative approximate error in the eigenvalues is $$|varepsilon| = left|{lambda^{(2)}-lambda^{(1)}over lambda^{(2)}} ight| = left|{1.3-2.5over1.3} ight| = 0.9230769$$ Conducting further iterations, the eigenvalue after 5 iterations is 1.02459 and its absolute relative approximate error is 0.012441. The exact value of the eigenvalue is $lambda = 1$ and the corresponding eigenvector is $$X=egin{bmatrix}1\-0.5\-0.5 end{bmatrix}$$
- R code
This function includes 4 parameters:
- A is the target matrix;
- x0 is the initial guess which is a vector;
- eps is the tolerance of the error which can be modified;
- maxit is the maximum number of iterations in the process.
A = matrix(c(1.5, -0.5, -0.5, 0, 0.5, 0, 1, -0.5, 0), ncol = 3) PowerEigen(A, x0 = c(1, 1, 1)) Converged after 23 iterations $value [1] 1 $vector [,1] [1,] 1.0 [2,] -0.5 [3,] -0.5
Selected Problems
1. The eigenvalues $lambda$ of matrix $[A]$ are found by solving the equation ( ).
Solution: $$|A-lambda I| = 0$$
2. Find the eigenvalues and eigenvectors of $$[A] = egin{bmatrix} 10& 9\ 2& 3end{bmatrix}$$ using the determinant method.
Solution: $$|A-lambda I| = 0$$ $$Rightarrow detleft(egin{bmatrix} 10-lambda & 9\ 2 & 3-lambdaend{bmatrix} ight) = 0$$ $$Rightarrow (10-lambda)(3-lambda) - 18=0$$ $$Rightarrow lambda^2 - 13lambda +12 =0$$ $$Rightarrow lambda_1=1, lambda_2=12$$ For $lambda_1=1$, we have $$egin{bmatrix} 10-lambda & 9\ 2 & 3-lambdaend{bmatrix} egin{bmatrix}x_1 \ x_2end{bmatrix} =egin{bmatrix} 0\ 0end{bmatrix} $$ $$Rightarrow egin{bmatrix} 9 & 9\ 2 & 2end{bmatrix}egin{bmatrix}x_1 \ x_2end{bmatrix} = egin{bmatrix} 0\ 0 end{bmatrix}$$ $$Rightarrow x_2 =-x_1$$ $$Rightarrow X=egin{bmatrix}x_1\ -x_1 end{bmatrix} = x_1egin{bmatrix} 1\ -1end{bmatrix}$$ Thus the eigenvector corresponding to $lambda_1=1$ is $egin{bmatrix} 1\ -1end{bmatrix}$. Similarly, we can find the second eigenvector corresponding to $lambda_2=12$: $$egin{bmatrix} 10-lambda & 9\ 2 & 3-lambdaend{bmatrix} egin{bmatrix}x_1 \ x_2end{bmatrix} =egin{bmatrix} 0\ 0end{bmatrix} $$ $$Rightarrow egin{bmatrix} -2 & 9\ 2 & -9end{bmatrix}egin{bmatrix}x_1 \ x_2end{bmatrix} = egin{bmatrix} 0\ 0 end{bmatrix}$$ $$Rightarrow -2x_1+9x_2 = 0 Rightarrow x_2 = {2over9}x_1$$ $$Rightarrow X=egin{bmatrix}x_1\ {2over9}x_1 end{bmatrix} = x_1egin{bmatrix} 1\ {2over9}end{bmatrix} Rightarrow egin{bmatrix} 9\ 2end{bmatrix}$$ Thus the eigenvector corresponding to $lambda_2=12$ is $egin{bmatrix} 9\ 2end{bmatrix}$.
3. Find the eigenvalues and eigenvectors of $$[A] = egin{bmatrix}4& 0& 1\ -2& 0& 1\ 2& 0& 1end{bmatrix}$$ using the determinant method.
Solution:
First of all, we can read off that $lambda = 0$ is an eigenvalue of this matrix since it is singular. Then from the definition we have $$|A-lambda I| = 0$$ $$Rightarrow detleft( egin{bmatrix} 4-lambda & 0 & 1\ -2 & -lambda & 1\ 2 & 0& 1-lambda end{bmatrix} ight) = 0$$ $$Rightarrow (4-lambda)left[(-lambda)(1-lambda) - 0 ight]+left[1cdot(0+2lambda) ight] =0$$ $$Rightarrow (4-lambda)(lambda^2-lambda) +2lambda= 0$$ $$Rightarrow lambda(-lambda^2+5lambda-4+2) =0$$ $$Rightarrow lambda(lambda^2-5lambda+2) =0$$ $$Rightarrow lambda_1=0, lambda_2 = 4.561553, lambda_3=0.4384472.$$ For $lambda_1 =0$, we have $$egin{bmatrix} 4-lambda & 0 & 1\ -2 & -lambda & 1\ 2 & 0& 1-lambda end{bmatrix}egin{bmatrix}x_1\ x_2\ x_3 end{bmatrix}=egin{bmatrix}0\ 0\ 0 end{bmatrix}$$ $$Rightarrow egin{bmatrix} 4 & 0 & 1\ -2 & 0 & 1\ 2 & 0& 1 end{bmatrix}egin{bmatrix}x_1\ x_2\ x_3 end{bmatrix} = egin{bmatrix} 0\ 0\ 0 end{bmatrix}$$ The coefficient matrix is $$egin{bmatrix} 4& 0& 1\ -2& 0& 1\ 2& 0& 1 end{bmatrix} Rightarrow egin{bmatrix} 0& 0& 3\ -2& 0& 1\ 0& 0& 2 end{bmatrix} Rightarrow egin{bmatrix} 0& 0& 0\ -2& 0& 0\ 0& 0& 2 end{bmatrix}$$ that is, $x_1=x_3=0$ and $x_2$ is arbitrary. Hence the eigenvector corresponding to $lambda_1=0$ is $egin{bmatrix}0 \ 1\ 0 end{bmatrix}$. For $lambda_2= 4.561553$, we have $$egin{bmatrix} 4-lambda & 0 & 1\ -2 & -lambda & 1\ 2 & 0& 1-lambda end{bmatrix} egin{bmatrix}x_1\ x_2\ x_3 end{bmatrix}=egin{bmatrix}0\ 0\ 0 end{bmatrix}$$ $$Rightarrow egin{bmatrix} -0.561553 & 0 & 1\ -2 & -4.561553 & 1\ 2 & 0& -3.561553 end{bmatrix}egin{bmatrix}x_1\ x_2\ x_3 end{bmatrix} = egin{bmatrix} 0\ 0\ 0 end{bmatrix}$$ The coefficient matrix is $$egin{bmatrix} -0.561553 & 0 & 1\ -2 & -4.561553 & 1\ 2 & 0& -3.561553 end{bmatrix} Rightarrow egin{bmatrix} -0.561553 & 0 & 1\ 0 & -4.561553 & -2.561553\ 0 & 0& 0 end{bmatrix}$$ $$Rightarrow egin{cases}x_1= 1.780776x_3\ x_2 = -0.5615528x_3end{cases}$$ where $x_3$ is arbitrary. Thus the eigenvector corresponding to $lambda_2=4.561553$ is $egin{bmatrix}1.780776\ -0.5615528\ 1 end{bmatrix}$. For $lambda_3= 0.4384472$, we have $$egin{bmatrix} 4-lambda & 0 & 1\ -2 & -lambda & 1\ 2 & 0& 1-lambda end{bmatrix} egin{bmatrix}x_1\ x_2\ x_3 end{bmatrix}=egin{bmatrix}0\ 0\ 0 end{bmatrix}$$ $$Rightarrow egin{bmatrix} 3.561553 & 0 & 1\ -2 & -0.4384472 & 1\ 2 & 0& 0.5615528 end{bmatrix} egin{bmatrix} x_1\ x_2\ x_3 end{bmatrix} = egin{bmatrix} 0\ 0\ 0 end{bmatrix}$$ The coefficient matrix is $$egin{bmatrix} 3.561553 & 0 & 1\ -2 & -0.4384472 & 1\ 2 & 0& 0.5615528 end{bmatrix} Rightarrow egin{bmatrix} 3.561553 & 0 & 1\ 0 & -0.4384472 & 1.561553\ 0 & 0& 0 end{bmatrix}$$ $$Rightarrow egin{cases}x_1= -0.2807764x_3\ x_2 = 3.561553x_3end{cases}$$ where $x_3$ is arbitrary. Thus the eigenvector corresponding to $lambda_3= 0.4384472$ is $egin{bmatrix}-0.2807764\ 3.561553 \ 1 end{bmatrix}$.
4. Find the eigenvalues of these matrices by inspection: (A) $egin{bmatrix}2& 0& 0\ 0& -3& 0\ 0& 0& 6end{bmatrix}$; (B) $egin{bmatrix}3& 5& 7\ 0& -2& 1\ 0& 0& 0end{bmatrix}$; (C) $egin{bmatrix}2& 0& 0\ 3& 5& 0\ 2& 1& 6end{bmatrix}$.
Solution:
The eigenvalues of a triangular matrix are the diagonal entries of the matrix. Thus, (A) $lambda_1=2, lambda_2=-3, lambda_3=6$. (B) $lambda_1=3, lambda_2=-2, lambda_3=0$. (C) $lambda_1=2, lambda_2=5, lambda_3=6$.
5. Find the largest eigenvalue in magnitude and its corresponding vector by using the power method $$[A] = egin{bmatrix}4& 0& 1\ -2& 0& 1\ 2& 0& 1 end{bmatrix}$$ Start with an initial guess of the eigenvector as $egin{bmatrix}1\ -0.5\ 0.5 end{bmatrix}$.
Solution:
We will use the R script directly,
A = matrix(c(4, -2, 2, 0, 0, 0, 1, 1, 1), ncol = 3)
PowerEigen(A, x0 = c(1, -0.5, -0.5))
Converged after 9 iterations
$value
[1] 4.561553
$vector
[,1]
[1,] 1.0000000
[2,] -0.3153416
[3,] 0.5615528
6. Prove if $lambda$ is an eigenvalue of $[A]$, then ${1overlambda}$ is an eigenvalue of $[A]^{-1}$.
Solution:
We hope to prove that $A^{-1}X={1overlambda}X$ where $AX=lambda X$. $$A^{-1}X=A^{-1}(lambda cdot {1overlambda}) X = {1overlambda} A^{-1}lambda X = {1overlambda} A^{-1}A X = {1overlambda}X$$
7. Prove that square matrices $[A]$ and $[A]^{T}$ have the same eigenvalues.
Solution:
We hope to prove that $det(A-lambda I) = det(A^{T}-lambda I)$, and an important result is $det(A) = detleft(A^{T} ight)$ for $A$ is a square matrix. $$det(A-lambda I) = detleft((A-lambda I)^{T} ight)$$ $$=detleft(A^{T}-(lambda I)^{T} ight)$$ $$=detleft(A^{T}-lambda I ight)$$
8. Show that $|det(A)|$ is the product of the absolute values of the eigenvalues of $[A]$.
Solution:
We hope to prove that $$|det(A)| =prod_{i=1}^{n}|lambda_i|$$ where $lambda_i$ is the eigenvalues of matrix $A$. By the definition we have $$|det(A-lambda I)| = |f(lambda)| =|(lambda_1-lambda)(lambda_2-lambda)cdots(lambda_n-lambda)| $$ Set $lambda=0$ (since it is a variable), we have $$|det(A)| = |lambda_1lambda_2cdotslambda_n|= prod_{i=1}^{n}|lambda_i|$$
9. What are the eigenvalues of the following matrix? $$egin{bmatrix}5& 6& 17\ 0& -19& 23\ 0& 0& 37 end{bmatrix}$$
Solution:
This is an upper triangular matrix, hence its eigenvalues are the diagonal elements, that is, 5, -19, and 37.
10. If $egin{bmatrix}-4.5\ -4\ 1 end{bmatrix}$ is an eigenvector of $egin{bmatrix}8& -4& 2\ 4& 0& 2\ 0& -2& -4 end{bmatrix}$, what is the eigenvalue corresponding to the eigenvector?
Solution:
From the definition we have $AX=lambda X$, that is $$egin{bmatrix}8& -4& 2\ 4& 0& 2\ 0& -2& -4 end{bmatrix} egin{bmatrix}-4.5\ -4\ 1 end{bmatrix} =lambda egin{bmatrix}-4.5\ -4\ 1 end{bmatrix}$$ $$Rightarrow egin{bmatrix}-18\ -16\ 4 end{bmatrix} = lambda egin{bmatrix}-4.5\ -4\ 1 end{bmatrix}$$ Hence $lambda = 4$.
11. The eigenvalues of the following matrix $$egin{bmatrix}3& 2& 9\ 7& 5& 13\ 6& 17& 19end{bmatrix}$$ are given by solving the cubic equation ( ).
Solution: $$|A-lambda I| =detleft( egin{bmatrix}3-lambda& 2& 9\ 7& 5-lambda& 13\ 6& 17& 19-lambdaend{bmatrix} ight)$$ $$= (3-lambda)egin{vmatrix}5-lambda & 13\ 17 & 19-lambdaend{vmatrix} - 2egin{vmatrix}7 & 13\ 6 & 19-lambdaend{vmatrix} + 9egin{vmatrix}7 & 5-lambda\ 6 & 17end{vmatrix}$$ $$= (3-lambda)left((5-lambda)(19-lambda) - 13 imes17 ight) - 2 imes left(7(19 - lambda) - 6 imes 13 ight) + 9 left(7 imes17-6(5-lambda) ight)$$ $$=lambda^3 - 27lambda^2 -122lambda -313$$
12. The eigenvalues of a $4 imes4$ matrix $[A]$ are given as 2, -3, 13, and 7. What is the $|det(A)|$?
Solution:
Since for a $n imes n$ matrix $$|det(A)| = prod_{i=1}^{n}|lambda_i|$$ Hence we have $$|det(A)| = |2 imes(-3) imes13 imes7| = 546$$
13. If one of the eigenvalues of $[A]_{n imes n}$ is zero, it implies ( ).
Solution:
If an eigenvalue is zero, then its determinant must be zero. Furthermore, this means it is a singular matrix (i.e. non-invertible).
14. Given that matrix $$[A] = egin{bmatrix}8& -4& 2\ 4& 0& 2\ 0& -2& -3 end{bmatrix}$$ has an eigenvalue value of 4 with the corresponding eigenvectors of $[x]=egin{bmatrix}-4.5\ -4\ 1end{bmatrix}$, then what is the value of $[A]^{5}[X]$?
Solution:
Firstly, we show that $A^{m}X=lambda^{m}X$, where $lambda$ is an eigenvalue of $[A]$. By Mathematical Induction, we can read off that $n=1$ is correct.\ Then suppose that $n=m-1$ is correct, that is, $A^{m-1}X = lambda^{m-1}X$ holds. For $n=m$, we have $$A^{m}X = AA^{m-1}X = Alambda^{m-1}X =lambda^{m-1}AX = lambda^{m-1}lambda X =lambda^{m}X$$ as desired. From this result, we have $$A^5X=lambda^{5}X = 4^5egin{bmatrix}-4.5\ -4\ 1end{bmatrix} = egin{bmatrix}-4608\ -4096\ 1024end{bmatrix}$$