Overview
This week we mainly talked about two parts. The difference between the fundamental matrix and the essential matrix. Using correspondences to estimate the fundamental matrix. Find the homography matrix from two images that are projected from a planer scene.
Knowledge and Thinking
Fundamental Matrix
Before we talk about the fundamental matrix, we should clarify the defination of camera matrix.In general, the image point ( x^prime sim KPi_0 egin{bmatrix} R & T \ 0 &1end{bmatrix}P ), where
( K=K_{s} K_{f}=left[egin{array}{ccc} f s_{x} & f s_{ heta} & o_{x} \ 0 & f s_{y} & o_{y} \ 0 & 0 & 1 end{array} ight] ).
We call (x^prime) point in uncalibrated camera. The point ( x = K^{-1}x^prime ) is called the point in calibrated camera. And we have the equation of essential matrix( x_2^ op Ex_1 =0 ) where ( E ) is specially for calibrated camera. In the uncalibrated camera, the equation can be modified as
( egin{equation}egin{aligned}x_2^ op Ex_1 = 0\ (K^{-1}x_2^prime )^ op E(K^{-1}x1') = 0\ x_2^{prime op} underbrace{K^{- op}E K^{- op}}_{:F}x_1^prime = 0end{aligned}end{equation} )
The matrix F is defined as fundamental matrix. The fundamental matrix contains the calibration information in addition to the transformation and rotation in the essential matrix. The essential matrix can be drived into (E = hat TR). In the fundamental matrix the transformation and rotation should be modiflied by (K) ( F sim hat T^prime KRK^{-1} ) and ( hat T^prime = K^{- op}hat T K^{-1} ).
The procedure to estimate the fundamental matrix has a bit of difference from the essential matrix since the svd of the fundamental matrix contains two different singular values. After using 8 point algorithm to capture the F, we also need singular value decomposition to add the rank constraint for the matrix. But we just set the third singular value to zero without modifying the first two singular values.
( SVD(F) = Uegin{bmatrix} sigma_1 & 0 & 0 \ 0 & sigma_2 & 0 \0 & 0 & 0 end{bmatrix} V^ op )
We need to find the corresponding transfomation and rotation of the points for reconstruction of 3d point. Therefore, it is not enough to find the 3d point only by using fundamental matrix, because we cannot estimate the calibration matrix (K).
Planer scene
The best way to reconstruct the 3d object is using the images which contain the different information of the object, such as by using the camera with different views. But in some situations, we can only obtain some images from only one view such as the satellite images. All of the object points are in the same flat. The reconstruction algorithm should be modified because the features of the planer scene have been changed. A general case can be shown in the figure1
Figure 1 the illustration of planer scene
The planer flat ( mathcal{H} ) is defined as ( mathcal H =: {Xinmathbb R^3 | n^ op x = d} ) where d donates the difference from the left origin to the flat. and n is the normalized vector.
Since (P_2) contains the rotation and transformation of the first point (P_1) . ( P_2 = RP_1+T )
( egin{equation}egin{aligned}P_2 &= RP_1+Tfrac{1}{d}n^ op P_1 \ &= (R+Tfrac{1}{d}n^ op )P_1 end{aligned}end{equation} Rightarrow x_2 sim (R+Tfrac{1}{d}n^ op)x_1 \H=:R+Tfrac{1}{d}n^ op )
( H ) describe the transformation from the image point (x_2) to (x_1).
The planer geometry matrix (H) can also be interpreted as a special situation of a two view geometry, which the object points are in the same plan, therefore a coordinate information can be canceled and set to zero. We suppose the Z coordinate is zero.
( left[egin{array}{l} x \ y \ 1 end{array} ight] simleft[egin{array} f & 0 & 0 & 0 \ 0 & f & 0 & 0 \ 0 & 0 & 1 & 0 end{array} ight]left[egin{array} r_{11} & r_{12} & r_{13} & t_{x} \ r_{21} & r_{22} & r_{23} & t_{y} \ r_{31} & r_{32} & r_{33} & t_{z} \ 0 & 0 & 0 & 1 end{array} ight]left[egin{array}{c} p \ q \ 0 \ 1 end{array} ight] )
The transformation matrix can be simplified as a (3 imes 3) matrix
( left[egin{array}{l} x \ y \ 1 end{array} ight] simleft[egin{array}{lll} f & 0 & 0 \ 0 & f & 0 \ 0 & 0 & 1 end{array} ight]left[egin{array}{lll} r_{11} & r_{12} & t_{x} \ r_{21} & r_{22} & t_{y} \ r_{31} & r_{32} & t_{z} end{array} ight]left[egin{array}{l} p \ q \ 1 end{array} ight] )
( left[egin{array}{l} x \ y \ 1 end{array} ight] sim left[egin{array}{lll} h_{11} & h_{12} & h_{13} \ h_{21} & h_{22} & h_{23} \ h_{31} & h_{32} & h_{33} end{array} ight]left[egin{array}{l} p \ q \ 1 end{array} ight] )
Form the view of the image points, we can say that ( x_2 ) can be obtained by a rotation and transformation of (x_1). (x_2 = Hx_1) H donates a linear transformation and H contains rotation and transformation. ( H = egin{bmatrix} R & T\ u^ op & vend{bmatrix} )
( x_2 sim Hx_1 ) and (x_2) is orthogonal to (u imes Hx_1) for all (u in mathbb R^3) (x_2^ op u imes Hx_1 = 0) according to 8 points algorithm (Ax=0) in the situation (dim(A) leq 6) we can not use it to solve the equation.
Because of the transformation matrix (H) we can say that for every ( x_1 ) there is a correspondence point ( x_2 )