vb.net - Extract transform and rotation matrices from homography?

Question

I have 2 consecutive images from a camera and I want to estimate the change in camera pose:

I calculate the optical flow:

Const MAXFEATURES As Integer = 100
imgA = New Image(Of [Structure].Bgr, Byte)("pic1.bmp")
imgB = New Image(Of [Structure].Bgr, Byte)("pic2.bmp")
grayA = imgA.Convert(Of Gray, Byte)()
grayB = imgB.Convert(Of Gray, Byte)()
imagesize = cvGetSize(grayA)
pyrBufferA = New Emgu.CV.Image(Of Emgu.CV.Structure.Gray, Byte) _
    (imagesize.Width + 8, imagesize.Height / 3)
pyrBufferB = New Emgu.CV.Image(Of Emgu.CV.Structure.Gray, Byte) _
    (imagesize.Width + 8, imagesize.Height / 3)
features = MAXFEATURES
featuresA = grayA.GoodFeaturesToTrack(features, 0.01, 25, 3)
grayA.FindCornerSubPix(featuresA, New System.Drawing.Size(10, 10),
                       New System.Drawing.Size(-1, -1),
                       New Emgu.CV.Structure.MCvTermCriteria(20, 0.03))
features = featuresA(0).Length
Emgu.CV.OpticalFlow.PyrLK(grayA, grayB, pyrBufferA, pyrBufferB, _
                          featuresA(0), New Size(25, 25), 3, _
                          New Emgu.CV.Structure.MCvTermCriteria(20, 0.03D),
                          flags, featuresB(0), status, errors)
pointsA = New Matrix(Of Single)(features, 2)
pointsB = New Matrix(Of Single)(features, 2)
For i As Integer = 0 To features - 1
    pointsA(i, 0) = featuresA(0)(i).X
    pointsA(i, 1) = featuresA(0)(i).Y
    pointsB(i, 0) = featuresB(0)(i).X
    pointsB(i, 1) = featuresB(0)(i).Y
Next
Dim Homography As New Matrix(Of Double)(3, 3)
cvFindHomography(pointsA.Ptr, pointsB.Ptr, Homography, HOMOGRAPHY_METHOD.RANSAC, 1, 0)

and it looks right, the camera moved leftwards and upwards: Now I want to find out how much the camera moved and rotated. If I declare my camera position and what it's looking at:

' Create camera location at origin and lookat (straight ahead, 1 in the Z axis)
Location = New Matrix(Of Double)(2, 3)
location(0, 0) = 0 ' X location
location(0, 1) = 0 ' Y location
location(0, 2) = 0 ' Z location
location(1, 0) = 0 ' X lookat
location(1, 1) = 0 ' Y lookat
location(1, 2) = 1 ' Z lookat

How do I calculate the new position and lookat?

If I'm doing this all wrong or if there's a better method, any suggestions would be very welcome, thanks!

Answer 1

For pure camera rotation R = A^-1HA. To prove this consider image to plane homographies H1=A and H2=AR, where A is camera intrinsic matrix. Then H12=H2*H1^-1=A^-1RA, from which you can obtain R

Camera translation is harder to estimate. If the camera translates you have to a find fundamental matrix first (not homography): x^TFx=0 and then convert it into an essential matrix E=A^TFA; Then you can decompose E into rotation and translation E=t_xR, where t_x means a vector product matrix. Decomposition is not obvious, see this.

The rotation you get will be exact while the translation vector can be found only up to scale. Intuitively this scaling means that from the two images alone you cannot really say whether the objects are close and small or far away and large. To disambiguate we may use a familiar size objects, known distance between two points, etc.

Finally note that a human visual system has a similar problem: though we "know" the distance between our eyes, when they are converged on the object the disparity is always zero and from disparity alone we cannot say what the distance is. Human vision relies on triangulation from eyes version signal to figure out absolute distance.