Lucas–Kanade

来源：互联网发布：3g模型淘宝编辑：程序博客网时间：2024/04/30 14:05

博主声明：本文英文内容全部来自于http://en.wikipedia.org/wiki/Lucas_Kanade_method,中文由博主据此进行翻译而来。

在计算机视觉中，Lucas-Kanade方法是一种广泛使用的光流估算差分方法，它由Bruce D.Lucas和Takeo Kanade共同开发。它假定在所考虑的像素的局部邻域内，本质上光流是恒定的，由此利用最小二乘原则对邻域内所有像素求解基本光流方程。[In computer vision, the Lucas–Kanade method is a widely used differential method for optical flow estimation developed by Bruce D. Lucas and Takeo Kanade. It assumes that the flow is essentially constant in a local neighbourhood of the pixel under consideration, and solves the basic optical flow equations for all the pixels in that neighbourhood, by the least squares criterion.^[1]^[2]

通过综合多个附近像素的信息，Lucas-Kanade方法通常可以解决光流方程固有的模糊性问题。与逐点方法相比，对于图像噪声它也不太敏感。另一方面，因为这是一种纯局部的方法，它不能够提供图像均匀区域内的光流信息。[By combining information from several nearby pixels, the Lucas-Kanade method can often resolve the inherent ambiguity of the optical flow equation. It is also less sensitive to image noise than point-wise methods. On the other hand, since it is a purely local method, it cannot provide flow information in the interior of uniform regions of the image.]

概念[Concept]

Lucas-Kanade方法假定两个附近帧图像内容的位移很小而且在所考虑的点p的邻域内大体不变。因此可以假定在以点p为中心的窗口内的所有像素，光流方程可以适用。就是说，局部图像光流矢量（Vx,Vy）必须满足以下方程：[The Lucas-Kanade method assumes that the displacement of the image contents between two nearby instants (frames) is small and approximately constant within a neighborhood of the point punder consideration. Thus the optical flow equation can be assumed to hold for all pixels within a window centered at p. Namely, the local image flow (velocity) vector $(V_x,V_y)$ must satisfy]

$I_x(q_1) V_x + I_y (q_1) V_y = -I_t(q_1)$

$I_x(q_2) V_x + I_y (q_2) V_y = -I_t(q_2)$

$\vdots$

$I_x(q_n) V_x + I_y (q_n) V_y = -I_t(q_n)$

这里，q1,q2,...,qn都是窗口内的像素，Ix(qi),Iy(qi),It(qi)是图像I相对于位置x,y和时间t的偏微分，这些量的估算是在点qi和当前时间进行的。[where $q_1,q_2,\dots,q_n$ are the pixels inside the window, and $I_x(q_i),I_y(q_i),I_t(q_i)$ are the partial derivatives of the image $I$ with respect to position x, yand time t, evaluated at the point $q_i$ and at the current time.]

这些方程可以写成矩阵Av=b的形式，其中：[These equations can be written in matrix form $A v = b$ , where]

$A = \begin{bmatrix} I_x(q_1) & I_y(q_1) \\[10pt] I_x(q_2) & I_y(q_2) \\[10pt] \vdots & \vdots \\[10pt] I_x(q_n) & I_y(q_n) \end{bmatrix}, \quad\quad v = \begin{bmatrix} V_x\\[10pt] V_y \end{bmatrix}, \quad \mbox{and}\quad b = \begin{bmatrix} -I_t(q_1)\\ [10pt] -I_t(q_2)\\ [10pt] \vdots \\[10pt] -I_t(q_n) \end{bmatrix}$

该系统方程的数量多于未知量，因此通常它是over-determined。Lucas-Kanade方法包括了根据最小二乘原理的一个折中解。[This system has more equations than unknowns and thus it is usually over-determined. The Lucas-Kanade method obtains a compromise solution by the least squares principle.]

即求解一个2*2的系统：[Namely, it solves the 2×2 system]

$A^T A v=A^T b$ or; $\mathrm{v}=(A^T A)^{-1}A^T b$
此处，AT是矩阵A的转置，即它计算：[where $A^T$ is the transpose of matrix $A$ . That is, it computes]

$\begin{bmatrix} V_x\\[10pt] V_y \end{bmatrix} = \begin{bmatrix} \sum_i I_x(q_i)^2 & \sum_i I_x(q_i)I_y(q_i) \\[10pt] \sum_i I_x(q_i)I_y(q_i) & \sum_i I_y(q_i)^2 \end{bmatrix}^{-1} \begin{bmatrix} -\sum_i I_x(q_i)I_t(q_i) \\[10pt] -\sum_i I_y(q_i)I_t(q_i) \end{bmatrix}$

上述求和是从j=1 到n [with the sums running from i=1 to n.]

矩阵ATA通常被称作图像在点p的结构张量。[The matrix $A^T A$ is often called the structure tensorof the image at the point p.]

加权窗口[Weighted window]

上述普通的最小二乘解对窗口内n个像素qi一视同仁。事实上，通常对于靠近中心像素p的像素更多的权重会更好。[The plain least squares solution above gives the same importance to all n pixels $q_i$ in the window. In practice it is usually better to give more weight to the pixels that are closer to the central pixel p.]

介于此，人们使用最小二乘方程的加权版本：[For that, one uses the weighted version of the least squares equation,]

$A^T W A v=A^T W b$

$\mathrm{v}=(A^T W A)^{-1}A^T W b$

此处，W是一个n*n的对角矩阵，包含权重Wii=wi,被指定到像素qi的方程。[where $W$ is an n×ndiagonal matrix containing the weights $W_{i i} = w_i$ to be assigned to the equation of pixel $q_i$ .]

亦即，它计算：[That is, it computes]

$\begin{bmatrix} V_x\\[10pt] V_y \end{bmatrix} = \begin{bmatrix} \sum_i w_i I_x(q_i)^2 & \sum_i w_i I_x(q_i)I_y(q_i) \\[10pt] \sum_i w_i I_x(q_i)I_y(q_i) & \sum_i w_i I_y(q_i)^2 \end{bmatrix}^{-1} \begin{bmatrix} -\sum_i w_i I_x(q_i)I_t(q_i) \\[10pt] -\sum_i w_i I_y(q_i)I_t(q_i) \end{bmatrix}$

权重wi通常被设置为qi和p之间距离的高斯函数。[The weight $w_i$ is usually set to a Gaussian function of the distance between $q_i$ and p.]

改进和提高[Improvements and extensions]

最小二乘方法暗含以下假设，即图像数据的误差具有零均值高斯分布。如果人们预期窗口包含一定百分比的outliers（粗大误差数据，它不遵循“通常”的高斯误差分布），人们可以通过统计分析来检测他们，并减小他们相应的权重。[The least-squares approach implicitly assumes that the errors in the image data have a Gaussian distribution with zero mean. If one expects the window to contain a certain percentage of "outliers" (grossly wrong data values, that do not follow the "ordinary" Gaussian error distribution), one may use statistical analysis to detect them, and reduce their weight accordingly.]

Lucas-Kanade方法（per se？）只可被用于当两帧图像之间的光流矢量Vx,Vy小到足以使微分光流方程得以维持的情况下，通常来说它小于像素间距。当流矢量超过此限制，比如在立体匹配或warped document registration, Lucas-Kanade方法仍然可被用于精细化一些由其它方法得来的同一运动的粗糙的估计，例如，通过外推由以前帧计算的流矢量，或通过在reduced scale versions of images上运行Lucas-Kanade算法。的确，后一种方法是广为人知的Kanade-Lucas-Tomasi（KLT）特征匹配算法的基础。[The Lucas-Kanade method per se can be used only when the image flow vector $V_x,V_y$ between the two frames is small enough for the differential equation of the optical flow to hold, which is often less than the pixel spacing. When the flow vector may exceed this limit, such as in stereo matching or warped document registration, the Lucas-Kanade method may still be used to refine some coarse estimate of the same, obtained by other means; for example, by extrapolating the flow vectors computed for previous frames, or by running the Lucas-Kanade algorithm on reduced-scale versions of the images. Indeed, the latter method is the basis of the popular Kanade-Lucas-Tomasi (KLT) feature matching algorithm.]

一种类似的技术可被用于计算图像内容的差分仿射形变。[A similar technique can be used to compute differential affine deformations of the image contents.]

亦请看[See also]

光流[Optical flow]
HS方法[Horn–Schunck method]
ST角检测算法[The Shi and Tomasi corner detection algorithm]

参考文献[References]

B. D. Lucas and T. Kanade (1981), An iterative image registration technique with an application to stereo vision. Proceedings of Imaging Understanding Workshop, pages 121--130
Bruce D. Lucas (1984) Generalized Image Matching by the Method of Differences (doctoral dissertation)

外部链接[External links]

The image stabilizer plugin for ImageJ based on the Lucas–Kanade method
Mathworks Lucas-Kanade Matlab implementation of inverse and normal affine Lucas-Kanade
FolkiGPU : GPU implementation of an iterative Lucas-Kanade based optical flow
Lucas-Kanade for the iPhone by Success Labs. A modified and enhanced port of the OpenCV lkdemo sample application to the iPhone.
KLT: An Implementation of the Kanade–Lucas–Tomasi Feature Tracker
Takeo Kanade

0 0