18 Optical Flow Field Computations and Use

18.1 Optical Flow Field Computation -- General Issues

Beauchemin, S.S.[Steven S.], Barron, J.L.[John L.],
The Computation of Optical-Flow,
Surveys(27), No. 3, September 1995, pp. 433-467.
DOI Link Survey, Optic Flow. BibRef 9509

Gibson, J.J.,
The Perception of the Visual World,
Boston: Houghton Mifflin1955. ?? BibRef 5500 BookBasic perception book where optical flow is formally introduced. BibRef

Gibson, J.J.,
Optical Motion and Transformations as Stimuli for Visual Perceptions,
PsychR(64), No. 5, 1957, pp. 288-295. BibRef 5700

Gibson, J.J.,
What Gives Rise to the Perception of Motion?,
PsychR(75), No. 4, 1968, pp. 335-346. BibRef 6800

Limb, J.O., and Murphy, J.A.,
Estimating Velocity of Moving Images in Television Signals,
CGIP(4), No. 4, December 1975, pp. 311-327.
Elsevier DOI BibRef 7512
And:
Measuring the Speed of Moving Objects from Television Signals,
Commun(23), No. 4, April 1975, pp. 474-478. Early gradient based method for computation directly from image measurements. The basic results here are that velocity estimates work only for single moving objects. Essentially the subpixel interpolation of the correlation peak in the matchpoint neighborhood.
See also Source-Receiver Encoding of Television Signals. BibRef

Cafforio, C., and Rocca, F.,
Methods for Measuring Small Displacements of Television Images,
IT(22), No. 5 September, 1976, pp. 573-579. BibRef 7609

Cafforio, C., Rocca, F.,
Tracking Moving Objects in Television Images,
SP(1), 1979, pp. 133-140. BibRef 7900

Cafforio, C.,
Remarks on the Differential Method for the Estimation of Movement in Television Images,
SP(4), 1982, pp. 45-52. BibRef 8200

Cafforio, C., and Rocca, F.,
The Differential Method for Image Motion Estimation,
ISPDSA83(104-124). BibRef 8300

Horn, B.K.P., and Schunck, B.G.,
Determining Optical Flow,
AI(17), No. 1-3, August 1981, pp. 185-203.
Elsevier DOI BibRef 8108
Earlier: DARPA81(144-156). BibRef
And: MIT AI Memo-572, April 1980.
WWW Link. Optical Flow. The standard reference for original optical flow equation computations. The exact formulations are not quite right since they only work in special cases. BibRef

Schunck, B.G., Horn, B.K.P.,
Constraints on Optical Flow Computation,
PRIP81(205-210). BibRef 8100

Horn, B.K.P.[Berthold K.P.], Schunck, B.G.,
Determining Optical Flow: A Retrospective,
AI(59), No. 1-2, January 1993, pp. 81-87.
Elsevier DOI Original paper important because it started the variational approach to optical flow and other vision problems. BibRef 9301

Willick, D.[Darryl], Yang, Y.H.[Yee-Hong],
Experimental Evaluation of Motion Constraint Equations,
CVGIP(54), No. 2, September 1991, pp. 206-214.
Elsevier DOI Evaluate
See also Determining Optical Flow.
See also Motion Constraint Equation for Optical Flow, The. and
See also On a Constraint Equation for the Estimation of Displacement Rates in Image Sequences. constraint equations and conclude that the original was the best for optical flow. BibRef 9109

Longuet-Higgins, H.C., and Prazdny, K.,
The Interpretation of a Moving Retinal Image,
RoyalP(B-208), 1980, pp. 385-397. Optical Flow. An early formulation of the flow pattern with rotation and translation.
See also Multiple Interpretations of a Pair of Images of a Surface. BibRef 8000

Ullman, S.,
The Interpretation of Visual Motion,
Cambridge: MIT Press1979. BibRef 7900 Ph.D.Thesis (EE). His thesis as a BibRef Book Relaxation. A network of points are generated for each image with a relaxation based matching scheme applied to find the 1-1 mapping between the views.
See also Interpretation of Structure from Motion, The. BibRef

Ullman, S.,
The Optical Flow of Planar Surfaces,
SV(1), 1986, pp. 263-276. BibRef 8600
And: MIT AI Memo-870, December 1985. BibRef

Ullman, S.,
Against Direct Perception,
MIT AI Memo-574, March 1980. BibRef 8003

Hildreth, E.C.[Ellen C.],
Computations Underlying the Measurement of Visual Motion,
AI(23), No. 3, August 1984, pp. 309-354.
Elsevier DOI BibRef 8408
And: IU8799-146). BibRef
And: MIT AI Memo-761, March 1984. BibRef
Earlier:
The Measurement of Visual Motion,
Cambridge: MIT Press1983. BibRef Book BibRef
And: Add A2: Ullman, S.[Shimon], MIT AI Memo-699, December 1982. BibRef

Hildreth, E.C.,
The Computation of the Velocity Field,
RoyalP(B-221), 1984, pp. 189-220. BibRef 8400
And: MIT AI Memo-734, September 1983.
See also Computing the Velocity Field along Contours. BibRef

Hildreth, E.C.[Ellen C.], Koch, C.[Christof],
The Analysis of Visual Motion: From Computational Theory to Neuronal Mechanisms,
MIT AI Memo-919, December 1986.
WWW Link. BibRef 8612

Mitiche, A., and Aggarwal, J.K.,
A Computational Analysis of Time-Varying Images,
HPRIP86(311-332). Survey, Motion. Motion, Survey. BibRef 8600

Jacobson, L.[Lowell], Wechsler, H.[Harry],
Derivation of Optical Flow Using a Spatiotemporal-Frequency Approach,
CVGIP(38), No. 1, April 1987, pp. 29-65.
Elsevier DOI Survey, Motion. Motion, Survey. The approach includes Hildreth and Schunck. The paper has a nice survey of techniques and a lot of equations. There may be something here if you want optical flow. BibRef 8704

Jacobson, L.[Lowell], Wechsler, H.[Harry],
A Theory for Invariant Object Recognition in the Frontoparallel Plane,
PAMI(6), No. 3, May 1984, pp. 325-331. BibRef 8405
And:
A Paradigm for Invariant Object Recognition of Brightness, Optical Flow and Binocular Disparity Images,
PRL(1), No. 1, October 1982, pp. 61-68. BibRef

Nagel, H.H.[Hans-Hellmut],
On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results,
AI(33), No. 3, November 1987, pp. 299-324.
Elsevier DOI Optical Flow. A unifying approach to optical flow. Nagel (
See also Displacement Vectors Derived from Second-Order Intensity Variations in Image Sequences. ), Haralick-Lee (
See also Facet Approach to Optic Flow, The. ), Tretiak-Pastor (
See also Velocity Estimation from Image Sequences with Second Order Differential Operators. ), Hildreth (
See also Computations Underlying the Measurement of Visual Motion. ). BibRef 8711

Werkhoveh, P., Toet, A., and Koenderink, J.J.,
Displacement Estimates Through Adaptive Affinities,
PAMI(12), No. 7, July 1990, pp. 658-663.
IEEE DOI Replace iterative approach of
See also On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results. with a noniterative scheme. BibRef 9007

Horn, B.K.P.,
Motion Fields Are Hardly Ever Ambiguous,
IJCV(1), No. 3, October 1987, pp. 239-258.
Springer DOI The cases where a flow field can be ambiguous are difficult to construct and thus are not a major concern for the solution. BibRef 8710

Negahdaripour, S.,
Critical Surface Pairs and Triplets,
IJCV(3), No. 4, November 1989, pp. 293-312.
Springer DOI Where can the field have multiple interpretations. At most, for a curved surface, it is three interpretations. BibRef 8911

Negahdaripour, S.[Shahriar],
Multiple Interpretations of the Shape and Motion of Objects from Two Perspective Images,
PAMI(12), No. 11, November 1990, pp. 1025-1039.
IEEE DOI BibRef 9011
Earlier:
Ambiguities of a Motion Field,
ICCV87(607-611). BibRef
And: MIT AI Memo-940, January 1987. Cases with ambiguous perspective motion fields are limited with know flow for all points on the surface. BibRef

Singh, A., and Allen, P.K.,
Image-Flow Computation: An Estimation-Theoretic Framework and a Unified Perspective,
CVGIP(56), No. 2, September 1992, pp. 152-177.
Elsevier DOI Two categories: conservation information and neighborhood information. BibRef 9209

Singh, A.,
Incremental Estimation of Image-Flow Using a Kalman Filter,
JVCIR(3), 1992, pp. 39-57. BibRef 9200
Earlier: Motion91(36-43). BibRef

Singh, A.,
An Estimation-Theoretic Framework for Image-Flow Computation,
ICCV90(168-177).
IEEE DOI BibRef 9000
And: DARPA90(314-328). Kalman Filter. Generate the depths from a spatio-temporal sequence. BibRef

Singh, A.,
Image-Flow Computation: An Estimation-Theoretic Framework, Unification and Integration,
MVA(4), 1991, pp. 55. BibRef 9100

Singh, A.,
Robust Computation of Image-Motion and Scene Depth,
CRA91(2730-2737). BibRef 9100

Singh, A.,
Optic Flow Computation: A Unified Perspective,
IEEE_Press1990. BibRef 9000
And: From the thesis:
Image-Flow Computation: Estimation-Theoretic Framework, Unification and Integration,
Ph.D.Thesis (CS), Columbia, Univ., May 1990. BibRef

Ioka, M., Kurokawa, M.,
Estimation Of Motion Vectors And Their Application To Scene Retrieval,
MVA(7), No. 3, 1994, pp. 199-208. BibRef 9400

Fitzpatrick, J.M.[J. Michael],
The Existence of Geometrical Density-Image Transformations Corresponding to Object Motion,
CVGIP(44), No. 2, November 1988, pp. 155-174.
Elsevier DOI Geometrical image transformation is identical to change in density image produced by motion of the object. (Primarily medical imagery.) BibRef 8811

Fitzpatrick, J.M.[J. Michael],
A Method for Calculating Velocity in Time Dependent Images Based on the Continuity Equation,
CVPR85(78-81). (Vanderbilt Univ.) CT or X-ray data is preferred, more equations than results. BibRef 8500

Sozou, P.D., and Loizou, G.,
New Perspectives on Optical-Flow,
PR(26), No. 8, August 1993, pp. 1125-1136.
Elsevier DOI Non-uniform medium. Refractive index changes the computation. BibRef 9308

Arnspang, J.[Jens],
Motion Constraint Equations Based on Constant Image Irradiance,
IVC(11), No. 9, November 1993, pp. 577-587.
Elsevier DOI BibRef 9311

Malladi, R., Sethian, J.A.,
Image-Processing: Flows under Min/Max Curvature and Mean-Curvature,
GMIP(58), No. 2, March 1996, pp. 127-141. Level Set Methods. BibRef 9603

Malladi, R., Sethian, J.A.,
Flows under Min/Max Curvature Flow and Mean Curvature: Applications in Image Processing,
ECCV96(I:251-262).
Springer DOI Image enhancement, noise suppression. BibRef 9600

Heikkonen, J.,
A Computer Vision Approach to Air-Flow Analysis,
PRL(17), No. 4, April 4 1996, pp. 369-385. 9605
BibRef

Ma, J., Lu, X., Wu, C.,
A Motion Constraint Equation under Space-Varying or Time-Varying Illumination,
PRL(5), 1987, pp. 203-205. BibRef 8700

Zanker, J.M.,
Second-Order Motion Perception in the Peripheral Visual-Field,
JOSA-A(14), No. 7, July 1997, pp. 1385-1392. 9708
BibRef

Brodský, T., Fermüller, C., Aloimonos, Y.,
Directions of Motion Fields Are Hardly Ever Ambiguous,
IJCV(26), No. 1, January 1998, pp. 5-24.
DOI Link 9804
BibRef
Earlier: ECCV96(II:119-128).
Springer DOI BibRef
And: UMDTR3501, 1995.
WWW Link. BibRef

Gros, B.L., Blake, R., Hiris, E.,
Anisotropies in Visual Motion Perception: A Fresh Look,
JOSA-A(15), No. 8, August 1998, pp. 2003-2011. 9808
BibRef

Åström, K.[Kalle], Heyden, A.[Anders],
Continuous Time Matching Constraints for Image Streams,
IJCV(28), No. 1, June 1998, pp. 85-96.
DOI Link 9807
Multilinear constraints for optical flow.
See also Simplifications of Multilinear Forms for Sequences of Images. BibRef

Mitiche, A., Mansouri, A.R.,
On convergence of the Horn and Schunck optical-flow estimation method,
IP(13), No. 6, June 2004, pp. 848-852.
IEEE DOI 0406

See also Determining Optical Flow. Analyze the equations to prove convergence via both the Jacobi and the Gauss-Seidel methods. BibRef

Bayerl, P.[Pierre], Neumann, H.[Heiko],
Disambiguating Visual Motion by Form-Motion Interaction: A Computational Model,
IJCV(72), No. 1, April 2007, pp. 27-45.
Springer DOI 0001
BibRef
Earlier:
Neural Mechanisms of Visual Flow Integration and Segregation: Insights from the Pinna-Brelstaff Illusion and Variations of It,
BMCV02(301 ff.).
Springer DOI 0303
Computational model of neural mechanisms for visual flow. BibRef

Beck, C.[Cornelia], Gottbehuet, T.[Thomas], Neumann, H.[Heiko],
Integration of Multiple Temporal and Spatial Scales for Robust Optic Flow Estimation in a Biologically Inspired Algorithm,
CAIP07(53-60).
Springer DOI 0708
BibRef

Beck, C.[Cornelia], Bayerl, P.[Pierre], Neumann, H.[Heiko],
Optic Flow Integration at Multiple Spatial Frequencies: Neural Mechanism and Algorithm,
ISVC06(I: 741-750).
Springer DOI 0611
BibRef

Bayerl, P.[Pierre], Neumann, H.[Heiko],
A Fast Biologically Inspired Algorithm for Recurrent Motion Estimation,
PAMI(29), No. 2, February 2007, pp. 246-260.
IEEE DOI 0701
Sparse coding framework to implement the method. BibRef

Meinhardt-Llopis, E.[Enric], Sánchez Pérez, J.[Javier], Kondermann, D.[Daniel],
Horn-Schunck Optical Flow with a Multi-Scale Strategy,
IPOL(2012), No. 2012, pp. xx-yy.
DOI Link 1309
Code, Optical Flow.
See also Determining Optical Flow. BibRef

Le Tarnec, L., Destrempes, F., Cloutier, G., Garcia, D.,
A Proof of Convergence of the Horn-Schunck Optical Flow Algorithm in Arbitrary Dimension,
SIIMS(7), No. 1, 2014, pp. 277-293.
DOI Link 1404

See also Determining Optical Flow. BibRef

Fortun, D.[Denis], Bouthemy, P.[Patrick], Kervrann, C.[Charles],
Optical flow modeling and computation: A survey,
CVIU(134), No. 1, 2015, pp. 1-21.
Elsevier DOI 1504
Survey, Optical Flow. Optical flow BibRef

Fortun, D.[Denis], Bouthemy, P.[Patrick], Kervrann, C.[Charles],
A Variational Aggregation Framework for Patch-Based Optical Flow Estimation,
JMIV(56), No. 2, October 2016, pp. 280-299.
Springer DOI 1609
BibRef
Earlier:
Sparse Aggregation Framework for Optical Flow Estimation,
SSVM15(323-334).
Springer DOI 1506
BibRef

Zhu, B.[Bin], Tian, L.F.[Lian-Fang], Du, Q.L.[Qi-Liang], Wu, Q.X.[Qiu-Xia], Sahl, F.Z.[Farisi Zeyad], Yeboah, Y.[Yao],
Adaptive dual fractional-order variational optical flow model for motion estimation,
IET-CV(13), No. 3, April 2019, pp. 277-284.
DOI Link 1904
BibRef

Bao, W., Zhang, X., Chen, L., Gao, Z.,
KalmanFlow 2.0: Efficient Video Optical Flow Estimation via Context-Aware Kalman Filtering,
IP(28), No. 9, Sep. 2019, pp. 4233-4246.
IEEE DOI 1908
BibRef
Earlier:
KalmanFlow: Efficient Kalman Filtering for Video Optical Flow,
ICIP18(3343-3347)
IEEE DOI 1809
image sequences, Kalman filters, motion estimation, video signal processing, KalmanFlow 2.0, convolutional neural networks. Estimation, Coherence, Optical imaging, Noise measurement, Adaptive optics, Optical filters, time-variant system BibRef

Zhai, M.L.[Ming-Liang], Xiang, X.Z.[Xue-Zhi], Lv, N.[Ning], Kong, X.D.[Xiang-Dong],
Optical flow and scene flow estimation: A survey,
PR(114), 2021, pp. 107861.
Elsevier DOI 2103
Survey, Optical Flow. Motion analysis, Optical flow, Scene flow, Variational model, Deep learning, Convolutional neural networks (CNNs) BibRef

Güney, F.[Fatma], Sevilla-Lara, L.[Laura], Sun, D.[Deqing], Wulff, J.[Jonas],
'What Is Optical Flow For?': Workshop Results and Summary,
OpticalFlow18(VI:731-739).
Springer DOI 1905
BibRef

Zikic, D.[Darko], Kamen, A.[Ali], Navab, N.[Nassir],
Revisiting Horn and Schunck: Interpretation as Gauss-newton Optimisation,
BMVC10(xx-yy).
HTML Version. 1009

See also Determining Optical Flow. BibRef

Govindu, V.M.[Venu Madhav],
Revisiting the Brightness Constraint: Probabilistic Formulation and Algorithms,
ECCV06(III: 177-188).
Springer DOI 0608
BibRef

Giaccone, P.R., Jones, G.A.,
Spatio-Temporal Approaches to Computation of Optical Flow,
BMVC97(xx-yy).
HTML Version. 0209
BibRef

Randriantsoa, A., Berthoumieu, Y.,
Optical Flow Estimation Using Forward-backward Constraint Equation,
ICIP00(Vol II: 578-581).
IEEE DOI 0008
BibRef

Iu, S.L.[Siu-Leong], Lin, Y.T.[Yun-Ting],
Re-examining the Optical Flow Constraint: A New Optical Flow Algorithm with Outlier rejection,
ICIP99(III:727-731).
IEEE Abstract. BibRef 9900

Moons, T., Pauwels, E.J., Van Gool, L.J., and Oosterlinck, A.,
Towards a General Framework for Feature Extraction,
CVPR92(865-868).
IEEE DOI Merge optical flow and recognition. BibRef 9200

Chapter on Optical Flow Field Computations and Use continues in
Optical Flow Field Computation and Analysis .