Object Tracking, Homographies, and Face Detection

Tracking Objects

This assignment focused on utilizing various forms of Kalman Filters and Particle Filters (implemented without using the cv2.KalmanFilter and cv2.matchTemplate functions). Several examples of each filter are shown and discussed.

Kalman Filters are typically used to track dynamical systems with uncertainty in their measurements by making use of linear quadratic estimation. Although the math for a Kalman filter may seem intimidating, in principle it is merely making use of matrix math with basic physics motion equations while adding steps to account for uncertainty. The Kalman filter is split up into a prediction step that multiplies the estimate by a state transition matrix followed by a measurement update step that considers the measurement and any noise associated with it.

Kalman Filter Equations

The pictures below show a Kalman Filter being utilized to follow a pedestrian as the cross a sidewalk.

Unlike Kalman Filters, Particle Filters are much more versatile for solving high dimensional problems. Particle filters have three phases: Prediction, measurement, and resampling. The prediction phase involves taking a distribution of particles based on prior beliefs (initially random) and the measurement phase compares the sensor model (measurements) with each particle to estimate the likelihood that it matches your location. This likelihood is converted into importance weights for each particle. This is followed by a resampling step where particles are re-distributed around the most likely weights and updated based on a motion model. Resampling is important as it prevents particle depletion and ensures that your particle density better matches a probability density function.

The examples below show how particles are used to track a ball from an animation as well as Mitt Romney from a 2012 presidential debate.

Homographies

This assignment involved utilizing projective geometry, corner detection, perspective imaging, and homographies without the help of powerful openCV functions like: cv2.findHomography, cv2.getPerspectiveTransform, cv2.findFundamentalMat, cv2.warpPerspective, and cv2.goodFeaturesToTrack.

A homography is a matrix that relates rotation and translation between two images. A video was provided with four markers, which had to be identified and then an image was fit within them. Ultimately, this was done with both a single image and a short video. The provided videos had varying amounts of noise and the inserted images/videos shift with the changing perspective of the images thanks to the homography matrix. Both results are shown below.