Object Tracking based on Optical Flow and Depth

In case of object tracking based on a cue such as depth or velocity, tracking will fail when a target cannot be distingished from the other object based on the cue.
We propose the method of object tracking based on the two cues, which are velocity in an image and depth. The overview of our system is shown below.

The advantage of the method based on multiple cues is that even if it cannot detect a target based on one cue, it is able to detect the target from the other cues. However the existing method based on velocity and depth uses depth data to eliminate objects which occlude the target. In this method, the target cannot be tracked based on the depth data when it cannot be distingished from the other objects based on velocity data.
We integrate optical flow and depth to track a target in case tracking will fail based only on optical flow or depth. In each frame, probability of the pixel belonging to the target is calculated to extract the target region in an image based on the prediction of the target retgion and the distribution of the target velocity and target depth. The region which have high probability is detemined to be the target region. Moreover occlusion of the target is detected based on the depth data. We can track the target even if the target is occluded.
In our method, the target region is extracted robustly bacause we integrate optical flow and depth. And the target can be tracked even if the target cannot be separated from the other object using either optical flow or depth.
The sequential image below is an result of tracking. At the beggining of the sequence, a person who walks in a similar velocity to the target accross between the camera and the target. In this case, the target cannot be distingished from him based on the optical flow. In addition, occlusion of the target is detected based on the depth. At the end of the sequence, a person stands at a similar depth to the target. In this case, the target cannot be distingish from the other objects based on depth, but can be distingish based on optical flow.

Result of Tracking (Red:Target，Green:Occluding Objects)
The proposed method is implemented to the image processor "ISHTAR-II" and we realize realtime tracking. The processings in each DSPs are shown by following figure.

Moreover we control the active camera head "PAHTHER-VZ" to pursuit the target. The system configuration is as follows.

Papers

Ryuzo Okada, Yoshiaki Shirai and Jun Miura "Object Tracking based on Optical Flow and Depth", Proc. of IEEE/SICE/RSJ Int. Conf. on MFI, pp.565-571, 1996
Ryuzo Okada, Yoshiaki Shirai and Jun Miura "Object Tracking based on Optical Flow and Depth", Proc. of RWC pp.432-439, 1997

back to SHIRAI Lab's Home Page

back to Object Tracking Page

okada@cv.mech.eng.osaka-u.ac.jp