For extracted objects in two cameras respectively, convex hulls have been derived in image coordinates. By transforming all points of the hull to world coordinates, the video to the left could be created. It shows the ongoings of two object extraction algorithms for pedestrians in two different view. The hulls are not further processed or intersected in this example. Screen boundaries are also projected into the plane to see the fields of views of the both cams.
Multi Sensor Data Fusion for Automatic Situation Awareness (SA)
Starting from a description of objects (or targets) in a scene (e.g. baseball players, traffic vehicles, pedestrians) statistical measures are to be derived to obtain an understanding of the interactions and the overall state of the whole scene. The measurement of single objects to compute state vectors for their description already is frequently done with data from multiple sources i.e. sensors.
Joining the data from several sensors can be done with several algorithms on different levels of the process, e.g. fusion can be done at data level where two sensors are observing the same range only in different spectral bands or sensitivities. In other cases, fusion can be done at information or decision level, where multiple sources already delivered sensor data, did a preprocessing, feature extraction, object detection and interpretation. Then the fusion can be done by blending the opinions of several processing lines, based on voting schemes, the outcomes of earlier attempts, the location of the sensors, the degree of belief/conviction, confirmed proofs in favor of or against particular estimations etc.
Multi Camera Fusion in a Multi-Object Complex Scene
Moving object extraction in complex scenes is done by background estimation and subtraction from current camera frames. The thresholded results show moving parts of the images as segment patches.
Having multiple cameras for the same scene, fusion approaches can be tried. Synchronization and spatial registration/calibration of multiple sensors are crucial parts of the process. The observed scene is a real-world university campus, and the cameras are almost 100 meters apart.
The video shows the projection of segment patches of camera 1 (green) and camera 2 (blue) into the ground plane. Overlapping patches are painted red. Since image processing is done in both views, temporal failure of the background estimation can be observed individually (green or blue flashing) for each view. A tracking fusion algorithm can be designed to handle these effects.
