On-line Learning from Multiple Cameras
Recently, combining information from multiple cameras
has shown to be very beneficial for object detection and
tracking. In contrast, the goal of this project is to train detectors
exploiting the vast amount of unlabeled data given by
geometry information of a specific multiple camera setup.
Starting from a small number (as small as one!) of positive training samples,
we apply a co-training strategy in order to generate new
very valuable samples from unlabeled data that could not be
obtained otherwise .
To cope with insufficiently (but correct) detections during the training process, we introduced a multi-camera MIL approach, which clearly improves the robustness during learning. Further, to extend the approach to more than two cameras a straight forward extension is to introduce a more efficient centralized information fusion approach . The approach, although not limited to this application, is applied for learning a person detector on different challenging scenarios. In fact, we can show that even though starting from a very small number of labeled samples we finally obtain a classifier yielding state-of-the-art detection results (for which typically 10,000s of labeled samples are required!).
Demo: Learning from Multiple Cameras
To demonstrate the proposed approach in the following we show two experiments: (I) scene specific on-line co-training and evaluation on the lab scenario. (II) generalizing classifier: on-line training on the lab scenario; evaluation on independent test sets. To play the videos, just click the corresponding images!
For Experiment I the left and the right video show the detection results obtained by evaluating the initial and the finally obtained classifier, respectively. The video in the middle is a visualization of the updates performed during co-training: a red bounding box indicates a negative update, a green bounding box an identified positive update, and a white bounding-box a detection that is not used for updating. Please note, that for initializing the training process only 5 (!) labeled samples were used!
For Experiment II we want to show that a classifier that was trained on our lab scenario also generalizes to different scenarios. For that purpose initialized a classifier as described above, co-trained it from multiple cameras in our lab, and applied the finally obtained classifier to a totally different data set. The detection results for the initial and final classifier, respectively, are shown below.
|Initial Classifier||Initial Process||Final Classifier|
|Initial Classifier||Final Classifier|
Learning Behavior and Final ResultsFirst, we demonstrate the on-line learning behavior of the proposed approaches. For that purpose, we trained an initial classifier using a small number of labeled samples. The classifier was cloned and used to initialize the co-training process for each camera. Later these initial classifiers were updated by co-training. To demonstrate the learning progress, after a pre-defined number of processed training frames we saved the corresponding classifier, which was then evaluated on an independent test sequence.
|Learning Over Time|
|Precision over time||Recall over time|
|Final Results: scene-specific setup|
|Forcourt Scenario||Lab Scenario|
|Final Results: generalized setup|
The data sets used in publications [1-4] can be downloaded below. We will also provide a ground truth for the evaluation sets soon.
- Easy Data Set (just one person)
- Medium Data Set (3-5 persons, used for the experiments)
- Hard Data Set (crowded scene, 5+ persons)
- Centralized Information Fusion for Learning Object Detectors in Multi-Camera Networks (bib) In Proc. Workshop of the Austrian Association for Pattern Recognition, 2010
- Multiple Instance Learning from Multiple Cameras (bib) In Proc. IEEE Workshop on Camera Networks (CVPR), 2010
- Online Learning of Person Detectors by Co-Training from Multiple Cameras (bib) In Multi-Camera Networks, Principles and Applications, pages 313-334, Academic Press, 2009
- Visual On-line Learning in Distributed Camera Networks (bib) In Proc. Int'l Conf. on Distributed Smart Cameras, 2008