Localizing and segmenting objects from weakly labeled videos

Abstract

In this thesis, we consider the problem of localizing and segmenting objects in weakly labeled videos. Specifically, we consider two related problems which include adapting object detectors and object segmentation in weakly labeled videos. In the context of object detectors, a video is weakly labeled if we know the presence/absence of an object in a video (or each frame), but we do not know the exact spatial location. In the context of segmenting objects, our concern is to generalize segmentation network based on only a single annotated frame so that it works better for the entire video stream. In addition to weakly labeled videos, we assume access to a set of fully labeled images for the problem of localizing objects. For segmenting objects, we have some other labeled information but those labeled classes are not included in the test set of our data. We incorporate domain adaptation in our framework and adapt the information from the labeled images (source domain) to the weakly labeled videos (target domain). We demonstrate the effectiveness of our proposed approach based on the experimental results on standard benchmark datasets. Our work can also be used for collecting large-scale video datasets for object detection.