Video Object Co-Segmentation

Video object co-segmentation refers to the problem of simultaneously segmenting multiple videos that have common objects.

The primary focus of this project is to address the following challenges of video segmentation in a co-segmentation setting:

1) Extraneous objects happen to be moving together in some videos;

2) Motion segmentation issue caused by the foreground object undergoing non-rigid motion such as articulated motion;

3) Interacting multiple objects to be regarded as a single foreground entity;

4) Segmentation issue caused by cluttered background or variegated appearance of foreground;

5) Different objects sharing some common parts.

Departing from the objectness attributes and motion coherence used by traditional figure-ground separation methods, the proposed framework in this project places central importance on the role of “common fate”, that is, the different parts of the foreground should persist together in all the videos. Our main contribution lies in the extraction of these segments sharing “common fates” and enforcing these “common fates” constraints during the clustering step that groups segments together.

We also collect a dataset, entitled CFViCS for Complex Foreground Video Co-Segmentation. It comprises videos that manifest the aforementioned foreground segmentation challenges.


The database is organized in terms of video sets, each defined by the common content shared by the videos in the set. Each video set contains 2-3 videos. Each video is given in terms of its individual image frames named “%3d.png”. The corresponding annotated ground truth is given under the gt folder with names “gt_%3d.png”.

CFViCS database (rar file) – 69.5Mb

Work based on the dataset should cite our ACCV 2014 paper: Consistent Foreground Co-segmentation.