Shape Sharing for Object Segmentation

University of Texas at Austin, Jaechul Kim and Kristen Grauman

[pdf] [code+data] [slides]

    
     (a) Sematically close                                                      (b) Semantically disparate
Figure1. Intuition for shape sharing. While one may expect shape sharing between objects of semantically close categories (a), we observe that similar shapes exist
even among semantically disparate objects (b). This suggests to transfer “object-level” shapes between categories, enabling category-independent shape priors for object segmentation.

Key contributions

Basis of Approach: Shape Trasnfer by Partial Shape Matching

                                                    Figure 2. Basis of approach: Transfer category-indepedent shape prior through partial shape matching.
                                            We see a shape of a bird in the exemplar is transfered to a person of the test image, where we did not assume any category specific knolwedges on those images.

Overview of the Method

Figure 3. Overview of our method. (a) Exemplars (first column) that share shape with the test image (second column) are projected in, no matter their category.
We identify those shared shapes (marked in blue) via local BPLR matches (marked in white). (b) Multiple exemplars that highly overlap are aggregated, to form
a shape prior and color model per each aggregated group. (c) The priors from each group are used to compute a series of graph-cut segmentation hypotheses.

Results

Segmentation accuracy in benchmark datasets

* Note: For both experiments, we use the common PASCAL training dataset for exemplars.
        

When does shape sharing help most? 

Figure 4. Shape Sharing's gain as a function of color  easiness and object size. The gain is compared to CPMC [Carreira and Sminchisescu, PAMI 2011].
As color easinesss is getting lower, our gain increases. In fact, at the leftmost point  in the graph, where color is hardest, we achieve as much as 15 points gain over the baseline.
Also, as object size is getting bigger, our gain increases. This is intuitive because as object size becomes larger, its shape becomes more visable. 

Figure 5. Shape sharing matrix for the 20 classes in PASCAL.
We compute the strength of sharing by counting how many times shape exemplars from one class are used to generate the best segmentation hypotheses for another class. Brighter=higher.

Example results

Figure 6. Example segmentations. Shape sharing is useful when objects consist of diversely-colored parts (1st example). Also, it helps delineate objects from confusing surrounding colors (2nd example).
However, shapes do not help segment tiny objects (3rd example), nor objects lacking shape like truncated sofa in the 4th example.

Code + Data [zip]

Papers