Foreground Focus: Finding Meaningful
Features in Unlabeled Images
Yong Jae Lee and
Kristen Grauman
Summary
We present a method
to automatically discover meaningful features in unlabeled image collections.
Each image is decomposed into semi-local features that describe neighborhood
appearance and geometry. The goal is to determine for each image which of these
parts are most relevant, given the image content in the remainder of the
collection. Our method first computes an initial image-level grouping based on
feature correspondences, and then iteratively refines cluster assignments based
on the evolving intra-cluster pattern of local matches. As a result, the
significance attributed to each feature influences an image’s cluster
membership, while related images in a cluster affect the estimated significance
of their features. We show that this mutual reinforcement of object-level and
feature-level similarity improves unsupervised image clustering, and apply the
technique to automatically discover categories and foreground regions in images
from benchmark datasets.
System Overview
The
images are grouped based on weighted semilocal
feature matchings (a), and then image-specific
feature weights are adjusted based on their contribution in the match relative
to all other intra-cluster images (b). These two processes are iterated (as
denoted by the block arrows in the center) to simultaneously determine
foreground features while improving cluster quality. Dotted arrows denote
images with updated cluster memberships.
Evaluation
We
performed experiments both to analyze the mutual reinforcement of foreground
and clusters, and to compare against existing unsupervised methods. We work
with images from the Caltech-101, because the dataset
provides object segmentations that we need as ground truth to evaluate our
foreground detection. We formed a
four-class (Faces, Dalmatians, Hedgehogs, and Okapi) and 10-class (previous
four plus Leopards, Car side, Cougar face, Guitar, Sunflower, and Wheelchair)
set. For each class, we use the first 50 images.
If
our algorithm correctly identifies the important features, we expect those
features to lie on the foreground objects since that
is what primarily re-occurs in these datasets. To evaluate this, we compare the
feature weights computed by our method with the ground truth list of foreground
features. We quantify accuracy by the percentage of total feature weight in an
image that our method attributes to true foreground features.
As
our method weights foreground features more highly, we also expect a positive
effect on cluster quality. Since we know the true labels of each image, we can
use the F-measure to measure cluster homogeneity.
Publication
Foreground
Focus: Finding Meaningful Features in Unlabeled Images [pdf]
[slides
(ppt)]
Yong Jae Lee and Kristen Grauman
In Proceedings of the 19th British Machine Vision Conference (BMVC), Leeds,
U.K., September 2008.