Snap Angle Prediction for 360° Panoramas

Bo Xiong 1                         Kristen Grauman 1,2

1The University of Texas at Austin     2Facebook AI Research

[paper] [dataset]

Abstract

360° panoramas are a rich medium, yet notoriously difficult to visualize in the 2D image plane. We explore how intelligent rotations of a spherical image may enable content-aware projection with fewer perceptible distortions. Whereas existing approaches assume the viewpoint is fixed, intuitively some viewing angles within the sphere preserve high-level objects better than others. To discover the relationship between these optimal snap angles and the spherical panorama‚Äôs content, we develop a reinforcement learning approach for the cubemap projection model. Implemented as a deep recurrent neural network, our method selects a sequence of rotation actions and receives reward for avoiding cube boundaries that overlap with important foreground objects. We show our approach creates more visually pleasing panoramas while using 5x less computation than the baseline.

What is new

Please check our paper!

Overview

Viewing 360° content presents its own challenges. All prior automatic content-based projection methods implicitly assume that the viewpoint of the input 360° image is fixed. We propose to eliminate the fixed viewpoint assumption. Our key insight is that an intelligently chosen viewing angle can immediately lessen distortions, even when followed by a conventional projection approach.

Introduction
Our Approach

Overview:

We formalize our snap angle objective in terms of minimizing the spatial mass of foreground objects near cube edges. We develop a reinforcement learning (RL) approach to infer the optimal snap angle given a 360° panorama. See below for an overview:

Introduction
Results

Examples Cubemaps with Snap Angle Prediction

feature
Downloads

We provide the youtube ids of the 360° videos used in the paper below:

Publication

B. Xiong and K. Grauman. "Snap Angle Prediction for 360° Panoramas". In ECCV, 2018. [bibtex]

@InProceedings{snap-angle-eccv2018,
  author = {B. Xiong and K. Grauman},
  title = {Snap Angle Prediction for 360° Panoramas},
  booktitle = {ECCV},
  month = {September},
  year = {2018}
}