CS 378H: Honors Machine Vision
Spring 2017

Tues/Thurs 9:30-11:00 am
GDC 5.302

Jump to current week (lectures, assignments)        Course overview        Syllabus        Requirements/Grades        Piazza        Textbook        Deadlines

Instructor: Kristen Grauman
Office location: GDC 4.726
Office hours: Tues 3-4 and by appointment

TA: Dongguang (Nick) You
Office location: GDC 3.818D
Office hours: Tues 3-4 pm, Wed 4-5 pm

TA: Paul Choi
Office hours location: GDC 3rd floor lab (by printers)
Office hours: Mon 2:30-3:30 pm, Thurs 3:30-4:30 pm

Please use Piazza for assignment help.


Course description

This is an advanced intro course in computer vision.   It
is intended for honors upper-level undergraduate students.

Billions of images are on the web---how can you find the ones you are interested in?  How could photo collections on social media be indexed automatically by the people or events they contain?  How could we interact with a computer using natural gestures or facial expressions?  How can a robot identify objects in complex environments, or navigate uncharted territory?  After capturing video with a wearable camera for days on end, how to determine those snapshots worth keeping?  How can we develop augmented reality systems that overlay visualizations relevant to the real-world content in sight, e.g., a menu for the restaurant you just passed on the street, or a field guide entry for the unusual insect you encountered while hiking?

All such questions demand high-level computer visionIn computer vision, the goal is to develop methods that enable a machine to “understand” or analyze images and videos.   In this introductory vision course, we will explore fundamental topics in the field ranging from low-level feature extraction to high-level visual recognition. 

After covering the fundamentals for image processing, grouping, and multiple views, we will emphasize machine learning-based methods.  While we will motivate the concepts from the vision problems, the learning algorithms we will study are also useful tools for other domains in AI and beyond.  Note, due to our emphasis on learning methods (and the time constraints for the syllabus), this course will omit or treat only briefly some core aspects of computer vision, such as multi-view geometry, 3d reconstruction, and tracking.


A high-level summary of the syllabus is as follows:
I. Features and filters: low-level vision
II. Grouping and fitting: mid-level vision
III. Multiple views
IV.  Recognition: high-level vision

The course textbook is:

    Computer Vision: Algorithms and Applications, by Rick Szeliski.

It is freely available online or may be purchased in hardcopy.  Course lecture slides will be posted below and are also a useful reference.

You may also find the following books useful.

Schedule (cumulative to date)

Readings and links

Tues Jan 17
Course intro

Textbook Sec 1.1-1.3

Course requirements

UTCS account setup

Basic Matlab tutorial

Running Matlab at UT
A0 out

See optional Latex info

filters features
Thurs Jan 19
Features and filters Sec 3.1.1-2, 3.2 Linear filters

slides handout
slides ppt

Tues Jan 24
Sec 3.2.3, 4.2

Seam carving paper

Seam carving video
Gradients and edges

slides handout
slides ppt
A0 due Friday Jan 27

A1 out

See optional Latex info

Thurs Jan 26
Sec 3.3.2-4 Binary image analysis

slides handout
slides ppt

Tues Jan 31
Sec 10.5

Texture Synthesis

Texture synthesis by non-parametric sampling, Efros & Leung

Video textures

Style transfer for video

slides handout
slides ppt

Thurs Feb 2
Sec 8.4 (up until 8.4.1)

Motion magnification

Optical flow

slides handout
slides ppt
A1 due Friday Feb 3

grouping Tues Feb 7 Grouping and Fitting Sec 5.2-5.4

Segmentation and clustering algorithms

slides handout
slides ppt

Thurs Feb 9
Sec 5.5 and skim papers below

Predicting Sufficient Annotation Strength for Interactive Segmentation, Jain and Grauman, ICCV 2013 [web]

Click Carving: Segmenting Objects in Video with Point Clicks, Jain and Grauman, HCOMP 2016 [videos]

Multi-Scale Combinatorial Grouping, Arbelaez et al. CVPR 2014
Interactive segmentation
Guest lecture: Suyog Jain

slides handout
slides ppt
A2 out

Tues Feb 14
Sec 4.3.2

Hough Transform line demo

Hough Transform video

Hough Transform circle demo

Clustering wrap-up and Hough transform, part 1

slides handout
slides ppt

Thurs Feb 16

Hough transform, part 2

slides handout
slides ppt

Tues Feb 21
Sec 5.1.1 Deformable contours

slides handout
slides ppt

Thurs Feb 23 Multiple views Sec 4.1 Local invariant features: detection

slides handout
slides ppt
A2 due Friday Feb 24

Tues Feb 28

Local invariant features: description and matching

slides handout
slides ppt

Thurs Mar 2

Sec 2.1.1, 2.1.2, 6.1.1, 6.1.4


slides handout
slides ppt

Tues Mar 7
Sec 3.6.1

HP frames video 1
HP frames video 2
Homography and image warping

slides handout
slides ppt

Thurs Mar 9

Midterm exam in class
1 sheet of notes is allowed
A3 out

Tues Mar 21
Sec 11.1.1, 11.2-11.5 Image warping, stereo part 1

slides handout
slides ppt

11 am faculty candidate talk in vision: Carl Vondrick, MIT, GDC auditorium 

Thurs Mar 23
Sec 11.1.1, 11.2-11.5

Audio camera
Stereo, part 1

slides handout
slides ppt

Thurs Mar 30
Graph cuts stereo matching demo

Middlebury stereo database

DeepStereo, Flynn et al.

Object labeling in RGB-D videos, Lai et al.
Stereo, part 2

slides handout
slides ppt

Tues April 4 Recognition and learning Synthesis Ch 4, 5, 6 (pdf on Canvas)

Szeliski 14.3

Video Google demo by Sivic et al., paper

David Lowe's SIFT and Generalized Hough approach (Lowe, IJCV 2004)
Instance recognition

11 am faculty candidate talk in vision: Phillip Isola, UC Berkeley, GDC auditorium

slides handout
slides ppt
A4 out

Stanford Mobile Visual Search Data Set, Chandrasekhar et al. Instance recognition
part 2

slides handout
slides ppt

Mining for objects
Intro to categories

slides handout
slides ppt

Viola-Jones face detection paper Discriminative classifiers
Face detection

slides handout
slides ppt

Burges SVM tutorial

Dalal-Triggs pedestrian detection paper

Hays-Efros im2gps paper
Discriminative classifiers

slides handout
slides ppt
A5 out

Vondrick et al. Hoggles paper Support vector machines, kernels, HOG

slides handout
slides ppt

Grauman-Darrell Pyramid match kernel paper

Lazebnik et al. Spatial pyramids paper
Pyramid match kernels, Neural networks intro

slides handout
slides ppt

Convolutional neural networks for visual recognition (Stanford)

Krizhevsky et al. Imagenet classification with deep convolutional neural networks paper

Clarifai demo
Convolutional neural networks

slides handout
slides ppt

Relative attributes

Visual recognition with humans in the loop

Fine-grained visual comparisons with local learning

UT Zappos 50K Dataset

WhittleSearch project

Visipedia project

Merlin bird app

slides handout
slides ppt

Course wrap-up

A5 leaderboard


Basic knowledge of probability and linear algebra; data structures, algorithms; programming experience.  Previous experience with image processing will be useful but is not assumed. 

Assignments will consist largely of Matlab programming problems.  There will be a warm-up assignment to get familiar with basic Matlab commands.  We will recommend useful functions to check out per assignment.  However, students are expected to practice and pick up Matlab on their own in order to complete the assignments.  The instructor and TA are happy to help with Matlab issues during office hours. 

If you are unsure if your background is a good match for this course, please come talk to the instructor.

Course requirements
Assignments:  Assignments will be given approximately every two weeks.  The programming problems will provide hands-on experience working with techniques covered in or related to the lectures.  All code and written responses must be completed individually.  Most assignments will take significant time to complete.  Please start early, and use Piazza and/or see us during office hours for help if needed.    Please follow instructions in each assignment carefully regarding what to submit and how to submit it.

Extension policy: If you turn in your assignment late, expect points to be deducted. Extensions will be considered on a case-by-case basis, but in most cases they will not be granted.  The greater the advance notice of a need for an extension, the greater the likelihood of leniency.  For programming assignments, by default, 10 points (out of 100) will be deducted for lateness for each day late.  We will use the submission program timestamp to determine time of submission.  One day late = from 1 minute to 24 hours past the deadline.  Two days late = from 24 hours and 1 minute to 48 hours past the deadline.  We will not accept assignments more than 4 days late, or once solutions have been discussed in class, whichever is sooner.

Exams:  There is an in-class midterm and a comprehensive final exam. 

Participation/attendance:  Regular attendance is expected.  If for whatever reason you are absent, it is your responsibility to find out what you missed that day.  Note that attendance does factor into the final grade.  (See Section II of the UTCS Code of Conduct regarding attendance expectations.) 

General responsibilities:  Beyond the above, your responsibilities in the class are:

Important Dates

Midterm exam: Thursday Mar 9 (in class, date tentative)
Last class meeting: Thursday May 4
Final exam:  Tuesday May 16, 9-12 noon in GDC 1.304 (as set by registrar).  The exam is given during the normal final exam period and will be offered at that time only.

Assignments are due about every two weeks.  The dates below are tentative and are provided to help your planning.  They are subject to minor shifts if the lecture plan needs to be adjusted slightly according to our pace in class. 

Grading Policy

Grades will be determined as follows.  You can check your current grades online using Canvas.

Academic Dishonesty Policy

You are encouraged to discuss the readings and concepts with classmates. However, all written work and code must be your own. All work ideas, quotes, and code fragments that originate from elsewhere must be cited according to standard academic practice.

Students caught cheating will automatically fail the course.  The case will also be reported to the Office of the Dean of Students, which may institute its own disciplinary measures. If in doubt, look at the departmental guidelines and/or ask.

Notice about Students with Disabilities

The University of Texas at Austin provides upon request appropriate academic accommodations for qualified students with disabilities. To determine if you qualify, please contact the Dean of Students at 471-6529; 471-4641 TTY. If they certify your needs, I will work with you to make appropriate arrangements.

Notice about Missed Work Due to Religious Holy Days

A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.

Latex Templates for Assignment Write-ups (Optional)

You may use any tool for preparing assignment write-ups that you like, so long as it is organized and clear.  Typically we ask for a mix of descriptions/explanations as well as embedded figures composed of images and/or plots produced in Matlab.

Below we provide some info about using Overleaf, a free online editor for Latex.  Overleaf provides various Latex templates and compiles your edited .tex files into a pdf automatically.  The basics:

    1) go to overleaf.com
    2) sign up/sign in

    3) click new project on the left

    4) scroll down to "Homework Assignment" and click on "more homework assignment templates"

    5) choose whichever template you feel comfortable with and click "open as template"

    6) start editing

    7) once you are done editing, click "PDF" in the panel above. A pdf file will be generated and downloaded automatically.

Here are instructions about inserting images.

How to position images.

Captioning, scaling, resizing.