COS429 Fall 2014: Computer Vision

Computer Vision, art by kirkh.deviantart.com

Overview:

On your one-minute walk from the coffee machine to your desk each morning, you pass by dozens of scenes – a kitchen, an elevator, your office – and you effortlessly recognize them and perceive their 3D structure. But this one-minute scene-understanding problem has been an open challenge in computer vision, since the field was first established 50 years ago. In this class, we will learn the state-of-the-art algorithms, and study how to build computer systems that automatically understand visual scenes, both inferring the semantics and extracting 3D structure.

This course requires programming experience as well as basic linear algebra. Previous knowledge of visual computing will be helpful.

Assignments:

Schedule:

Many of the following slides are modified from the excellent class notes of similar courses offered by Prof. Fredo Durand, Alexei Efros, Rob Fergus, William Freeman, Thomas Funkhouser, James Hays, Kristen Grauman, Svetlana Lazebnik, Fei-fei Li, Srinivasa Narasimhan, Aude Oliva, Szymon Rusinkiewicz, Silvio Savarese, Steve Seitz, Noah Snavely, Richard Szeliski, Antonio Torralba, Yair Weiss, and Li Zhang. We are extremely grateful to them.

The following schedule is preliminary and subject to change as the term evolves.

WLDateTopicSlide ReadingDeadline
11Thu Sep 11How to save the world?IntroductionSz1
22Tue Sep 16What is a camera? Accidental pinhole!Image FormulationSz2 Acc
3Thu Sep 18What color is the sky?ColorSz2
Fri Sep 19Precept 1 [Friend Center 006 12:00-1:00]Precept1
34Tue Sep 23Let's test your beer gogglesFilteringSz3
5Thu Sep 25Natural image statisticsStatistics Sz10
46Tue Sep 30Let's make the world simpler.Edges and LinesPS1 due
7Thu Oct 2Where have all the flowers gone? (Andras Ferencz @ Mobileye)Tracking Sz8
58Tue Oct 7Superman vision: revealing invisible changes in the worldMotion Sz8 Wig Mag Mic
9Thu Oct 9Life story of a 3D pointGeometry Sz2 Sz7
Fri Oct 10Precept 2 [CS105 12:20-1:30]Precept2
610Tue Oct 14Life story of a 3D point (continued)
11Thu Oct 16The fundamental matrix songMulti-view Geometry Sz2 Sz7 Sz4 Sz6 Sz9 PS2 due
712Tue Oct 21The fundamental matrix song (continue)
Wed Oct 22Precept 3 [COS105 12:30-13:30]Precept3
13Thu Oct 23Building rome in a dayStructure From Motion Sz11 Sz12
8 Tue Oct 28No class (Fall Recess)
Thu Oct 30No class (Fall Recess)
914Tue Nov 4Where is "the birth of venus"?Image-based ModelingSz13PS3 due
15Thu Nov 6What is a chair?Object DetectionSz14
1016Tue Nov 11Guest lecture (Kevin Zhou @ Siemens)Medical Image Parsing
17Thu Nov 13Guest lecture (Kevin Zhou @ Siemens)Medical Image Parsing
11Mon Nov 17Precept 4 [CS105 12:20-1:30]Precept4
18Tue Nov 18Using the forest to see the treesScene UnderstandingSz14
19Thu Nov 20Using the forest to see the trees (continued)
1220Tue Nov 25The unreasonable effectiveness of big visual dataData-driven Brute-forcePS4 due
Thu Nov 27No class (Thanksgiving Recess)
1321Tue Dec 2RGB-D object recognition (Shuran Song) RGB-D Recognition Sliding Shapes
Wed Dec 3Precept 5 [CS105 12:20-1:30] Precept5
22Thu Dec 4Final project proposal presentationProposal due
1423Tue Dec 9Shape Matching (Prof. Tom Funkhouser) Shape Matching
24Thu Dec 11This is how the brain really works + QuizDeep LearningPS5 due
Tue Jan 13Final report due

Matlab Workshop:

We will use Matlab as our programming language in this class, which is very simple and you should be able to pick it up in 1-2 hours. Princeton OIT provides Matlab for installation (Please install the latest version R2014a with all available toolboxes.) If you have no experience in Matlab, please attend the following Matlab training workshop by Keller Center (Registration Required).

Requirements/Grading:

Grading will be based on our assessment of your understanding of the class material, and will be roughly comprised of:

Final Project:

The final project will allow you to explore in depth a topic covered in class which you found interesting and like to know more about. Projects can be performed in teams of 1-3 students. During the semester we will propose ideas for projects in the problem sets and lectures, and we also encourage the students to come up with their own ideas that entice them. The topic for the final project and its scope should be approved by the class staff. Overall, the final project is comprised of (a) a project proposal, (b) a class presentation for your proposal, and (c) a report (and source code) documenting your work, results and conclusions. You are encouraged to be as creative as possible.

Final Project Report:

We expect the report to be between 2-8 pages. If you have more members in your team, we expect your report to be longer. Your report should be written using the CVPR latex template and compiled into a PDF file. (downloaded from http://www.pamitc.org/cvpr15/files/cvpr2015AuthorKit.zip).

Here is a guideline for the report writing:
1. Define the problem: What is the input? What is the output?
2. Motivation: Why do this? What makes it difficult?
3. Related work: What has been done? What are the problems?
4. Algorithm: What did you try? What are the alternatives? Why you choose to try this? Justify your design decision.
5. Result: Visualization of your result: screen capture, plot, example outputs. Add figures into your report.
6. Evaluation: Does it work? How well it works?
7. Analysis: Why it works or doesn't work? Can we make it better? How?
8. [For 2 or 3 person teams] Contribution division: Who did what?

Communication:

All the course information, announcements and material will be available on this class website. Announcements will also be sent to the class email list COS429_F2014 [the at sign] princeton.edu (make sure you are registered). piazza.com is used as a discussion forum for the class. We encourage students to post questions and remarks about the material and assignments in that forum.

Late Policy:

Assignments are due at 11:59PM on the due date, as determined by the file date of the file upload. Late assignments are marked down 1/4 of the full grade per day. One minute late is the same as one day late. An up to one-week late submission is allowed once, no questions asked, and you can use it at your discretion (don't use it needlessly). Any additional unapproved late submission will be considered as unsubmitted work. Late submission is not allowed for the final project, class presentation and quiz.

Extra Credit Policy:

Most assignments will include an opportunity for "extra credit." Please note that we will not explicitly award extra points to the assignment score for this extra credit -- i.e.,, it is not possible to receive more than 100% on any assignment. However, we will consider extra credit when assigning grades to students near a grade boundary (e.g., it might push a borderline B+ up to an A-, or from A to A+).

Academic Integrity:

All students pledge to adhere to the Honor Code in the conduct of all assignments, quizzes and final projects that take place in class. We took this very seriously. Your submission will be checked both manually by the TAs and automatically by the state-of-the-art computer software. We will use an automatic program to run your code and compare your code with other students' (including both this year and all previous years) and public available implementations (e.g. from Google, Bing, Github), to verify your result and detect plagiarism to make sure there is no cheating. The computer software is very powerful and we will have zero tolerance policy on plagiarism. For more information, please refer to the university policy here.

Collaboration Policy

The COS 429 assignment collaboration policy is derived from that of Princeton's COS 217 ...

Concerning receiving help from others...

Programming is an individual creative process much like composition. You must reach your own understanding of the problem and discover a path to its solution. During this time, discussions with other people are permitted and encouraged. However, when the time comes to write code that solves the problem, such discussions (except with course staff members) are no longer appropriate: the code must be your own work. If you have a question about how to use some feature of C, Unix, etc., you certainly can ask your friends or the teaching assistants, but specific questions about code you have written must be treated more carefully.

For each assignment you must specifically state, in your writeup file, the names of any individuals from whom you received help, and the nature of the help that you received. That includes help from friends, classmates, lab TAs, course staff members, etc.

Do not, under any circumstances, copy another person's code. Incorporating someone else's code into your code in any form is a violation of academic regulations. This includes adapting solutions or partial solutions to assignments from any offering of this course or any other course. There is one exception to the code-sharing rule: You may adapt code from the COS 429 course materials provided that you explain what code you use, and cite its source in your writeup file.

Copying and transforming someone else's code (by rearranging independent code, renaming variables, rewording comments, etc.) is plagiarism. Some inexperienced programmers have the misconception that detecting such plagiarism is difficult. Actually, detecting such plagiarism is quite easy. Not only does such plagiarism quickly identify itself during the grading process, but also we can (and do) use software packages, such as Alex Aiken's renowned MOSS software, for automated help.

If we suspect a student of plagiarism on an assignment, then we will refer the case to the Committee on Discipline. If the Committee on Discipline finds the student guilty of plagiarism, then the standard penalty is automatic failure of the COS 429 course. The Committee on Discipline may impose additional penalties.

Concerning providing help to others...

For each assignment you must specifically state, in your writeup file, the names of any individuals to whom you provided help, and the nature of the help that you provided.

Abetting plagiarism or unauthorized collaboration by "sharing" your code is prohibited. Sharing code in digital form is an especially egregious violation. Do not e-mail your code or make your code available to anyone. Do not share your code with anyone even after the due date/time of the assignment.

You are responsible for keeping your solutions to the COS 429 programming assignments away from prying eyes. If someone else copies your code, we have no way to determine who is the owner and who is the copier; the Committee on Discipline decides. If you are working on a public cluster computer, make sure that you do not leave the computer unattended, and that you delete your local files and logout before leaving.

You should store all of your assignment files in a private directory. You can create a private directory using commands similar to these:

$ mkdir cos429
$ chmod 700 cos429

Concerning electronic communication...

If you have a question or comment that will be helpful to other students, and you need not reveal any parts of your work to express the question or comment properly, then you should post it to the course's Piazza page. One of the course's instructors will reply as soon as possible. We welcome replies from other students, and may "endorse" a student's response instead of composing an instructor's response.

If you have a question or comment that will not be helpful to other students, or if you must reveal parts of your work to express your question or comment adequately, then you should post it privately to the appropriate preceptor on Piazza.

Final note

Please do not publish solutions to programming assignments in a way that could compromise their utility as pedagogical tools. At Princeton, this is a violation of the basic rights, rules and responsibilities of members of the university community.

Books:

Reference Books

There is no textbook for this class. The main reference book and reading assingment will be mainly based on:

Other references:

Vision:

Learning:

Graphical models:

Resources:

Related Courses:

Computer Vision Class at Princeton

By Antonio Torralba at MIT:

By Alyosha Efros at CMU/Berkeley:

By James Hays at Brown:

By Noah Snavely at Cornell:

By Steven Seitz at UW:

By others:

Code and Datasets

Resource collection by Bob Fisher