COS429 Fall 2015: Computer Vision

Computer Vision, art by kirkh.deviantart.com

Overview:

On your one-minute walk from the coffee machine to your desk each morning, you pass by dozens of scenes – a kitchen, an elevator, your office – and you effortlessly recognize them and perceive their 3D structure. But this one-minute scene-understanding problem has been an open challenge in computer vision, since the field was first established 50 years ago. In this class, we will learn the state-of-the-art algorithms, and study how to build computer systems that automatically understand visual scenes, both inferring the semantics and extracting 3D structure.

This course requires programming experience as well as basic linear algebra. Previous knowledge of visual computing will be helpful.

Assignments:

Schedule:

Many of the following slides are modified from the excellent class notes of similar courses offered by Prof. Fredo Durand, Alexei Efros, Rob Fergus, William Freeman, Thomas Funkhouser, James Hays, Kristen Grauman, Svetlana Lazebnik, Fei-fei Li, Srinivasa Narasimhan, Aude Oliva, Szymon Rusinkiewicz, Silvio Savarese, Steve Seitz, Noah Snavely, Richard Szeliski, Antonio Torralba, Yair Weiss, and Li Zhang. We are extremely grateful to them.

The following schedule is preliminary and subject to change as the term evolves.

W L Date Topic Slide Reading Deadline
1 1 Thu Sep 17 How to save the world? Introduction Sz1
2 2 Tue Sep 22 What is a camera? Image Formulation Sz2
Tue Sep 22 Precept 1 for PS1 (4:30PM-5:30PM Room CS 105) Precept1
3 Thu Sep 24 What is a camera? (Continued) Image Formulation Sz2
3 4 Tue Sep 29 What color is the sky? (UG add/drop deadline without fee) Color Sz2 PS1 due
5 Thu Oct 1 What color is the sky? (Continued)
4 6 Tue Oct 6 Let's make the world simpler. Edges
Tue Oct 6 Precept 2 for PS2 (4:30-5:30PM CS Room 105) precept2
7 Thu Oct 8 Let's make the world simpler. (Continued)
5 8 Tue Oct 13 Life story of a 3D point. Geometry Sz2 Sz7 PS2 due
9 Thu Oct 15 Life story of a 3D point. (Continued)
6 10 Tue Oct 20 The fundamental matrix song Multi-view Geometry Sz2 Sz7 Sz4 Sz6 Sz9
11 Thu Oct 22 Guest Lecture: Harpreet Sawhney (Alignment and Its Applications in Computer Vision) Alignment
7 12 Tue Oct 27 Guest Lecture: Andras Ferencz (Where have all the flowers gone?) Tracking
Thu Oct 27 Precept 3 for PS3 (4:30PM-5:30PM Room CS 105) precept3
13 Thu Oct 29 Guest Lecture: Andras Ferencz (Where have all the flowers gone?)
8 Tue Nov 3 No Class [Fall recess]
Thu Nov 5 No Class [Fall recess]
9 14 Tue Nov 10 Building rome in a day Structure From Motion Sz11 Sz12 PS3 due
15 Thu Nov 12 Building rome in a day (Continued)
10 16 Tue Nov 17 Building rome in a day (Continued)
17 Thu Nov 19 Building rome in a day (Continued)
Fri Nov 20 Precept 4 for PS4 (12:00pm-1:00pm Room CS 105) precept4
11 18 Tue Nov 24 This is how the brain really works Introduction to Marvin
Thu Nov 26 No Class [Thanksgiving recess]
12 Mon Nov 30 Undergraduate deadline last day to drop fall term courses PS4 due
19 Tue Dec 1 Marvin, the Paranoid Android Introduction to Marvin
20 Thu Dec 3 Guest Lecture: Harpreet Sawhney (TBD)
13 21 Tue Dec 8 Marvin, the Paranoid Android (Continued)
Tue Dec 8 Precept 5 for PS5 (4:30pm-5:30pm Room CS 105) precept5
22 Thu Dec 10 What is a chair? Object Detection Sz14
14 23 Tue Dec 15 What is a chair? (Continued)
Tue Dec 15 Precept 6 for Quiz Review (4:30pm-5:30pm Room CS 105) precept6
24 Thu Dec 17 Quiz PS5 due
Fri Dec 18 Precept 7 for Final Project (12:00pm-1:00pm Room CS 105) precept7 Final project proposal due
Jan 12 Final project submission deadline at 5PM

Matlab Workshop:

We will use Matlab as our programming language in this class, which is very simple and you should be able to pick it up in 1-2 hours. Princeton OIT provides Matlab for installation (Please install the latest version with all available toolboxes.) If you have no experience in Matlab, please attend the following Matlab training workshop by Keller Center (Registration Required).

Requirements/Grading:

Grading will be based on our assessment of your understanding of the class material, and will be roughly comprised of:

Communication:

All the course information, announcements and material will be available on this class website. Announcements will also be sent to the class email list COS429_F2015 [the at sign] princeton.edu (make sure you are registered). piazza.com is used as a discussion forum for the class. We encourage students to post questions and remarks about the material and assignments in that forum.

Late Policy:

Assignments are due at 11:59PM on the due date, as determined by the file date of the file upload. TA support will stop 6 hours before the deadline (i.e. at 5:59PM on the deadline day) to avoid last-minute panic attack. Please start working as early as possible! Each assignment is allowed to be late for 12 hours from the deadline (i.e. 11:59AM the day right after the deadline day) without any penalty. Any additional unapproved late submission will be considered as unsubmitted work. The 12-hour late time is for each deadline only, and cannot be carried on to be used in the following ones. Late submission is not allowed for the final project, class presentation and quiz.

Extra Credit Policy:

Most assignments will include an opportunity for "extra credit." Please note that we will not explicitly award extra points to the assignment score for this extra credit -- i.e.,, it is not possible to receive more than 100% on any assignment. However, we will consider extra credit when assigning grades to students near a grade boundary (e.g., it might push a borderline B+ up to an A-, or from A to A+).

Academic Integrity:

All students pledge to adhere to the Honor Code in the conduct of all assignments, quizzes and final projects that take place in class. We took this very seriously. Your submission will be checked both manually by the TAs and automatically by the state-of-the-art computer software. We will use an automatic program to run your code and compare your code with other students' (including both this year and all previous years) and public available implementations (e.g. from Google, Bing, Github), to verify your result and detect plagiarism to make sure there is no cheating. The computer software is very powerful and we will have zero tolerance policy on plagiarism. For more information, please refer to the university policy here.

Collaboration Policy

The COS 429 assignment collaboration policy is derived from that of Princeton's COS 217 ...

Concerning receiving help from others...

Programming is an individual creative process much like composition. You must reach your own understanding of the problem and discover a path to its solution. During this time, discussions with other people are permitted and encouraged. However, when the time comes to write code that solves the problem, such discussions (except with course staff members) are no longer appropriate: the code must be your own work. If you have a question about how to use some feature of C, Unix, etc., you certainly can ask your friends or the teaching assistants, but specific questions about code you have written must be treated more carefully.

For each assignment you must specifically state, in your writeup file, the names of any individuals from whom you received help, and the nature of the help that you received. That includes help from friends, classmates, lab TAs, course staff members, etc.

Do not, under any circumstances, copy another person's code. Incorporating someone else's code into your code in any form is a violation of academic regulations. This includes adapting solutions or partial solutions to assignments from any offering of this course or any other course. There is one exception to the code-sharing rule: You may adapt code from the COS 429 course materials provided that you explain what code you use, and cite its source in your writeup file.

Copying and transforming someone else's code (by rearranging independent code, renaming variables, rewording comments, etc.) is plagiarism. Some inexperienced programmers have the misconception that detecting such plagiarism is difficult. Actually, detecting such plagiarism is quite easy. Not only does such plagiarism quickly identify itself during the grading process, but also we can (and do) use software packages, such as Alex Aiken's renowned MOSS software, for automated help.

If we suspect a student of plagiarism on an assignment, then we will refer the case to the Committee on Discipline. If the Committee on Discipline finds the student guilty of plagiarism, then the standard penalty is automatic failure of the COS 429 course. The Committee on Discipline may impose additional penalties.

Concerning providing help to others...

For each assignment you must specifically state, in your writeup file, the names of any individuals to whom you provided help, and the nature of the help that you provided.

Abetting plagiarism or unauthorized collaboration by "sharing" your code is prohibited. Sharing code in digital form is an especially egregious violation. Do not e-mail your code or make your code available to anyone. Do not share your code with anyone even after the due date/time of the assignment.

You are responsible for keeping your solutions to the COS 429 programming assignments away from prying eyes. If someone else copies your code, we have no way to determine who is the owner and who is the copier; the Committee on Discipline decides. If you are working on a public cluster computer, make sure that you do not leave the computer unattended, and that you delete your local files and logout before leaving.

You should store all of your assignment files in a private directory. You can create a private directory using commands similar to these:

$ mkdir cos429
$ chmod 700 cos429

Concerning electronic communication...

If you have a question or comment that will be helpful to other students, and you need not reveal any parts of your work to express the question or comment properly, then you should post it to the course's Piazza page. One of the course's instructors will reply as soon as possible. We welcome replies from other students, and may "endorse" a student's response instead of composing an instructor's response.

If you have a question or comment that will not be helpful to other students, or if you must reveal parts of your work to express your question or comment adequately, then you should post it privately to the appropriate preceptor on Piazza.

Final note

Please do not publish solutions to programming assignments in a way that could compromise their utility as pedagogical tools. At Princeton, this is a violation of the basic rights, rules and responsibilities of members of the university community.

Books:

There is no textbook for this class. The main reference book and reading assingment will be mainly based on:

Other references:

Vision:

Learning:

Graphical models: