On your one-minute walk from the coffee machine to your desk each morning, you pass by dozens of scenes – a kitchen, an elevator, your office – and you effortlessly recognize them and perceive their 3D structure. But this one-minute scene-understanding problem has been an open challenge in computer vision, since the field was first established 50 years ago. In this class, we will learn the state-of-the-art algorithms, and study how to build computer systems that automatically understand visual scenes, both inferring the semantics and extracting 3D structure.
This course requires programming experience as well as basic linear algebra. Previous knowledge of visual computing will be helpful.
Many of the following slides are modified from the excellent class notes of similar courses offered by Prof. Fredo Durand, Alexei Efros, Rob Fergus, William Freeman, Thomas Funkhouser, James Hays, Kristen Grauman, Svetlana Lazebnik, Fei-fei Li, Srinivasa Narasimhan, Aude Oliva, Szymon Rusinkiewicz, Silvio Savarese, Steve Seitz, Noah Snavely, Richard Szeliski, Antonio Torralba, Yair Weiss, and Li Zhang. We are extremely grateful to them.
The following schedule is preliminary and subject to change as the term evolves.
|1||1||Thu Sep 17||How to save the world?||Introduction||Sz1|
|2||2||Tue Sep 22||What is a camera?||Image Formulation||Sz2|
|Tue Sep 22||Precept 1 for PS1 (4:30PM-5:30PM Room CS 105)||Precept1|
|3||Thu Sep 24||What is a camera? (Continued)||Image Formulation||Sz2|
|3||4||Tue Sep 29||What color is the sky? (UG add/drop deadline without fee)||Color||Sz2||PS1 due|
|5||Thu Oct 1||What color is the sky? (Continued)|
|4||6||Tue Oct 6||Let's make the world simpler.||Edges|
|Tue Oct 6||Precept 2 for PS2 (4:30-5:30PM CS Room 105)||precept2|
|7||Thu Oct 8||Let's make the world simpler. (Continued)|
|5||8||Tue Oct 13||Life story of a 3D point.||Geometry||Sz2 Sz7||PS2 due|
|9||Thu Oct 15||Life story of a 3D point. (Continued)|
|6||10||Tue Oct 20||The fundamental matrix song||Multi-view Geometry||Sz2 Sz7 Sz4 Sz6 Sz9|
|11||Thu Oct 22||Guest Lecture: Harpreet Sawhney (Alignment and Its Applications in Computer Vision)||Alignment|
|7||12||Tue Oct 27||Guest Lecture: Andras Ferencz (Where have all the flowers gone?)||Tracking|
|Thu Oct 27||Precept 3 for PS3 (4:30PM-5:30PM Room CS 105)||precept3|
|13||Thu Oct 29||Guest Lecture: Andras Ferencz (Where have all the flowers gone?)|
|8||Tue Nov 3||No Class [Fall recess]|
|Thu Nov 5||No Class [Fall recess]|
|9||14||Tue Nov 10||Building rome in a day||Structure From Motion||Sz11 Sz12||PS3 due|
|15||Thu Nov 12||Building rome in a day (Continued)|
|10||16||Tue Nov 17||Building rome in a day (Continued)|
|17||Thu Nov 19||Building rome in a day (Continued)|
|Fri Nov 20||Precept 4 for PS4 (12:00pm-1:00pm Room CS 105)||precept4|
|11||18||Tue Nov 24||This is how the brain really works||Introduction to Marvin|
|Thu Nov 26||No Class [Thanksgiving recess]|
|12||Mon Nov 30||Undergraduate deadline last day to drop fall term courses||PS4 due|
|19||Tue Dec 1||Marvin, the Paranoid Android||Introduction to Marvin|
|20||Thu Dec 3||Guest Lecture: Harpreet Sawhney (TBD)|
|13||21||Tue Dec 8||Marvin, the Paranoid Android (Continued)|
|Tue Dec 8||Precept 5 for PS5 (4:30pm-5:30pm Room CS 105)||precept5|
|22||Thu Dec 10||What is a chair?||Object Detection||Sz14|
|14||23||Tue Dec 15||What is a chair? (Continued)|
|Tue Dec 15||Precept 6 for Quiz Review (4:30pm-5:30pm Room CS 105)||precept6|
|24||Thu Dec 17||Quiz||PS5 due|
|Fri Dec 18||Precept 7 for Final Project (12:00pm-1:00pm Room CS 105)||precept7||Final project proposal due|
|Jan 12||Final project submission deadline at 5PM|
We will use Matlab as our programming language in this class, which is very simple and you should be able to pick it up in 1-2 hours. Princeton OIT provides Matlab for installation (Please install the latest version with all available toolboxes.) If you have no experience in Matlab, please attend the following Matlab training workshop by Keller Center (Registration Required).
The COS 429 assignment collaboration policy is derived from that of Princeton's COS 217 ...
Programming is an individual creative process much like composition. You must reach your own understanding of the problem and discover a path to its solution. During this time, discussions with other people are permitted and encouraged. However, when the time comes to write code that solves the problem, such discussions (except with course staff members) are no longer appropriate: the code must be your own work. If you have a question about how to use some feature of C, Unix, etc., you certainly can ask your friends or the teaching assistants, but specific questions about code you have written must be treated more carefully.
For each assignment you must specifically state, in your
writeup file, the names of any individuals from whom you received help, and the nature of the help that you received. That includes help from friends, classmates, lab TAs, course staff members, etc.
Do not, under any circumstances, copy another person's code. Incorporating someone else's code into your code in any form is a violation of academic regulations. This includes adapting solutions or partial solutions to assignments from any offering of this course or any other course. There is one exception to the code-sharing rule: You may adapt code from the COS 429 course materials provided that you explain what code you use, and cite its source in your
Copying and transforming someone else's code (by rearranging independent code, renaming variables, rewording comments, etc.) is plagiarism. Some inexperienced programmers have the misconception that detecting such plagiarism is difficult. Actually, detecting such plagiarism is quite easy. Not only does such plagiarism quickly identify itself during the grading process, but also we can (and do) use software packages, such as Alex Aiken's renowned MOSS software, for automated help.
If we suspect a student of plagiarism on an assignment, then we will refer the case to the Committee on Discipline. If the Committee on Discipline finds the student guilty of plagiarism, then the standard penalty is automatic failure of the COS 429 course. The Committee on Discipline may impose additional penalties.
For each assignment you must specifically state, in your
writeup file, the names of any individuals to whom you provided help, and the nature of the help that you provided.
Abetting plagiarism or unauthorized collaboration by "sharing" your code is prohibited. Sharing code in digital form is an especially egregious violation. Do not e-mail your code or make your code available to anyone. Do not share your code with anyone even after the due date/time of the assignment.
You are responsible for keeping your solutions to the COS 429 programming assignments away from prying eyes. If someone else copies your code, we have no way to determine who is the owner and who is the copier; the Committee on Discipline decides. If you are working on a public cluster computer, make sure that you do not leave the computer unattended, and that you delete your local files and logout before leaving.
You should store all of your assignment files in a private directory. You can create a private directory using commands similar to these:
$ mkdir cos429 $ chmod 700 cos429
If you have a question or comment that will be helpful to other students, and you need not reveal any parts of your work to express the question or comment properly, then you should post it to the course's Piazza page. One of the course's instructors will reply as soon as possible. We welcome replies from other students, and may "endorse" a student's response instead of composing an instructor's response.
If you have a question or comment that will not be helpful to other students, or if you must reveal parts of your work to express your question or comment adequately, then you should post it privately to the appropriate preceptor on Piazza.
There is no textbook for this class. The main reference book and reading assingment will be mainly based on: