CMPSCI 574/674:
Intelligent Visual Computing: A Neural Network Approach
Instuctor:
Evangelos Kalogerakis (kalo AT cs DOT umass DOT edu)
Location:
Engineering Lab II Room 119
Students can also join remotely - check Moodle for zoom link
TAs:
Dmitry Petrov, Nelson Evbarunegbe, Tuan Ngo, Vikas Thamizharasan
Meeting times:
Tuesdays + Thursdays 5:30-6:45pm ET
Remote Instructor and TA office hours:
Check Moodle for zoom link, office hours, and Piazza link for asynchronous discussions
Course Objectives
This course covers deep learning methods for 3D computer graphics and 3D computer vision. It introduces students to a wide range of problems related to the processing and creation of visual content, including 3D models, scenes, animations, and images. For each problem, the course describes state-of-the-art learning-based solutions based on recent research findings. Topics include 3D deep learning, 3D object recognition, segmentation, correspondences, 3D object and scene reconstruction, image synthesis, 3D shape synthesis, animation, and neural rendering. Students review research papers, and complete 5 programming assignments in Python/PyTorch. Students taking the 674 section will also work on an open-ended course project related to the topics covered in this course.
Course Prerequisites
CMPSCI graduate students: there are no prerequisites for CMPSCI graduate students, although familiarity with probability, statistics, linear algebra, undergraduate computer graphics and machine learning is essential!
Undergrads and others: the course has the following prerequisites:
- Introduction to Computer Graphics (CMPSCI 373) OR Machine Learning (COMPSCI 589) OR Neural Networks (COMPSCI 682) with grade B or better.
This course counts as a CS Elective toward the CMPSCI major (BA/BS) and can satisfy a second or third AI core requirement in the CMPSCI PhD and MSc program. It offers 3 credits.
Class plan
Lecture material will be available asynchronousy. Course work can be done asychronouly. Students can join in-person or watch live lectures over zoom (attendance is optional). There will be office hours with the instructor and the TAs remotely over zoom. Zoom links, lecture notes, recorded videos, programming resources, and assignments are posted on Moodle. Piazza will be used for asynchronous Q & A with the instructor and TAs.Week-by-week outline (tentative)
Week 1: Image, Shape, Scene Representation
Week 2: Introduction to Neural Networks
Week 3: 3D Deep Learning: the multi-view approach
Week 4: 3D Deep Learning: the voxel-based approach
Week 5: 3D Deep Learning: the point-based approach
Week 6: 3D Deep Learning: the mesh-based approach
Week 7: Introduction to Deep Generative Models (GANs, VAEs, Diffusion)
Week 8: 3D generative models: generating views, voxels, points
Week 9: 3D generative models: generating implicits and primitives
Week 10: 3D generative models: generating meshes and scenes
Week 11: Generative models for rigging and animation
Week 12: Neural Photorealistic Rendering
Week 13: Neural Artistic Rendering
Expectations/Requirements
574:
- Complete five assignments
- Research paper presentation
674:
- Complete five assignments
-
A term project + presentation. This should be an implementation of a 3D/4D deep learning algorithm related to the topics of the course. The result of the project should be a prototype with some preliminary results, or could lead to a submission of a research paper in the future.
Grading Criteria
574:
85% assignments
15% research paper presentation
(no exam)
674:
60% assignments
30% project (code, report)
10% project presentation
(no exam)
Scores to letter conversions:
95-100 A
90-94 A-
85-89 B+
80-84 B
75-79 B-
70-74 C+
65-69 C
60-64 C-
55-59 D+
50-54 D
Note: A course grade below C is not a passing grade for graduate students.
Should you take 574 or 674?
Both 574 and 674 sections meet at the same time, same location, and have access to the same lectures and course material. The programming assignments are the same. The difference is that 574 requires students to present an existing research paper towards the end of the course (~15 min presentation), while 674 requires students to develop on their own a deep learning method related to the topics of the course as a final course project. The project can be an implementation of a (small-scale) research idea/prototype, or could lead to a submission of a research paper in the future. The project plan must be first discussed with the instructor (soon after the middle of the semester). Students also present their project work towards the end of the course. As a result, 674 requires more effort and is more targeted for students interested in graphics+vision research.
Accommodation Statement
The University of Massachusetts Amherst is committed to providing an equal educational opportunity for all students. If you have a documented physical, psychological, or learning disability on file with Disability Services (DS), you may be eligible for reasonable academic accommodations to help you succeed in this course. If you have a documented disability that requires an accommodation, please notify me within the first two weeks of the semester so that we may make appropriate arrangements.Academic Honesty Statement
Since the integrity of the academic enterprise of any institution of higher education requires honesty in scholarship and research, academic honesty is required of all students at the University of Massachusetts Amherst. Academic dishonesty is prohibited in all programs of the University. Academic dishonesty includes but is not limited to: cheating, fabrication, plagiarism, and facilitating dishonesty. Appropriate sanctions may be imposed on any student who has committed an act of academic dishonesty. Instructors should take reasonable steps to address academic misconduct. Any person who has reason to believe that a student has committed academic dishonesty should bring such information to the attention of the appropriate course instructor as soon as possible. Instances of academic dishonesty not related to a specific course should be brought to the attention of the appropriate department Head or Chair. Since students are expected to be familiar with this policy and the commonly accepted standards of academic integrity, ignorance of such standards is not normally sufficient evidence of lack of intent (https://www.umass.edu/dean_students/codeofconduct/acadhonesty/).COVID/Other Airborne Diseases Policy
Students who are ill with COVID or other airborne diseases must not attend class in-person, and they will be offered accommodations for missed attendance. Students are also encouraged to wear masks during in-person classes.Equity and Inclusion Statement
We are committed to fostering a culture of diversity and inclusion, where everyone is treated with dignity and respect. This course is for you, regardless of your age, background, citizenship, disability, ethnicity, family status, gender identity, geographical origin, language, military experience, political views, race, religion, sexual orientation, or work experience. We bring different skills to the course and we will all be learning from and with each other.In both live and online settings we all are expected to uphold and promote a welcoming environment for learning. Politeness, kindness, and tolerance are expected at all times. Respect that people have differences of opinion, and work and approach problems differently. Please keep unstructured critique to a minimum and make sure that any criticism is constructive. Try and be aware of your own biases and avoid micro-aggressions. Listen to others and let them participate; ask yourself whether you dominating a conversation and not giving others a chance to contribute. Disruptive behavior is not welcome, and insulting, demeaning, or harassing anyone is unacceptable. We follow the university's guidelines for classroom civility. In particular, we don't tolerate behavior that excludes people in socially marginalized groups. If you feel you have been or are being harassed or made uncomfortable by someone in this class, please contact a member of the course staff immediately, or if you feel uncomfortable doing so, contact the Dean of Students office.
Everyone has the right to be addressed by the name and pronouns that they use for themselves. Students can indicate their preferred/chosen first name and pronouns on SPIRE, which appear on class rosters. Please let the instructor/TAs know what name and pronouns they should use for you if they are not on the roster. A student's chosen name and pronouns are to be respected at all times in the classroom.
Title IX Statement
In accordance with Title IX of the Education Amendments of 1972 that prohibits gender-based discrimination in educational settings that receive federal funds, the University of Massachusetts Amherst is committed to providing a safe learning environment for all students, free from all forms of discrimination, including sexual assault, sexual harassment, domestic violence, dating violence, stalking, and retaliation. This includes interactions in person or online through digital platforms and social media. Title IX also protects against discrimination on the basis of pregnancy, childbirth, false pregnancy, miscarriage, abortion, or related conditions, including recovery. There are resources here on campus to support you. A summary of the available Title IX resources (confidential and non-confidential) can be found at the following link: https://www.umass.edu/titleix/resources. You do not need to make a formal report to access them. If you need immediate support, you are not alone. Free and confidential support is available 24 hours a day / 7 days a week / 365 days a year at the SASA Hotline 413-545-0800.