Neural rendering and reconstruction is the cornerstone of digitizing our world, with several applications in VR/AR, movie and media production, robotics, and many more. The digitization pipeline usually consists of three main stages; data capture, 3D model building, and finally, reconstruction and rendering/synthesis. This course will cover advanced topics in this digitization pipeline, with a focus on data-driven approaches using neural-based formulations. We will cover 3D scene representations, including explicit approaches as well as the more recent learnable implicit-based approaches. We will discuss how to build 3D morphable models for important objects such as the human face and body. We will also cover approaches for both 2D and 3D neural rendering. While the vast majority of the topics will focus on using data captured by RGB cameras, we will also discuss other means of capturing data using advanced sensors such as IMUs and event cameras. Finally, we will discuss quantum visual computing and the impact it can bring to the field.
Computer Vision strives to develop algorithms for understanding, interpreting and reconstructing information about real-world scenes from image and video data. Computer Graphics focuses on image synthesis: algorithms to build and edit static and dynamic virtual worlds and to display them in photorealistic or stylized ways. Machine Learning is concerned with studying and developing algorithms which use statistical models to solve problems by analyzing and drawing inference from data. In recent years, these fields have converged more and more. Both Computer Vision and Computer Graphics create and exploit models describing the visual appearance of objects and scenes, while the most successful models heavily utilize ideas from Machine Learning. In this seminar series, we will cover advanced research topics that cross the boundaries between the fields of Computer Vision, Computer Graphics, and Machine Learning. This seminar will cover research papers from the following topics: