Chung yi weng
Chung yi weng
一• Honors & Fellowships
1• UW Reality Lab Google Fellowship 2022 ~ 2023
2• UW Reality Lab Research Fellow 2018 ~ 2023
3• Bob Bandes Best Teaching Assistant Award Honorable
Mention 2020
4• UW Reality Lab Huawei Fellowship 2018 ~ 2019
5• People’s Choice Award, Allen School Industry Affiliates
Research Day 2018
6• The David Notkin Endowed Graduate Fellowship in Computer Science & Engineering
2015 ~ 2016
7• Best Paper Award, ACM Multimedia Conference 2006 2006
二• Teaching & Services
1• TA Lead, CSE457 Computer Graphics
2• Instructor: Adriana Schulz
3• Awarded as Best Teaching Assistant (Bob Bandes
Award)
University of Washington 2020 Spring
4• TA, CSEP557 Trends in Computer Graphics
Instructor: Brian Curless University of
Washington 2019 Spring
三• Reviewers
1• CVPR, ICCV, SIGGRAPH, SIGGRAPH Asia
四• Press and News
[N1] MIT
Technology Review, “Machine vision can create Harry Potter–style photos
for
muggles”
[N2] NVidia News,
“Transforming Paintings and Photos Into Animations With AI”
[N3] UW News,
“Behind the magic: Making moving photos a reality”
五• Open-Source Software
1• HumanNeRF: Free-Viewpoint Rendering of Moving People
from Monocular Video
o
https://github.com/chungyiweng/humannerf (> 600 stars)
六•2022 Invited Talks
[T1] CSNext,
“HumanNeRF: Free-Viewpoint Rendering of Moving People From Monocular Video”
2022
[T2] Stanford
Graphics Group, “Vid2Actor: Free-viewpoint Animatable Person Synthesis from
Video” 2021
[T3] UW Allen
School Colloquium
七 •Abstract of Phd Chung Yi Weng's dissertation Introduction to doctoral thesis
Reconstructing and producing photorealistic renderings of dynamic humans from RGB images has long been considered a holy grail in the fields of computer vision and graphics. Such a capability would open up a wide range of possibilities for applications in areas such as virtual and augmented reality, teleconferencing, and the entertainment industry. Despite more than 25 years of research and development, the problem remains challenging, primarily due to difficulties posed by inherent 3D-to-2D ambiguity, highly dynamic motions, appearance variance, and non-rigid deformation. Moreover, the high cost of the technology has also been a major barrier to widespread adoption, as the reconstruction pipelines often rely on calibrated multi-camera systems and are typically only found in professional studios. In this thesis, I address the challenge of reconstructing and rendering high-quality dynamic humans using unstructured data in the wild, such as photos from the internet or YouTube videos. The goal is to make this expensive technology more accessible to amateur artists and even the general public, democratizing its use beyond just movie studios. To begin, I provide a review of the literature on this long-established problem, starting with the seminal work of Kanade et al. in 1997 and tracing the evolution of the technology through advances in image-based rendering, surface reconstruction, and more recently, modern deep neural networks. Then I present three novel approaches for tackling this problem, each designed to work with different types of source material, including monocular videos, personal photo collections, and single photographs. Through these approaches, my research enables a range of new applications.
My proposed first approach, Photo Wake-Up, allows for creating 3D human animations viewable on AR devices like HoloLens using only single images.
https://grail.cs.washington.edu/projects/wakeup/
The second method, known as HumanNeRF, enables free-viewpoint rendering of moving persons from a YouTube video.
https://grail.cs.washington.edu/projects/humannerf/
Finally, I present PersonNeRF, an approach that is capable of reconstructing a person, including tennis superstars like Roger Federer, from photo collections, enabling rendering with arbitrary combinations of their viewpoints, appearances, and body poses.
https://grail.cs.washington.edu/projects/personnerf/
In the final section, I discuss the open problems that still exist in this field, as well as how this technology will potentially shape our future world.
一•榮譽與獎學金
1• 華盛頓大學現實實驗室 Google 獎學金
2022 ~ 2023
2• 華盛頓大學現實實驗室研究員 2018 ~ 2023
3• 2020 年鮑伯班德斯最佳助教獎榮譽獎
4• 華盛頓大學現實實驗室華為獎學金 2018 ~ 2019
5• 2018 年艾倫學院產業附屬研究日人民選擇獎
6• David Notkin 電腦科學與工程研究生獎學金 2015 ~ 2016
7• 2006 年 ACM 多媒體會議最佳論文獎 2006
二•教學與服務
1• 助教主管，CSE457 電腦圖形學 講師：阿德里亞娜‧舒爾茨
2• 榮獲最佳助教獎（鮑伯班德斯獎）華盛頓大學 2020 春季
3• TA、CSEP557 電腦圖形學趨勢 講師：布萊恩柯利斯 華盛頓大學
2019 春季
三• 審稿人
CVPR、ICCV、SIGGRAPH、SIGGRAPH 亞洲
四•新聞與新聞
[N1] 麻省理工科技評論，“機器視覺可以為麻瓜製作哈利波特風格的照片”
[N2] NVidia 新聞，“利用 AI 將繪畫和照片轉化為動畫”
[N3] 華盛頓大學新聞，“魔法背後：讓移動照片成為現實”
五•開源軟體
1• HumanNeRF：從單眼影片自由視點渲染移動人物
https://github.com/chungyiweng/ humannerf （> 600 顆星）
六•2022年特邀報告
[T1] CSNext，「HumanNeRF：單眼影片移動人物的自由視點渲染」2022
[T2] 史丹佛圖形集團，「Vid2Actor：從影片合成自由視點動畫人物」2021
[T3] 華盛頓大學大學艾倫分校學術討論會
七 • Chung yi weng 博士論文簡介
從 RGB 影像中重建和生成動態人體的逼真渲染一直被認為是電腦視覺和圖形領域的聖杯。這種能力將為虛擬和擴增實境、電話會議和娛樂業等領域的應用開闢廣泛的可能性。儘管經過 25 多年的研究和開發，該問題仍然具有挑戰性，主要是由於固有的 3D 到 2D 模糊性、高度動態運動、外觀變化和非剛性變形所帶來的困難。此外，該技術的高成本也是其廣泛採用的主要障礙，因為重建流程通常依賴校準的多攝影機系統，並且通常僅在專業工作室中找到。在本論文中，我解決了使用野外非結構化資料（例如來自互聯網或 YouTube 影片的照片）重建和渲染高品質動態人類的挑戰。目標是讓業餘藝術家甚至一般大眾更容易使用這項昂貴的技術，使其用途不僅限於電影製片廠。首先，我對這個長期存在的問題的文獻進行了回顧，從 Kanade 等人的開創性工作開始。於 1997 年提出，並透過基於影像的渲染、表面重建以及最近的現代深度神經網路的進步來追蹤該技術的演變。然後我提出了三種解決這個問題的新方法，每種方法都旨在處理不同類型的來源資料，包括單眼影片、個人照片集和單張照片。透過這些方法，我的研究實現了一系列新的應用。
我提出的第一種方法，Photo Wake-Up，允許僅使用單一影像建立可在 HoloLens 等 AR 裝置上查看的 3D 人體動畫。
https://grail.cs.washington.edu/projects/wakeup/
第二種方法稱為 HumanNeRF，可實現 YouTube 影片中移動人物的自由視點渲染。
https://grail.cs.washington.edu/projects/humannerf/
最後，我介紹了 PersonNeRF，這種方法可以透過照片集重建人物，包括像羅傑費德勒這樣的網球巨星，並能夠以任意組合的視角、外觀和身體姿勢進行渲染。
在最後一部分，我討論了該領域仍然存在的未解決的問題，以及這項技術將如何潛在地塑造我們的未來世界
留言
張貼留言