人工智慧 | ShowMeAI資訊日報 #2022.06.22 @@@@ 論文：HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video（轉貼）

 

人工智慧 | ShowMeAI資訊日報 #2022.06.22

語言: CN / TW / HK

時間 2022-06-23 03:16:08 ShowMeAI

主題: 人工智慧 機器學習 演算法

引用 : https://www.gushiciku.cn/pl/a1Fu/zh-tw

論文：HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video

論文標題：HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video

論文時間：CVPR 2022

所屬領域：計算機視覺

論文地址：https://arxiv.org/abs/2201.04127

程式碼實現：https://github.com/chungyiweng/humannerf

論文作者：Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, Ira Kemelmacher-Shlizerman

論文簡介：Our method optimizes for a volumetric representation of the person in a canonical T-pose, in concert with a motion field that maps the estimated canonical representation to every frame of the video via backward warps./我們的方法優化了人在標準 T 姿勢中的體積表示，與運動場相一致，該運動場通過向後扭曲將估計的標準表示對映到影片的每一幀。

論文摘要：We introduce a free-viewpoint rendering method -- HumanNeRF -- that works on a given monocular video of a human performing complex body motions, e.g. a video from YouTube. Our method enables pausing the video at any frame and rendering the subject from arbitrary new camera viewpoints or even a full 360-degree camera path for that particular frame and body pose. This task is particularly challenging, as it requires synthesizing photorealistic details of the body, as seen from various camera angles that may not exist in the input video, as well as synthesizing fine details such as cloth folds and facial appearance. Our method optimizes for a volumetric representation of the person in a canonical T-pose, in concert with a motion field that maps the estimated canonical representation to every frame of the video via backward warps. The motion field is decomposed into skeletal rigid and non-rigid motions, produced by deep networks. We show significant performance improvements over prior work, and compelling examples of free-viewpoint renderings from monocular video of moving humans in challenging uncontrolled capture scenarios.

我們介紹了一種自由視點渲染方法 - HumanNeRF - 它適用於人類執行復雜身體運動的給定單目影片，例如：來自 YouTube 的影片。我們的方法可以在任何幀暫停影片，並從任意新的攝像機視點甚至是該特定幀和身體姿勢的完整 360 度攝像機路徑渲染主體。這項任務特別具有挑戰性，因為它需要合成身體的逼真細節，從輸入影片中可能不存在的各種攝像機角度看，以及合成精細的細節，如布料褶皺和麵部外觀。我們的方法優化了典型 T 姿勢中人的體積表示，與運動場相一致，該運動場通過向後扭曲將估計的典型表示對映到影片的每一幀。運動場被分解為由深度網路產生的骨骼剛性和非剛性運動。我們展示了相對於先前工作的顯著效能改進，以及在具有挑戰性的不受控制的捕獲場景中移動人類的單目影片的自由視點渲染示例。