2-Minute Modeling! AI Framework GauHuman: Achieving High-Quality 3D Human Reconstruction and Real-Time Rendering
-
The S-Lab team from Nanyang Technological University and SenseTime recently released an exciting research achievement, introducing a high-efficiency 3D human modeling framework based on Gaussian Splatting, named GauHuman. This framework has achieved significant breakthroughs in rapid reconstruction and real-time rendering, providing an efficient solution for human modeling in the digital domain.
One of the main features of GauHuman is its ability to complete 3D human modeling in a short time. By leveraging Gaussian Splatting technology, the framework can complete modeling from a monocular human video in just 1 to 2 minutes, a speed far ahead of existing solutions. Moreover, GauHuman achieves real-time rendering at up to 189 frames per second, offering users a smoother and more realistic experience.
The application prospects of this framework are also very broad, covering fields such as gaming, film production, and virtual reality. Users only need to provide a monocular human video along with corresponding camera parameters and human motion shape parameters (SMPL) to complete high-quality 3D digital human modeling in a short time. This gives GauHuman enormous potential in the digital creative field, providing creators with more flexible and efficient tools. GauHuman's modeling framework is based on Gaussian Splatting and inspired by previous Human Neural Radiance Fields (Human NeRF). By modeling 3D humans in standard space and then transforming them into target space using Linear Blend Skinning (LBS), GauHuman effectively addresses several challenges present in conventional methods. In terms of optimization algorithms, GauHuman further enhances modeling efficiency and quality through 3D Gaussian sphere initialization, split/clone/merge operations, and pruning techniques.
In experiments, GauHuman was compared on two monocular human datasets (ZJU_MoCap and MonoCap) against multiple advanced 3D human reconstruction methods including NB, AN, AS, and HumanNeRF. The results showed outstanding performance in PSNR, SSIM, and LPIPS metrics, validating its superior capabilities.
The research team acknowledges that while GauHuman has achieved significant results, some challenges remain unsolved—such as extracting human meshes from 3D Gaussians and recovering detailed 3D human structures from monocular videos. However, they express strong confidence in GauHuman's future development and have made the code fully open-source to encourage broader developer participation in advancing research and innovation in this field. Reference Links: