For decades, we’ve imagined a digital world where we can experience the physical world in all its three-dimensional glory, but until recently, making that happen was a huge challenge. While we were able to communicate with others through calls, videos, and photos, these experiences were limited to a two-dimensional representation of reality. We have always wanted more – the ability to see people, things and places in three dimensions, to immerse ourselves in the world around us. However, accurately reconstructing scenes and objects in 3D has been a complex and challenging task, requiring significant advances in technology and computational methods.
Accurate 3D scene and object reconstruction is a crucial problem in various fields such as robotics, photogrammetry, AR/VR, etc. They can synthesize new renderings with perfect precision using a 3D representation where every location in space can emit radiation. The impressive results of NeRF have attracted attention in the literature, and there have been many attempts to improve its performance.
Most of the work has focused on improving NeRF in terms of image quality, robustness, training speed, and rendering speed. Though, there is a problem with this business; Almost all of them focus on optimizing NeRF for the new view synthesis (NVS) task. Therefore, we cannot use them to obtain accurate 3D grids of radiation fields, which is why we cannot directly integrate NeRF with most computer graphics pipelines.
🚀 Join the fastest ML Subreddit community
What if we wanted to extract geometrically accurate networks from NeRFs so we could actually integrate them into computer graphics pipelines? How can we extract precise 3D networks from NeRFs? Good time to meet NeRFMeshing.
NeRFMeshing It is designed to efficiently extract geometrically accurate networks from trained NeRF based networks. It can produce 3D meshes with precise geometry that can be rendered in real time on commodity hardware.
NeRFMeshing It is built on top of trained NeRF networks by introducing a new architecture called signed surface approximation network (SSAN). SSAN acts as a post-processing pipeline that defines the surface and appearance of a NeRF presentation. It generates an accurate 3D triangular grid for the scene and uses a small appearance grid to generate width-dependent colors. NeRFMeshing It is compatible with any NeRF and allows easy integration of new developments, such as better handling of unlimited scenes or reflective objects.
SSAN calculates both the distance traveled field (TSDF) and the feature appearance field. Through the use of NeRF estimated geometry and training perspectives, the trained NeRF is distilled into the SSAN model. The 3D mesh is then extracted from the SSAN and can be rendered on embedded devices using rasterization and appearance mesh at a high frame rate. This method is very flexible, allowing for rapid 3D mesh creation of not only object-centered scenes, but also the ability to model complex surfaces.
NeRFMeshing is a new method for capturing precise 3D networks of NeRFs. It can be integrated into any existing NeRF network, enabling developments in NeRF to be used with it. With this breakthrough, we can now extract precise 3D networks from NeRFs, which can be used in different fields such as AR/VR, robotics, and photogrammetry.
scan the paper. Don’t forget to join 19k+ML Sub RedditAnd discord channelAnd Email newsletter, where we share the latest AI research news, cool AI projects, and more. If you have any questions regarding the above article or if we’ve missed anything, feel free to email us at Asif@marktechpost.com
🚀 Check out 100’s AI Tools in the AI Tools Club
Ekrem Cetinkaya has a Bachelor’s degree. in 2018 and MA. in 2019 from Ozyegin University, Istanbul, Türkiye. He wrote his master’s degree. Thesis on image noise reduction using deep convolutional networks. He is currently pursuing his Ph.D. degree at the University of Klagenfurt, Austria, and works as a researcher on the ATHENA project. His research interests include deep learning, computer vision, and multimedia networks.
🚀 Join the fastest ML Subreddit community