ZHOU Fei, ZHOU Zhiyuan, ZHANG Yutong, XIE Yuanyuan. Hybrid Scene Representation Method Integrating Neural Radiation Fields and Visual Simultaneous Localization and Mapping[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240316 Citation: ZHOU Fei, ZHOU Zhiyuan, ZHANG Yutong, XIE Yuanyuan. Hybrid Scene Representation Method Integrating Neural Radiation Fields and Visual Simultaneous Localization and Mapping[J]. Journal of Electronics & Information Technology . doi: 10.11999/JEIT240316 ZHOU Fei, ZHOU Zhiyuan, ZHANG Yutong, XIE Yuanyuan. Hybrid Scene Representation Method Integrating Neural Radiation Fields and Visual Simultaneous Localization and Mapping[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240316 Citation: ZHOU Fei, ZHOU Zhiyuan, ZHANG Yutong, XIE Yuanyuan. Hybrid Scene Representation Method Integrating Neural Radiation Fields and Visual Simultaneous Localization and Mapping[J]. Journal of Electronics & Information Technology . doi: 10.11999/JEIT240316 目前，传统显式场景表示的同时定位与地图构建(SLAM)系统对场景进行离散化，不适用于连续性场景重建。该文提出一种基于神经辐射场(NeRF)的混合场景表示的深度相机(RGB-D)SLAM系统，利用扩展显式八叉树符号距离函数(SDF)先验粗略表示场景，并通过多分辨率哈希编码以不同细节级别表示场景，实现场景几何的快速初始化，并使场景几何更易于学习。此外，运用外观颜色分解法，结合视图方向将颜色分解为漫反射颜色和镜面反射颜色，实现光照一致性的重建，使得重建结果更加真实。通过在Replica和TUM RGB-D数据集上进行实验，Replica数据集场景重建完成率达到93.65%，相较于Vox-Fusion定位精度，在Replica数据集上平均领先87.50%，在TUM RGB-D数据集上平均领先81.99%。 同时定位与地图构建系统 /  神经辐射场 /  混合场景表示 / Abstract: Currently, traditional explicit scene representation Simultaneous Localization And Mapping (SLAM) systems discretize the scene and are not suitable for continuous scene reconstruction. A RGB-D SLAM system based on hybrid scene representation of Neural Radiation Fields (NeRF) is proposed in this paper. The extended explicit octree Signed Distance Functions (SDF) prior is used to roughly represent the scene, and multi-resolution hash coding is used to represent the scene with different details levels, enabling fast initialization of scene geometry and making scene geometry easier to learn. In addition, the appearance color decomposition method is used to decompose the color into diffuse reflection color and specular reflection color based on the view direction to achieve reconstruction of lighting consistency, making the reconstruction result more realistic. Through experiments on the Replica and TUM RGB-D dataset, the scene reconstruction completion rate of the Replica dataset reaches 93.65%. Compared with the Vox-Fusion positioning accuracy, it leads on average by 87.50% on the Replica dataset and by 81.99% on the TUM RGB-D dataset. Key words: Simultaneous Localization And Mapping (SLAM) system /  Neural Radiation Fields (NeRF) /  Hybrid scene representation /  Specular reflection

中国科学院电子学研究所, 北京市2702信箱, 邮编：100190

电话：010-58887066 传真：021-64253812 Email： [email protected]

北京仁和汇智信息技术有限公司