Hololens 2 – Evaluating 3D Mapping and Technical Capabilities

In this study, the technical and 3D mapping capabilities of Hololens 2 was evaluated. The Microsoft Hololens 2 is a head-worn mobile mixed reality device that is capable of mapping its direct environment in real time. It is equipped with different sensors including four visible light tracking cameras and a depth sensor. The 3D map created using these sensor streams can be accessed by research mode. This makes Hololens 2 a powerful tool for mapping an indoor space. In this work, we evaluate the capabilities of Hololens 2 with respect to the task of the 3D indoor mapping, semantic segmentation and 3D modelling as the quality of scanned data highly influences the accuracy of reconstruction and segmentation.
For the mapping quality measure , the distance between depth sensor of Hololens 2 and flat wall surface was varied and point cloud representation of the wall was recorded. There was a variation in point density that was observed attributed to the error in the depth sensor. The semantic segmentation for the capture data using Hololens was benchmarked using deep learning for classes ceiling, wall and floor with an IoU of 89.4, 94.7 and 75.3 respectively. Also the mapping quality error was further observed in the 3D modelling the indoor data. This was benchmarked using the ISPRS indoor benchmark to better understand the completeness, correctness and accuracy in this thesis. 
Description of the figure: Left shows the apparatus to capture the benchmarking data and the fig on right shows the 3D model from this data.