Semantic Segmentation of Point Clouds using Semi Supervised Transfer Learning

Semantic Segmentation of Point Clouds using Semi Supervised Transfer Learning

DFG-Graduiertenkolleg i.c.sens; Torben Peters

Many state of the art solutions in the field of artificial intelligence are based on Deep learning. In autonomous driving, Deep learning is used e.g. for motion planning, object classification or even end-to-end learning. In classical supervised learning a convolutional neural network (CNN) is trained with much data of a specific domain for the given task. However if one domain intersects with another domain the knowledge can be transferred to another task. This procedure is therefore called transfer learning or relational knowledge transfer. In contrast to supervised learning there is no need to label data by humans. For example the knowledge of image classification can be applied to Lidar classification or vice versa. One aim of this project is to examine the capabilities of transfer learning in 3D semantic segmentation.

Semantic segmentation in 3d describes a point wise classification of point clouds. We think this task is challenging because on one side it is hard for humans to annotate the necessary data, because objects may appear ambiguous and labelling in 3d can be time consuming. On the other hand it appears that there is still no preferred way of how the data should be processed in order to use it with deep neural networks.

We believe that laserscanner scan strips have good properties in order to do fast and accurate semantic segmentation of point clouds. This comes from the fact that they have a dense representation of 3d-point clouds without loosing to much spatial information. We tried to circumvent the human labelling process by exploiting the geometric correspondence between camera pixels and 3d points. Our goals are to create a dataset and train a neural network for 3d point cloud classification in scan strips

The figure above shows a part of Nienburger Straße scanned with our Mobile Mapping system and analyzed with the proposed approach. The different colours indicate different classes. In order to classify this point cloud nearly no human annotation was involved.