Christian Haase Schuetz

Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

IEEE Transactions on Intelligent Transportation Systems, 22(3), pp. 1341-1360

This review paper systematically summarizes methodologies and discusses challenges for deep multi-modal object detection and semantic segmentation in autonomous driving, providing an overview of on-board sensors, open datasets, and background information for object detection and semantic segmentation in autonomous driving research. We then summarize the fusion methodologies and discuss challenges and open questions.

Adaptive Stochastic Resonance for Unknown and Variable Input Signals

Scientific Reports, 7(1), 2450

All sensors have a threshold, defined by the smallest signal amplitude that can be detected. The detection of sub-threshold signals, however, is possible by using the principle of stochastic resonance, where noise is added to the input signal so that it randomly exceeds the sensor threshold. The choice of an optimal noise level that maximizes the mutual information between sensor input and output, however, requires knowledge of the input signal, which is not available in most practical applications. Here we demonstrate that the autocorrelation of the sensor output alone is sufficient to find this optimal noise level. Furthermore, we demonstrate numerically and analytically the equivalence of the traditional mutual information approach and our autocorrelation approach for a range of model systems. We furthermore show how the level of added noise can be continuously adapted even to highly variable, unknown input signals via a feedback loop. Finally, we present evidence that adaptive stochastic resonance based on the autocorrelation of the sensor output may be a fundamental principle in neuronal systems.

Iterative Label Improvement: Robust Training by Confidence Based Filtering and Dataset Partitioning

2020 25th International Conference on Pattern Recognition (ICPR), pp. 9483-9490

State-of-the-art, high capacity deep neural networks not only require large amounts of labelled training data, they are also highly susceptible to labelling errors in this data, typically resulting in large efforts and costs and therefore limiting the applicability of deep learning. To alleviate this issue, we propose a novel meta training and labelling scheme that is able to use inexpensive unlabelled data by taking advantage of the generalization power of deep neural networks. We show experimentally that by solely relying on one network architecture and our proposed scheme of combining self-training with pseudo-labels, both label quality and resulting model accuracy, can be improved significantly. Our method achieves state-of-the-art results, while being architecture agnostic and therefore broadly applicable. Compared to other methods dealing with erroneous labels, our approach does neither require another network to be trained, nor does it necessarily need an additional, highly accurate reference label set. Instead of removing samples from a labelled set, our technique uses additional sensor data without the need for manual labelling. Furthermore, our approach can be used for semi-supervised learning.

Estimating Labeling Quality with Deep Object Detectors

2019 IEEE Intelligent Vehicles Symposium (IV), pp. 33-38

Deep Learning methods are widely applied in Robotics and Automated Driving scenarios. The task of perception for Automated Driving in the real world is particularly challenging and requires a sufficient amount of high quality labeled training data for the algorithms to perform well. However, the means of obtaining real world datasets are limited. It is common practice to have human labelers involved at least to some extent. Regardless of whether the process is partially automated or not, these labels never represent perfectly accurate ground-truth. By investigating the recognition performance of a state-of-the-art object detector as a function of the quality of a labeled real world training set, we study the effect of labeling errors of various types and severity. To this end, the given labels are treated as a reference to which synthetic errors are added systematically in order to determine the performance of the object detector if trained on the erroneous dataset.

Taking Advantage of Sensor Modality Specific Properties in Automated Driving

Computer Science in Cars Symposium, ACM German Chapter

Deep Learning Based Multi-modal Perception and Semi-automatic Labelling Algorithms for Automotive Sensor Data

University of Kassel, PhD Thesis

Automated driving has great potential in solving the mobility needs of the future and reducing the amount of traffic accidents. Advances in the environment perception for automated driving are to a large extent driven by deep learning and huge amounts of sensor data. Multiple-sensors are used in automated driving to make use of complementary information to increase performance and reliability of the system. Many different artificial neural network architectures have been proposed, increasing the recognition accuracy. In this work it is shown how deep learning based object detection can be used to fuse measurements from several different kinds of sensors. Multiple methods combining, e.g., lidar and camera are compared and existing research datasets are introduced. Differences between the approaches are discussed and recommendations derived. From those a flexible deep-learning based fusion algorithm is developed and evaluated on real-world data. Even more important than the specific architectural details are the amount and quality of available data needed to properly represent the joint distribution of potential inputs and targets. This thesis thoroughly studies the effect of differences in the labelling quality on deep learning based classification and deep learning based detection. From the results a novel quality estimation algorithm is derived and a method for estimating the maximum reachable accuracy with a given network and dataset as well as the trade-off between increasing labelling effort and reducing gains as approaching this threshold is developed. While hand-labelling vast amounts of real-world data becomes quickly infeasible, semi-automated methods have great potential of enabling labelling relevant datasets at scale. This thesis develops and evaluates means to handle partially labelled and unlabelled data with the aim of providing sufficient data to generalize to real driving scenarios. To this end, algorithms for semi-supervised learning are developed and improved, as well as iterative algorithms to increase the labelling quality of a given labelled dataset.