Deep learning sensor fusion of images and other data

In this PhD we develop new methods to enable deep learning neural networks to use a combination of data of different nature as input. In contrast to classic image processing CNNs, that only take video data as input, we study the possibilities of adding other sensor data to the same neural network architecture, with the aim to yield an improved accuracy. We will develop and compare novel architectures and fusing methods and evaluate it against non-fused baseline image processing CNNs. This technique will be advantageous in many industrial applications, where plenty of non-image data is available. We will demonstrate these sensor fusion methods for a number of real-life industrial application cases, including automatic tool wear monitoring in metal machining and stainless-steel surface inspection during production.