Artificial Intelligence-based Reconstructive Procedures for Environment Modelling

Author: Miller, Markus (Hochschule München)

With the introduction of Head Mounted Devices (HMDs) for Virtual Reality (VR) and Augmented Reality (AR), companies increasingly incorporate those devices in different fields, such as product presentations, product audits, and employer training. A realistic display of the virtual environment in those applications is important for a satisfying user experience, especially in AR applications, in which virtual content should seamlessly merge with real scenes.

In our research project, we are building a deep learning based Artificial Intelligence (AI) to reconstruct illumination parameters, such as the position or direction of a dominant light source, the light colour, or the intensity of the light source, from input images. The derived illumination parameters can then further be used for example in AR applications to dynamically adjust the virtual illumination to match the real illumination situation.

3D-Printed Bunny
Virtual Bunny

As our first step in this research field, we trained a VGG-16 network with synthetic RGB data. The task of the network is to regress the dominant light source direction in azimuth and elevation angles from the RGB input. The IBM OpenPower system aided us in the training process and we really appreciate the help we received from the OpenPower team.

Posted by OpenPower Team