Eye in the Sky

In the summer term in 2015, Bachelor students in Applied Cognitive and Media Science as well as Applied Computer Science worked within the Lab Course "Eye in the Sky" on the idea of the realization of a 3rd Person Experience. The basic idea of this joint project of the Entertainment Computing Group (Prof. Masuch) and the High-Performance Computing Group (Prof. Krüger) was to setup a drone which follows a user and films her from above while the video is streamed to a head-mounted virtual reality display the person wears. With this technical equipment the students were asked to realize both, the technical implementation of a 3rd Person Perspective and the investigation of psychological and physiological effects of this perspective.

Therefore, the students were divided into two groups. The first group explored different approaches to implement the necessary tracking, steering and streaming algorithms. They had to find a solution for both autonomous tracking, as well as for the transmission of the video data. Therefore, the group decided to develop a prototype using a head-mounted smartphone (HMS) as display. A colour tracking algorithm was implemented in order to allow the drone to recognize and follow the user autonomously. Thus, any coloured object can serve as a marker which is determined in the systems setup routine. The difference between drone position and marker is translated in to control commands. For security reasons a "panic function" was implemented, which forces the drone to land. All communication and calculations including the streaming of the video images captured by the drone’s attached camera is done on an Android smartphone and transmitted using a local Wi-Fi.

The other part of the group conducted a study where subjects solve a variety of easy and complex tasks (walking a curved line, throwing a ball, and so on) while viewing themselves in different perspectives. This study was meant to simulate the basic scenario and to investigate psychological phenomena like Out-of-Body Experience or the feeling of presence in non-virtual environments. In order to parallelize the work of both groups, the study-group developed a mobile camera wagon, which can be used to follow a person, filming her from behind and above. The captured stereoscopic video image is streamed to a HMS.  By this, they tried to simulate the drone as good as possible.

A lot of hard work had to be done in this ambitious project. The participants had to solve sophisticated technological issues and learned much about teamwork, project management and empirical research. The project enables a new field of research concerning the effects of perceiving oneself in 3rd Person Perspective and gives many exciting possibilities for future work and applications.