Recent Projects

We aim to improve primary school children's learning of mathematics through spatial sensorimotor experiences in virtual reality (VR). In school, mathematical structures are predominantly taught cognitively with abstract-symbolic representations. This poses great challenges for many primary school children. However, not only cognitive but also motor and sensory processes play an important role in learning. VR offers new possibilities to connect concrete spatial experiences of one's own body with mathematical phenomena. Playful interaction can also reduce the fear of making mistakes and encourage interest in trying things out. In this way, we want to offer primary school children an additional or alternative way to gain mathematical understanding. We want to explore this with VR-based tasks for number space concepts on the number line and for understanding the place value system on the decimal system. In the user-centred development process children are to be involved in the user-centred development process.  Whether this embodied learning in VR approach will be successful, we want to evaluate in smaller focus group tests in the school to create a basis for further research.  read more ››

We want to determine if we can use digital virtual characters in virtual reality (VR) to improve the detection and diagnosis of depressed children and adolescents in real life.For this purpose, we will develop a game-based virtual reality application that uses emotionally believable virtual characters to learn how to recognize complex mental illnesses. The application will simulate a diagnostic situation where one can experience an anamnesis with a mentally ill person. The initial target groups are medical students, therapists in training, and other persons involved in diagnosis and therapy. Later, a simplified learning module will be made available to youth trainers and recreational sports coaches to improve awareness of the first symptoms of depression or other mental illnesses. Teaching in medicine has evolved into competency-based teaching. The National Competence-Based Learning Objectives Catalogue for Medicine (NKLM) includes clinical-practical skills and medical interviewing as concrete learning objectives. At the University of Duisburg-Essen, this content is taught as a longitudinal curriculum (ECKO). Communication and interaction are also essential to all child and adolescent psychiatry courses. A significant challenge is practical teaching, which requires actual or simulated patients. There are no corresponding simulation patients for children and adolescents up to 16 years of age, so young adults have to be used, which is, at best, an approximation. Video material is not a good alternative due to patients' personal protection needs and the medium's limitations. Here, virtual reality can be a possible solution precisely because of the intense presence experience.Together with our partners PD Dr. Gertraud Gradl-Dietsch from the LVR-Klinikum, we design the application and emphasize the credible design of the virtual agents and motivating (playful) interaction possibilities to offer students the best possible experience. Conveying emotions with digital characters is challenging because humans have a fine sense of other people's appearance, language, facial expressions, and gestures. An uneasiness towards non-real but human-like characters is known in psychology as the "Uncanny Valley Effect." This makes it challenging to portray convincing virtual characters, but recent technical possibilities from computer games show promise (Mavrogiorgou et al., 2021). Such virtual characters, also called agents, are computer-controlled beings that can be virtual humans. They are not avatars representing an actual human in a virtual world. Studies show that people treat virtual characters equally and perceive them as social entities if they are designed to be believable (Harth, 2017; Emmerich et al., 2018). In particular, believability can be enhanced by representing different affective communication channels (Clavel et al., 2009; Ennis et al., 2013; Demeure et al., 2011), which can also lead to virtual characters triggering empathy in their counterparts (Rativa et al., 2020). Thus, emotional virtual characters bring excellent prerequisites for using them to model the complex emotional world of a mentally ill child or adolescent. The development of a youthful character initially for the diagnosis of depression takes into account the high lifetime prevalence of depressive illnesses in adolescence of up to 10% (Avenevoli et al., 2015). Each agent would be the same for each student, which ensures that the content is conveyed in a standardized manner. In addition to the realization and embedding of virtual patients, VR offers the students the feeling of actually interacting with living characters due to the high degree of immersion (Nowak & Biocca, 2003). An application that is available to all students and includes a range of different mentally ill characters can also offer a motivational aspect to encourage regular use by students. Here, playful aspects (e.g., prepared dialogues similar to computer games) could be used, which makes learning with the application an instructive and exciting experience for the students. Game mechanics, such as collecting achievements, can motivate gamers to want to play the game again. Therefore, students would be motivated to learn in addition to the advantages of such a VR application. As additional motivation and for better integration into the curriculum, the VR training could be provided with a certificate that the students can acquire. Our goal for this project is to conduct a comparative study in the winter semester of 2024/25 to test the effectiveness of our application. For this purpose, a group of 10th-semester medical students will practice with the VR application concurrently with their preparation for the "Psychiatry and Psychotherapy" exam. In contrast, a control group will receive no additional preparation. The students are to get to know eight different virtual patients in several sessions spread over the semester and practice the procedure of a diagnostic interview with them. They are to be accompanied by the investigators. In this course, we can also collect measures for assessing the emotional virtual characters. If you are interested in our project or would like to support us, don't hesitate to contact us directly!  read more ››

Past Projects

Digital media open up new potentials in the areas of knowledge transfer, information provision and management, learning support, communication, and entertainment in the care of children and adolescents suffering from cancer, especially with regard to dealing with the psychosocial needs of young patients. The LOUISA project which is funded by the Gert und Susanna Mayer Stiftung aims at creating a patient centered infotainment system which opens up access to a new form of individualized medicine for children suffering from cancer and thus can sustainably improve patient care.  The basic characteristic of the system will focus on offering individualized information and access to learning materials, hospital navigation, and simulations of medical examinations. One key element of the system will be the system-wide integration of a usage concept which is oriented towards motivating aspects of digital games. Thus, we hope to create and sustain a high motivation and thus to enable a long-term use of the system which benefits the young patients as well as the medical staff. To achieve this, our work will focus on creating a motivational concept and implementing a prototype which is evaluated with focus group tests during the whole development process.   read more ››

    Development of a mobile attention training software to support the therapy of attention deficit hyperactivity disorder (ADHD) ADHD is a mental developmental disorder affecting children at an early age. It is characterized by salient inattentiveness, impulsiveness, and/or excessive physical activity, leading to severe problems in patients’ everyday life, particularly regarding educational and social contexts. Current treatment concepts often combine pharmacological and behavioral therapy, but seldomly include digital approaches such as computer-supported cognitive trainings.Our research project MATS aims at developing and evaluating a playful, digital learning and attention training software to support the ADHD therapy of children aged between five and twelve. The application will address the core symptoms of ADHD by transferring and combining approved non-digital methods into an innovative digital concept. It shall support patients’ impulse control and ability to concentrate in order to increase their cognitive performance. By deploying game mechanics known from digital games and creating a playful overall concept, we will address children’s intrinsic motivation and foster their willingness to use the training tool. Our mobile solution (running on smartphones and tablet PCs with iOS, Android, and Amazon Fire OS) will allow the flexible and versatile use in different contexts in children’s daily routine. Moreover, we do not only consider the children’s needs and demands but will also integrate their parents and therapists in our concept. By automatically logging and processing training data, the app enables them to monitor the patients’ activities and progress. This allows valuable insights that can be used to further adapt and customize the individual therapy steps. Furthermore, parents can be addressed by the application as well, for instance by providing information, giving recommendations for action, or including them as co-players.The development of our application will follow an iterative process including rapid prototypes and the evaluation of different game mechanics and styles in user tests. The final prototypical implementation of our concept will be evaluated in a comprehensive user study with the target group to proof its efficiency. Therefore, we will work together closely with a range of therapists to test the application in the field. Overall, the project MATS is meant to result in an approved mobile application that can be integrated into established therapy concepts as an additional tool. We want to support children with ADHD as well as their parents and therapist by supporting the therapy, reducing symptoms, fostering motivation and, thus, increasing their overall quality of life. To reach the project’s goals, we work together with two experienced partners: Meister Cody (formerly part of Kaasa Health) and the TU Dortmund University. The Kaasa Health GmbH is an expert in creating playful digital therapies for children and adults to solve healthcare problems. They have already developed two successful products for children, MeisterCody Talasia (addressing dyscalculia and math weakness) and MeisterCody Namagi (addressing dyslexia). The research group of Prof. Dr. Tobias Kuhn at the TU Dortmund University deals with empirical educational research methods and provides the psychological expertise regarding ADHD therapy approaches.   read more ››

Multi-User Virtual Reality Experience for Retail  Together with our partner, we aim at creating a virtual reality (VR) environment for the presentation, exploration, and live adaptation of retail systems with multiple users. For access to a room-scale VR environment, head-mounted displays will be used. Thus, the envisioned environment provides a collaborative virtual space that addresses specific user needs within the business case at hand. Thereby, the scenario must ensure easy entry, onboarding, and exit processes since it focuses on non-professional VR users. Besides the development of a technological prototype, the Entertainment Computing Group is specifically assigned to develop game-inspired interaction techniques, that enhance the usability and user experience among users. More specifically, we want to identify guidelines, to grant pleasurable multi-user interaction and storytelling in VR. These guidelines shall ensure technology acceptance and user motivation for enthusiasts and inexperienced VR users alike. This project involves setting up a multi-user VR lab to research into general paradigms of multi-user interaction within co-located as well as distributed scenarios. The research includes questions on usability, user experience, and user guidance for the specific VR scenario. The project MUVER is a cooperation with Diebold Nixdorf Systems GmbH.   read more ››

Virtual Reality-System for the Reduction of Children's Anxiety during MRI scans The project VR-RLX focuses the idea of using Virtual Reality technology to reduce stress and anxiety of patients during medical treatments. Distracting patients and shifting their attention to a virtual relaxing world is the key aspect of this research project. Though the medical examination in MRI scanners is harmless many patients, especially children often react with strong defense reactions. Hence, the use of sedatives is a common medication-based approach to relax patients though several risks exist. We believe in an innovative use of entertainment technology for the improvement of the healthcare and the subjective comfort of the patient while reducing potential risks. To achieve this, we joined forces with our partners from medical technology and the entertainment industry to create a VR system including hard- and software tailored to this application. For the use of VR in MRI scanners, this project aims to introduce a special MRI-suitable Head Mounted Display (HMD) that will allow creating an immersive and playful virtual environment, thus a joyful experiences regardless of the oppressing narrowness and deafening sounds. Developing effective strategies to reduce anxiety and stress by identifying and implementing immersive media will be the main task for the core concept of this future-oriented project. Our project has been extended! We gained a lot of exciting results and will now devote another six months to the completion and dissemination of our work.     read more ››

The goal of Therapy Assist is the development of an therapeutic assistance system, which can support and supervise the patients’ independent training at home during all phases of rehabilitation. Playful exercises and direct feedback should increase the patients’ motivation to train, achieving a better and faster recovery. The project combines technical, psychological and interactive components. Especially the technical component will be an extension of earlier models, as it includes optical, sensory and kinematical measurement, whereby an integrated and complete motion detection system will be developed. Therapy Assists is especially focused on patients with complaints regarding their motion sequences of the lower extremities or even amputations. For the success of the project it is very important, that from the beginning of the development process all wishes, objectives and characteristics of the target group are taken into consideration. Thus, we follow a user-centred design process, with continuous usability and user experience testing. The Entertainment Computing Group focuses on the design of a motivating and playful training experience. Our knowledge in the area of Serious Games and Gamification will be helpful in order to design sophisticated strategies to engage the patient in their training and to elicit long-term motivation by the use of games.  read more ››

The project “PIC - Playful Interaction Concepts” is part of the initiative  Mittelstand Digital, which is sponsored by the German Federal Ministry of Economic Affairs and Energy. The project explores the transfer of experiences known from digital games in order to improve software-based work processes in small to medium sized enterprises, with a focus on Human Computer Interaction. Apart from groundwork about when and how the application of games could work, with respect to the German industry and local cultural background, PIC explores development models and tools. Our work is based on practical expertise and scientific backgrounds in the fields of human computer interaction, game development, storytelling and psychology.                                                                                                                                                               read more ››

The goal of this project is the development of gesture based interaction concepts for Virtual Reality applications. In close collaboration with our partners Pixelkontor and teamcrossmedia, we elaborate innovative ways to interact naturally in VR presentations created with the present4D suite. Therefore, we experiment with state of the art VR hardware such as the VR headset Oculus Rift and in conjunction with the Leap Motion gesture controller.   We, the Entertainment Computing Group, design interaction concepts for core functions of the software, and implement a prototype in order to evaluated user experience, learnability, and practicability. We focus a scientifically and empirically validated innovative interaction design and create a  sophisticated library for the recognition of different gestures in Unity. As the Leap Motion should not be the only gesture controller supported, we design use a modular software structure, which can easily be extended by other sensors. The project is a great first step into the yet unexplored field of modern VR interaction design.   read more ››

"Wolkenlos" ("cloudless") stands for a networked media offer with a focus on medical and psychosocial effective Serious Games and applications. It was initiated to strongly improve the living conditions of children and youths, who suffer from cancer, with the humanitarian appliance of modern technologies. In close dialog with the participating consultants of the Essen University Hospital, the Hannover Medial School and the Charité Berlin, we specifically identify questions of medical everyday life, which are coping with the following subgoals: 1. Reduction of psychosocial accompanying symptoms for the patients, especially the social isolation. 2. Strengthening of self-efficacy and the wellbeing of the patients. 3. Provision of information and playful learning 4. Improvement of medical processes with targeted media use A multicentric Online-Community platform supports the communication between all participants and connects single subprojects.  read more ››

The Kroschke foundation supports our project in which we work on the development of an age-appropriate, interactive explanation video for the health education of patients in the pediatric oncology. Therefore, we come together with the Hannover Medical School to create a medial realisation of medical briefings in the field of cancer therapy.  read more ››

The goal of the “bee” project is the development of a modern and game based assistance system for the software bee, developed by ESZETT. To facilitate the entrance and the use in an efficient manner the application focuses on motivational concepts and product evaluation. Further the user acceptance and learnability are consider core concerns in the development process. Hence the expert knowledge of the Entertainment Computing Group in the field of motivational psychology and concerning gamification processes is valuable for creating a comprehensive and playful assistance system. The theoretical knowledge and conceptual work is united in a prototype as a proof of concept.   read more ››

Within the project Playful FiBu we work together with the company Diamant on a concept of further development in the field of usability for an operational software for accounting. (Diamant Software GmbH & Co. KG,2013)   read more ››

With the “Smart Gaming” project the department of Media and Computer Science and Entertainment Computing explores new game concepts for Smart TVs in cooperation with the independent game developer Nurogames from Cologne. The combination of different devices such as smartphones and tablets is to enable new interaction concepts, to support common game activities in the family circle as well as the circle of friends and therefore to evoke positive social game experience. Raised questions, such as how various devices can be connected, which game combinations and illustration facilities are available and how a preferably intuitive control can be achieved, are analyzed in conceptual as well as technical ways. The complete official press release of the project can be found here.   read more ››

In this application-oriented research and development project we are working together with the Loewe Opta GmbH on new 2D/3D-interaction patterns for the field of entertainment electronics. The research focus are game based concepts as well as stereoscopic experiences.  read more ››

PRIME is a consortium consisting of DVS Digital Video Systems AG, FLYING EYE, Fraunhofer Heinrich Hertz Institute HHI, Fraunhofer IIS, Film University Babelsberg Konrad Wolf, Kinoton, KUK Filmproduktion, Loewe and the workgroup Media and Computer Science | Entertainment Computing of the University of Duisburg-Essen. Launching of digital technology in cinemas enables new ways of media consumption. The realistic, three-dimensional illustration of motion pictures and live-events is just one out of many possibilities. Different from 50 years ago the realization of 3D cinema is not connected to restrictions of analog techniques anymore. The worldwide increasing number of Digital Cinema Installations and the great success of 3D movies took the next step in the field of media consumption. The change of paradigm will consequently take hold in living rooms: The keyword “3D-TV” will make three-dimensional experiences possible for everyone. Both areas 3D-cinema and 3D-TV grew for the most part independently of each other yet. On the one hand screens, that are metres long and on the other hand high definition displays, which bring movies, video games and events into living rooms by the help of digital transmission. In order to develop pioneering techniques and workable business models for an initiation of the three-dimensional media consumption concerning cinema, TV and games, eight leading firms and research institutions have teamed up to a consortium. Our project “PRIME – Produktions- und Projektionstechniken für Immersive MEdien-“ (PRoduction and projection techniques for Immersive Media) gets support from the Federal Ministry for Economic Affairs and Energy BMWi (http://www.bmwi.de/).   read more ››

“Voxgrinder” is the working title for a project, which deals with the development of a database-supported project management and production resolution for the multilingual process of audio and localization. We work together with a nameable expert for voice recording. The research focus is the semi-automization through target-group-specific user interfaces and modular data models of the process.   read more ››

The research project Serious Game for Dental Health runs in cooperation with the clinic for stomatology of the Freiburg University Hospital and adds the field of Serious Games to the research area of our workgroup. The goal of the project is to conceptualize an innovative learning game, which motivates children at the age of eight to twelve years to perform teeth and oral hygiene improving activities on a regular basis. In addition to an increased feeling regarding their self-efficacy, the game should impart wide factual knowledge concerning coherencies among nutrition and the condition of the oral flora as well as an overview on different possibilities to improve the oral hygiene. Within the context of the project an implementation of possible interaction scenarios by using the Nintendo WiiMote was realized. Students of the master degree course Applied Cognitive and Media Science were involved during the process. Besides, the development of a playable prototype was successfully completed in October 2009. Visual Studio in combination with Game Studio 3.0 act as the development environment for the project. Furthermore, after the prototype development was completed, a continuation of the project in context of a clinical evaluation study by the Freiburg University Hospital is planned.   read more ››