Project Context

An overview of what has already been made and the philosophy behind it.

VIBE Project

The VIBE project is a collaboration between various educational and corporate partners such as Fontys, BUAS, Tilburg University and BlueTea. Its goal is to create autonomous, AI-driven avatars that can be used in social interactions within the context of medical procedures or training. An example of this is the use of facial expressions to aid the training of medical professionals with childbirth. In doing this, the goal is to make these training sessions more realistic.

BUAS Research

BUAS is contributing to the VIBE project by creating the pipeline that generates these avatars. Because of this, they are the most important external partner in this internship. They have created a suite using Unreal Engine in which avatars are displayed and their expressions can be edited. There are currently three different avatar models, all of which are generated using modelling tools and a photogrammetry rig.

Images of the photogrammetry rig used to capture images for the avatars.

For manual use, the current tooling is already ideal. Certain expressions are tied to emotions, which each have a slider. The slider determines to what degree the emotion should be expressed. Expressions have specific muscles or muscle groups that they manipulate. Many expressions are very realistic, and custom-made animations are already very convincing. An issue comes up, however, when an AI is tasked with generating expressions: the freedom within the tooling can cause very peculiar expressions to crop up. For example, the image below seems like a face any person could make (though perhaps not in ideal circumstances).

The VIBE software used by BUAS to generate avatars.

Manipulating the sliders can cause the expressions to conflict, however, causing some jarring results:

A jarring combination of expressions.

Though this is an exaggerated example, it would be ideal to filter these kinds of expressions out if they crop up. This is where the validation pipeline comes in. More information on its goals can be found here.