> For the complete documentation index, see [llms.txt](https://teaguemm.gitbook.io/internship-avatar-pipeline/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://teaguemm.gitbook.io/internship-avatar-pipeline/reading-guide.md). # Reading Guide ## Myself and the project My name is Teague Murray-Marshall, and I will be kicking off this portfolio with a short introduction introducing both myself and my internship project. [Here](https://sites.google.com/view/teaguemm) is a personal introduction. As for my project: The goal of the project is to develop a pipeline whereby the realism of AI-generated avatars can be validated. This means that I will continue to work on previous research into AI-generated avatars. The current situation and my plans to add to it can be found in my [Project Plan](/internship-avatar-pipeline/problem-definition/project-plan.md), [Project Goals](/internship-avatar-pipeline/problem-definition/stated-goals.md) and [Project Context](/internship-avatar-pipeline/problem-definition/project-context.md). The goal of this portfolio is to indicate what I have worked on, why I prioritise something at any given time and what my methodology is. In this, I hope to offer a glimpse into the how behind the what. Other documentation will be linked where necessary. ## The company The Interaction Design lectorate presides over all (serious) game or interaction design related research projects within Fontys. Though it used to be directly involved with the creation of serious games, it has now transitioned to what it calls "play as a service". In this, it aims to use games or game technology as a solution to various problems in complex industries. These goals are realised by working together with various other organisations, such as the TU/e or various government agencies. There is also a collaboration with students, which was the catalyst for my internship. More information can be found [here](https://fontys.nl/Onderzoek/Interaction-Design.htm). ## Problem Statement ### Design Challenge The design challenge is as follows: "*Implement a cloud-based pipeline that can provide concrete feedback on the realism of (a) provided image(s) or video(s) of an AI-generated avatar so that the developers of the avatars can verify the applicability thereof in social situations and ensure continued immersion in the interaction."* This rather elaborate design challenge can be split into multiple parts which will need separate verification along with the general verification throughout the project. * A pipeline should be created that is cloud based with the techniques and properties one would expect of such an application. * It should aid the verification of avatar realism. * Developers need a concrete way to check and verify the results of the analysis. ### Shifting Emphasis As the project progressed, it was clear that while the overall project should stay more or less the same, the emphasis should be shifted from the analysis itself to the supporting architecture and pipeline. The analysis itself requires more data science expertise, which is best suited to a follow-up project. The main deliverable of the project is now the pipeline itself. This means the [architecture ](/internship-avatar-pipeline/software-architecture/c4-model.md)and [modularity ](/internship-avatar-pipeline/project-contents/pipeline-components.md)of the project take precedence. ## Phases ### Analysis Before anything could be implemented, it was important to get a good idea of the [context ](/internship-avatar-pipeline/problem-definition/project-context.md)- both of the related domain (analysing faces) and the project as a whole. The pipeline can broadly be split into three parts: getting metrics, analysing them and returning the data. Each of these steps required further research. My aim was to reflect these in the research questions: * #### What makes certain facial expressions realistic? More info: [What Makes a Face Realistic?](/internship-avatar-pipeline/research/what-makes-a-facial-expression-realistic.md) * #### How will I create a service that can use the results from the analysis to determine whether a face is realistic? More info: [OpenFace Container](/internship-avatar-pipeline/project-contents/pipeline-components/openface-container.md), [Analysis Module](/internship-avatar-pipeline/project-contents/pipeline-components/analysis-module.md), [Comparison of the Available Facial Analysis Frameworks](/internship-avatar-pipeline/research/technical-research/comparison-of-the-available-facial-analysis-frameworks.md) * #### What technological choices (frameworks, languages, facial analysis library etc.) best suit the project? More info: [Interfacing With OpenFace: C++ & Terminal](/internship-avatar-pipeline/research/technical-research/interfacing-with-openface-c++-and-c.md), [Choosing (a) Language(s)/Framework(s)](/internship-avatar-pipeline/research/technical-research/choosing-a-language-s-framework-s.md) * #### How can I interface with the desired facial analysis library? More info: [Interfacing With OpenFace: C++ & Terminal](/internship-avatar-pipeline/research/technical-research/interfacing-with-openface-c++-and-c.md) * #### What kind of results are important to the users of this pipeline? More info: [Target Audience](/internship-avatar-pipeline/problem-definition/target-audience.md) * #### How will results and/or analysis differ in videos vs. still images or posed vs in-the-wild shots? This research question was of lesser importance due to the emphasis shift from the results of the analysis to the supporting architecture. More info on general analysis validation: [Functional Validation](/internship-avatar-pipeline/validation/what-should-be-validated/functional-validation.md) ### Design Once the initial analysis was finished and I had a more complete overview of the project and its goals, I worked on the technical design of the pipeline. The [initial flow](/internship-avatar-pipeline/problem-definition/general-flow.md) was described early on, upon which I could base my technical design. This technical design was done with the input of Mark Klerkx on the Azure infrastructure and Bartosz Paszkowski on the software engineering and is described in the Software Architecture section. All concepts in the pipeline are seen in the [data flow](/internship-avatar-pipeline/software-architecture/data-flow.md). The main overview of the pipeline is in the [C4 Model](/internship-avatar-pipeline/software-architecture/c4-model.md). User experience (UX) is of relatively little impact on this project, as it is more focused on sound architectural choices. The front-end has little for users to interact with, but the considerations made are described in the [Front-End](/internship-avatar-pipeline/project-contents/pipeline-components/front-end.md) page. At the end of the design process I came to an up-to-date [MVP](/internship-avatar-pipeline/problem-definition/minimum-viable-product-mvp.md). ### Realisation and Evaluation Now that analysis and design are covered, what remains is the actual implementation of the project and the subsequent validation. The core of the implementation is in the different modules: [Front-End](/internship-avatar-pipeline/project-contents/pipeline-components/front-end.md), [OpenFace Container](/internship-avatar-pipeline/project-contents/pipeline-components/openface-container.md), [Analysis Module](/internship-avatar-pipeline/project-contents/pipeline-components/analysis-module.md), [Reporting Module](/internship-avatar-pipeline/project-contents/pipeline-components/reporting-module-not-implemented.md), [Data Storage](/internship-avatar-pipeline/project-contents/pipeline-components/data-storage.md) and [Azure Functions](/internship-avatar-pipeline/project-contents/pipeline-components/azure-functions.md). In order to connect my various applications, I made use of different Azure services (mainly the BLOB Storage and Azure Functions). More on that can be found in the [Deploying to Azure](/internship-avatar-pipeline/project-contents/deploying-to-azure.md) page. As time is limited, certain features or non-functionals have had to be cut. [These and other suggestions to expand the application ](/internship-avatar-pipeline/final-recommendations/handoff.md)are mentioned in the Final Recommendations section. ## Reflection Reflecting on the project, its goals and my personal goals are core parts of ensuring the project was a success. A reflection on the goals of the project and my personal thoughts surrounding it can be seen in the [Evaluation and Conclusion](/internship-avatar-pipeline/evaluation-and-personal-development/evaluation-and-conclusion.md) section. Furthermore, I reflect on each personal goal in its own corresponding section: [Technological Skill and Proficiency](/internship-avatar-pipeline/evaluation-and-personal-development/technological-skill-and-proficiency.md), [Research-Based Decision Making](/internship-avatar-pipeline/evaluation-and-personal-development/research-based-decision-making.md), [Independent Development](/internship-avatar-pipeline/evaluation-and-personal-development/independent-development.md) and [Career Prospects](/internship-avatar-pipeline/evaluation-and-personal-development/career-prospects.md).