OVERVIEW

The aim of the course is to start with the knowledge on the fundamentals of graphics, modeling and animation of 3D digital objects, to arrive at the programming skills necessary to build applications and systems based on simulation in virtual / mixed / augmented / extended reality (VR / AR / MR/ XR).

This objective will serve to make students aware of the necessary interdisciplinarity of contributions from mobile programming, biomechanics, sensory perception, robotics and video games to manage complex interactions between simulated and / or physical objects and actors (both in first person view FPV and third person view TPV).

AIMS AND CONTENT

LEARNING OUTCOMES

In this course students will become aware of the necessary interdisciplinarity of Augmented Reality and Virtual Reality technologies for Computer Engineering: starting from the knowledge of the fundamentals of computer graphics, modeling and animation of 3D digital objects, the aim of the course is to get to the programming skills necessary to build applications and systems in virtual / mixed / augmented / extended reality (VR / AR / MR / XR). Considering recent approaches from mobile programming to biomechanics, sensory perception and generative-AI, humanoid robotics and video games, students will face challenging tasks in order to manage complex interactions between simulated and / or physical objects and actors (both FPV first-person view and TPV third-person view).

AIMS AND LEARNING OUTCOMES

Understanding that Virtual Reality and Augmented Reality are "media" is the starting point of this course. On the one hand, through the techniques of Computer Graphics & Animation, Visual Effects, Generative AI and the introduction of immersive systems (sensors and actuators), today we are able to simulate and render the real world in such a realistic way as to deceive our senses in an extremely sophisticated way; on the other hand, through the insertion of artificial elements into the real perception we are able to increase our operational and intellectual abilities. Finally, combining VR and AR to arrive at a Mixed or rather an Extended Reality (XR) is important to understand how simulation, sensing, AI and implementation methodologies must be adequately integrated with Digital Storytelling techniques.

At the end of this course, the student will be able to:

• design and implement simple immersive applications on game-engine and mobile platforms,
• realize XR solutions that are not only effective but also funny,
• learn how to channel creativity, technique, and problem-solving within XR.

PREREQUISITES

There are no particular prerequisites, although having C ++ programming and Computer Graphics / Computer Vision basics can certainly be useful.

TEACHING METHODS

Lectures (in presence and/or online), supported by video tutorials and laboratory exercises (even virtual) and by in-depth seminars.

The teaching methodology used in the course will be based on the active involvement of students in the learning process and will be re-contextualized for integrated digital didactics (IDD). It will therefore be a hybrid methodology between frontal lessons, Flipped Classroom.

SYLLABUS/CONTENT

PART ONE: Virtual Reality - from 3D Modeling to Animation / Tracking of 3D objects (lessons and video tutorials)

• Introduction to Virtual Reality (VR); differences between VR, Cinematic VR, Augmented Virtuality, Mixed reality, Augmented Reality (AR) and Extended Reality.
• Recall from Perception and Biomechanics: principles of 3D Computer Graphics and Stereoscopy for XR. Reference systems and 3D transformations. 3D modeling and rendering.
• The Graphics Pipeline.
• Game Design and Game-based Learning theories and methodologies.
• Storytelling and MDA framework: theories and methods for Gamification.

PART TWO: VR Design & Coding (lessons and exercises)

• Languages, programming environments and tools for VR: programming exercises with game engine.
• Game Engine Editor: Worlds, Assets, Actors and Geometry, Components, Level Design.
• VR Programming: Gameplay Architecture, UI Framework, Gameplay Framework, Character and object animation.
• Tracking: Principles and Algorithms.
• Interactions: Locomotion, Teleportation, Wayfinding, Manipulation, Social interaction.

PART THREE: From Augmented Reality to Extended Reality (lessons and seminars)

• From VR to AR: algorithms and tools for marker-based AR, marker-less AR, Augmented Vision.
• Extended Reality and the relationship with IoT, Robotics and Al.
• Simulation of Robots and Drones
• Believable AI Entities in UE5: Navigation System, Behavior Trees, EQS for Spatial Reasoning, AI Perception
• The devices: sensors, viewers, actuators for the XR
• Future AI-based trends in AR / MR / VR / XR.

RECOMMENDED READING/BIBLIOGRAPHY

• Steven M. La Valle, Virtual Reality, 2023, Cambridge Univ. Press, freely available at http://vr.cs.uiuc.edu/
• S.lacono - G.Vercelli, Dalla fotogramnmetria con i droni alla Virtual Reality (cod.113560) (in Italian),  https://raiseliguria.aulaweb.unige.it/course/view.php?id=33
• Epic Games, Tutorial su Unreal Engine, available at:  https://dev.epicgames.com/community/unreal­ engine/learning

TEACHERS AND EXAM BOARD

LESSONS

LESSONS START

The timetable for this course is available here: Portale EasyAcademy

Class schedule

The timetable for this course is available here: Portale EasyAcademy

EXAMS

EXAM DESCRIPTION

The exam modality is based on a team-based or individual project, followed by an oral discussion.

Requested submissions:

• Pitch Document before the end of the lessons
• Report of the Project (or GDD Game Design Document), 7 working days before the chosen exam date.
• Final Demo of the project, 2 working days before the chosen exam date.
• Presentation of the project and oral discussion, on the exam date

ASSESSMENT METHODS

The assessment method is based on the continuous assessment of learning and the final group or individual project, followed by an oral discussion.

The distribution of grades will be as follows:

• Active partecipation and submission on time of the Pitch Document, 10%
• Submission on time of the Report of the Project (or Game Design Document), 25%
• Submission on time of the Final Demo of the project, 40%
• Presentation of the project and oral discussion, 25%

FURTHER INFORMATION

Students with disabilities or learning disorders are allowed to use specific modalities and supports that will be determined on a case-by-case basis in agreement with the Delegate of the Engineering courses in the Committee for the Inclusion of Students with Disabilities, prof. Francesco Curatelli. Students are invited to contact the teacher and copy the Delegate (francesco.curatelli@unige.it).