OVERVIEW

The main goal of affective computing is to develop models and systems that can recognize, interpret, process, and simulate human affective states. This emerging field of Computer Science has several applications, such as the creation of artificial agents, entertainment (e.g., video games), serious games, virtual training environments, well-being, and more. The course will provide knowledge related to the design and development of affective computing systems, including theoretical foundations (a brief introduction to the psychology of emotions) and practical skills (such as designing data collection protocols and building computational models).

AIMS AND CONTENT

LEARNING OUTCOMES

The objective of the course is to provide students with the knowledge needed to design and develop models and systems for processing information about emotional states and social attittudes. The first part of the course will be dedicated to theoretical foundations, including an overview of main emotion theories. The second part will focus on techniques for data acquisition, processing, and development of computational models. The focus will be on nonverbal behaviors such as facial expressions, prosody, body movements, gaze, and touch gestures, which may communicate attitudes and affective states to interaction partners, whether human or artificial (e.g., social robots). Students will also participate in hands-on activities, learning to use tools and devices for data collection and processing, as well as developing emotion recognition models.

AIMS AND LEARNING OUTCOMES

The aim of this course is to introduce the basic concepts of the psychology of emotions and to show how these concepts can be modelled computationally. Such models can be used to:

1. recognize and classify humans’ internal states, personality traits, and social attitudes (e.g., emotion recognition from facial expressions);

2. reason about human emotions (e.g., to build empathic artificial companions);

3. simulate emotions in artificial agents (such as social robots or virtual agents) and enable them to appropriately communicate these simulated emotions during interaction.

The course focuses on detecting and processing nonverbal behaviors such as facial expressions, body movements, gaze, and touch gestures, which provide information about interpersonal attitudes, social relationships and affective states of an interaction partner (human or artificial agent). During the lessons, students will participate in hands-on activities and learn to use tools and devices for collecting and processing data, as well as for building simple computational models.

Upon completion of this course, a student will be able to:

• apply main psychological theories of emotions, social relations, and attitudes to design computational models for affective computing,
• design data collection protocols, choose appropriate techniques and technologies to collect nonverbal behavior data,
• collect the nonverbal behavior data in single-person and interactive scenarios,
• apply techniques of the data assessment and annotation (e.g., external annotation, self-assessment, crowdsourcing),
• develop new datasets for emotion modelling,
• apply basic methods of feature extraction and selection from video data,
• build computational models, e.g., for the classification of internal states using machine learning techniques,
• evaluate developed models.

PREREQUISITES

Basic computer skills.
 

TEACHING METHODS

The course is structured as a series of lectures combined with practical learning. The theoretical lectures would introduce the concepts and techniques used in affective computing. They will be based on the latest research publications and will be delivered with the help on visual aids such as videos and other multimedia.

The practical learning will consist of using the devices and software for data collection, annotation, and processing. The hands-on activities will allow students to apply the concepts presented during the lectures to solve specific tasks. A part of practical activities will be dedicated to developing (in a classroom) a joint propaedeutic project with the active participation of all students.

SYLLABUS/CONTENT

The following concepts will be discussed during the lectures:

• psychological background: emotion theories and emotion regulation, appraisal theories, interpersonal stances/attitudes and so on;
• relation between nonverbal behaviors (e.g., facial expressions, body movements, postures, voice prosody, touch) and internal states;
• multimodality and (in)-congruent modalities;
• overview of techniques and protocols for multimodal data collection, as well as devices/sensors. Passive and active techniques of emotion induction,
• overview of freely available datasets in affective computing, creation of the datasets using online content;
• design of data collection protocols;
• methodological issues related to the data assessment: methods and tools, design of questionnaires, manual data annotation (e.g., Elan), validation, and inter-rater agreement;
• features extraction from collected data using freely available software (e.g., OpenFace, OpenPose, Python Libraries), designing own hand-crafted features;
• design, development and validation of hand-crafted feature-based computational models;
• state-of-the-art examples of automatic emotion/social relations recognition.

RECOMMENDED READING/BIBLIOGRAPHY

• Calvo, R., D'Mello, S., Gratch, J., Kappas, A. (Eds.), The Oxford Handbook of Affective Computing, Oxford University Press, 2015.
• Gratch, J., Marsella, S. (Eds.), Social Emotions in Nature and Artifact, Oxford Series on Cognitive Models and Architectures, Oxford Series on Cognitive Models and Architectures, 2014.
• Burgoon, J., Magnenat-Thalmann, N., Pantic, M., Vinciarelli, A. (Eds.), Social Signal Processing, Cambridge University Press, 2017.

TEACHERS AND EXAM BOARD

LESSONS

LESSONS START

https://easyacademy.unige.it/portalestudenti/index.php?view=easycourse&_lang=it&include=corso

Class schedule

The timetable for this course is available here: Portale EasyAcademy

EXAMS

EXAM DESCRIPTION

The course assessment will be based on stages: 1) tasks systematically carried out during the course, 2) final project.

Regarding stage 1, during lessons, the students, under the supervision of the lecturer, will develop together one project composed of several subtasks, and students will deliver partial solutions.

Regarding stage 2, the students will apply the knowledge and practical skills learned during the course to solve concrete challenges related to the course topics outside the classroom. The topics must be agreed upon with the lecturer. The projects consist of the design of a study protocol, data collection, data analysis, and model development.

Students with certification of Specific Learning Disabilities (SLD), disabilities, or other special educational needs must contact the instructor at the beginning of the course to agree on teaching and examination methods that, while respecting the course objectives, take into account individual learning styles and provide appropriate compensatory tools. It is reminded that the request for compensatory/dispensatory measures must be sent to the course instructor, the School representative, and the “Settore servizi per l'inclusione degli studenti con disabilità e con DSA” office (dsa@unige.it), as per the guidelines available at the link: https://unige.it/disabilita-dsa

ASSESSMENT METHODS

The evaluation will be based on how the concepts discussed during the lectures were implemented to solve a pre-defined problem. IT will measure students' practical skills related to affective computing.