OVERVIEW

The main goal of affective computing is to develop models and systems that can recognize, interpret, process, and simulate human affective states. This emerging field of Computer Science has several applications, such as the creation of artificial agents, entertainment (e.g., video games), serious games, virtual training environments, positive computing, and systems to improve well-being (e.g., self-monitoring, personal development, therapy), as well as learning, marketing, and more. The course will provide knowledge related to the design and development of affective computing systems, including theoretical foundations (a brief introduction to the psychology of emotions) and practical skills (such as designing data collection protocols and building computational models).

AIMS AND CONTENT

LEARNING OUTCOMES

The course will provide students with knowledge for the design and development of models and systems for affective computing. The first part will be dedicated to theoretical foundations (e.g., an overview of main emotion theories and emotion regulation). The second part will focus on techniques for data acquisition, processing, and development of computational models. Students will also participate in hands-on activities, learning tools and devices for data collection and processing, as well as creating models for emotion recognition. The focus will be on nonverbal behaviors such as facial expressions, body movements, gaze, and touch gestures, which may communicate interpersonal attitudes, social relations, affective states, and personality traits to interaction partners (humans or artificial agents). In this context, data may consist of video recordings collected with simple webcams but can also be gathered with other sensors and devices such as RGB-D cameras, microphones, accelerometers, tactile sensors, and so on.

AIMS AND LEARNING OUTCOMES

The aim of this course is to introduce the basic concepts of the psychology of emotions, social psychology, and positive psychology, and to show how these concepts can be modelled computationally. Such models can be used to:

1. recognize and classify humans’ internal states, personality traits, and social attitudes (e.g., emotion recognition from facial expressions);

2. reason about human emotions (e.g., to build empathic artificial companions);

3. simulate emotions in artificial agents (such as social robots or virtual agents) and enable them to appropriately communicate these simulated emotions during interaction.

The course focuses on detecting and processing nonverbal behaviors such as facial expressions, body movements, gaze, and touch gestures, which provide information about interpersonal attitudes, social relationships, affective states, and personality traits of an interaction partner (human or artificial agent). During the lessons, students will participate in hands-on activities and learn to use tools and devices for collecting and processing data, as well as for building simple computational models.

During the lessons, students will participate in hands-on activities and learn to use tools and devices for collecting and processing data, as well as building simple computational models.

Upon completion of this course, a student will be able to:

• apply main psychological theories of emotions, social relations, and attitudes to design computational models for affective and social computing,
• design data collection protocols, choose appropriate techniques and technologies to collect nonverbal behavior data,
• collect the nonverbal behavior data in single-person and interactive scenarios,
• apply techniques of the data assessment and annotation (e.g., external annotation, self-assessment, crowdsourcing),
• develop new datasets for emotion modelling,
• apply basic methods of feature extraction and selection,
• build computational models, e.g., for the classification of internal states using machine learning techniques,
• evaluate the developed models.

PREREQUISITES

Basic computer skills.
Basic competencies in programming (any programming language) .

TEACHING METHODS

The course is structured as a series of lectures combined with practical learning. The theoretical lectures would introduce the concepts and techniques used in affective computing and social signal processing. Such lectures will be based on the latest research publications and will be delivered with the help on visual aids such as videos and other multimedia.

The practical learning will consist of using the devices and software for data collection, annotation, and processing. The hands-on activities will allow students to apply the concepts presented during the lectures to solve specific tasks.

A part of practical activities will be dedicated to developing (in a classroom) one joint propaedeutic project. This project will be realized with the active participation of all students.

SYLLABUS/CONTENT

The following concepts will be discussed during the lectures:

• psychological background: emotion theories and emotion regulation, appraisal theories, interpersonal stances/attitudes, social relations, personality models, positive psychology and well-being, and so on;
• relation between nonverbal behaviors (e.g., facial expressions, body movements, postures, voice prosody, touch) and internal states;
• multimodality and (in)-congruent modalities;
• computational models of emotions;
• overview of techniques and protocols for multimodal data collection, as well as devices/sensors. Passive and active techniques of emotion induction,
• overview of freely available datasets in affective computing, creation of the datasets using online content, ethical and privacy issues related to the data collection;
• design of data collection protocols;
• practical activities (new data collection) using sensors and devices (e.g., Kinect, accelerometers) and freely available software (e.g., OBS);
• methodological issues related to the data assessment: methods and tools, design of questionnaires, manual data annotation (e.g., Elan), validation, and inter-rater agreement;
• features extraction from collected data: extracting the features using freely available software (e.g., OpenFace, OpenPose, Python Libraries), designing own hand-crafted features;
• design, development and validation of hand-crafted feature-based computational models;
• state-of-the-art examples of automatic emotion/social relations recognition;
• models for affect communication, and their applications to artificial agents.

RECOMMENDED READING/BIBLIOGRAPHY

• Calvo, R., D'Mello, S., Gratch, J., Kappas, A. (Eds.), The Oxford Handbook of Affective Computing, Oxford University Press, 2015.
• Gratch, J., Marsella, S. (Eds.), Social Emotions in Nature and Artifact, Oxford Series on Cognitive Models and Architectures, Oxford Series on Cognitive Models and Architectures, 2014.
• Burgoon, J., Magnenat-Thalmann, N., Pantic, M., Vinciarelli, A. (Eds.), Social Signal Processing, Cambridge University Press, 2017.

TEACHERS AND EXAM BOARD

Exam Board

RADOSLAW NIEWIADOMSKI (President)

ANTONIO CAMURRI

GUALTIERO VOLPE (President Substitute)

LESSONS

LESSONS START

https://easyacademy.unige.it/portalestudenti/index.php?view=easycourse&_lang=it&include=corso

Class schedule

The timetable for this course is available here: EasyAcademy

EXAMS

EXAM DESCRIPTION

The course assessment will be based on stages: 1) tasks systematically carried out during the course, 2) final project.

Regarding stage 1), during lessons, the students, under the supervision of the lecturer, will develop together one project focusing several aspects of affective computing and social signal processing. The project will be composed of several subtasks, and students will deliver partial solutions.

Regarding stage 2, the students will apply the theory and practical skills learned during the course to solve concrete challenges related to the course topics outside the classroom. The final projects can be carried out in small teams. The topics must be agreed upon with the lecturer. The projects consist of the design of a study protocol, data collection, data analysis, and model development.

Students not attending the course (i.e., who do not participate in stage 1) are expected to prepare individual projects (stage 2).

Students with certification of Specific Learning Disabilities (SLD), disabilities, or other special educational needs must contact the instructor at the beginning of the course to agree on teaching and examination methods that, while respecting the course objectives, take into account individual learning styles and provide appropriate compensatory tools. It is reminded that the request for compensatory/dispensatory measures must be sent to the course instructor, the School representative, and the “Settore servizi per l'inclusione degli studenti con disabilità e con DSA” office (dsa@unige.it), as per the guidelines available at the link: https://unige.it/disabilita-dsa

ASSESSMENT METHODS

Project evaluation will be based on how the concepts discussed during the lectures were implemented to solve real-life problems. The evaluation of the projects will measure students' practical skills related to affective computing. For teamwork, it is expected that the contribution of each team member is clearly highlighted.