OVERVIEW

The course will provide students with the knowledge needed for the design and development of models and systems for affective computing. The first part will be dedicated to theoretical foundations (e.g., an overview of main emotion theories and emotion regulation). The second part will focus on techniques for data acquisition, processing, and development of computational models. Students will participate in hands-on activities, learn tools and devices for data collection and processing, and create models for emotion recognition. The focus will be on nonverbal behaviors such as facial expressions, body movements, gaze, and touch gestures, which may communicate interpersonal attitudes, social relations, affective states, and personality traits to interaction partners (humans or artificial agents). In this context, data may consist of video recordings but they can also be gathered with other sensors/devices such as RGB-D cameras, microphones, accelerometers and tactile sensors.

AIMS AND CONTENT

LEARNING OUTCOMES

This course focuses on computational models and on the design of technologies to automatically measure, analyze, and communicate emotions in interactive systems. The course will cover topics including the interaction of emotion with cognition and perception, the communication of human emotion via face, voice, physiology, full-body movement, and social behavior. Lab experiments and exercises will include affective technologies for education, cultural welfare, rehabilitation, and performing arts.

AIMS AND LEARNING OUTCOMES

The aim of this course is to introduce the basic concepts of the psychology of emotions, social psychology, and positive psychology, and to show how these concepts can be modelled computationally. Such models can be used to: 1) recognize and classify humans’ internal states, personality traits, and social attitudes (e.g., emotion recognition from facial expressions), 2) reason about human emotions (e.g., to build empathic artificial companions), 3) simulate emotions in artificial agents (such as social robots, virtual agents) and to allow them to communicate simulated emotions appropriately to their interaction.
The course relies on detecting and processing nonverbal behaviors such as facial expressions, body movements, gaze, and touch gestures which provide information on interpersonal attitudes, social relations, affective states, and personality traits of an interaction partner (human or artificial agent).
During the lessons, students will participate in hands-on activities and learn to use tools and devices for collecting and processing data, as well as building simple computational models.

Upon completion of this course, a student will be able to:

• apply main psychological theories of emotions, social relations, and attitudes to design computational models for affective and social computing,
• design data collection protocols, choose appropriate techniques and technologies to collect nonverbal behavior data,
• collect the nonverbal behavior data in single-person and interactive scenarios,
• apply techniques of the data assessment and annotation (e.g., external annotation, self-assessment, crowdsourcing),
• develop new datasets for emotion modelling,
• apply basic methods of feature extraction and selection,
• build computational models, e.g., for the classification of internal states using machine learning techniques,
• evaluate the developed models.

PREREQUISITES

Basic computer skills.
Basic competencies in programming (any programming language) .

TEACHING METHODS

The course is structured as a series of lectures combined with practical learning. The theoretical lectures would introduce the concepts and techniques used in affective computing and social signal processing. Such lectures will be based on the latest research publications and will be delivered with the help on visual aids such as videos and other multimedia.

The practical learning will consist of using the devices and software for data collection, annotation, and processing. The hands-on activities will allow students to apply the concepts presented during the lectures to solve specific tasks.

A part of practical activities will be dedicated to developing (in a classroom) one joint propaedeutic project. This project will be realized with the active participation of all students.

SYLLABUS/CONTENT

The following concepts will be discussed during the lectures:
· Psychological background: emotion theories and emotion regulation, appraisal theories, interpersonal stances/attitudes, social relations, personality models, positive psychology and well-being, and so on;
· relation between nonverbal behaviors (e.g., facial expressions, body movements, postures, voice prosody, touch) and internal states;
multimodality and (in)-congruent modalities;
· computational models of emotions;
· overview of techniques and protocols for multimodal data collection, as well as devices/sensors. Passive and active techniques of emotion induction,
 overview of freely available datasets in affective computing, creation of the datasets using online content, ethical and privacy issues related to the data collection;
· design of data collection protocols;
· practical activities (creation of your own data collection) using sensors and devices (e.g., Kinect, accelerometers) and freely available software (e.g., EyesWeb);
· methodological issues related to the data assessment: methods and tools, design of questionnaires, manual data annotation (e.g., Elan), validation, and inter-rater agreement;
· features extraction: extracting the features using freely available software (e.g., OpenFace, OpenPose), designing own hand-crafted features;
· design, development and validation of feature-based computational models;
· state-of-the-art examples of automatic emotion/social relations recognition;
· models for affect communication, and applications to artificial agents.

RECOMMENDED READING/BIBLIOGRAPHY

• Calvo, R., D'Mello, S., Gratch, J., Kappas, A. (Eds.), The Oxford Handbook of Affective Computing, Oxford University Press, 2015.
• Gratch, J., Marsella, S. (Eds.), Social Emotions in Nature and Artifact, Oxford Series on Cognitive Models and Architectures, Oxford Series on Cognitive Models and Architectures, 2014.
• Burgoon, J., Magnenat-Thalmann, N., Pantic, M., Vinciarelli, A. (Eds.), Social Signal Processing, Cambridge University Press, 2017.

TEACHERS AND EXAM BOARD

Exam Board

RADOSLAW NIEWIADOMSKI (President)

LESSONS

LESSONS START

https://easyacademy.unige.it/portalestudenti/index.php?view=easycourse&_lang=it&include=corso

Class schedule

The timetable for this course is available here: Portale EasyAcademy

EXAMS

EXAM DESCRIPTION

The course assessment will be based on stages: 1) tasks systematically carried out during the course, 2) final project.

Regarding stage 1), during lessons, the students, under the supervision of the lecturer, will develop together one project focusing several aspects of affective computing and social signal processing. The project will be composed of several subtasks, and students will deliver partial solutions.

Regarding stage 2, the students will apply the theory and practical skills learned during the course to solve concrete challenges related to the course topics (out of classroom). The final projects can be realized in small teams. The topics need to be agreed upon with the lecturer. The projects consists of a design of a new data collection protocol, data collection, data analysis, and model development.

Students not attending the course (i.e., who do not participate in stage 1) are expected to prepare individual projects (stage 2).

Students with certification of Specific Learning Disabilities (SLD), disabilities, or other special educational needs must contact the instructor at the beginning of the course to agree on teaching and examination methods that, while respecting the course objectives, take into account individual learning styles and provide appropriate compensatory tools. It is reminded that the request for compensatory/dispensatory measures must be sent to the course instructor, the School representative, and the “Settore servizi per l'inclusione degli studenti con disabilità e con DSA” office (dsa@unige.it), as per the guidelines available at the link: https://unige.it/disabilita-dsa

ASSESSMENT METHODS

Project evaluation will be based on how the arguments discussed during the lectures were implemented to solve real-life problems. The evaluation of the projects will measure students' practical skills related to Affective Computing. For team work, it is expected that the contribution of each team member will be clearly highlighted.