|SCIENTIFIC DISCIPLINARY SECTOR||ING-INF/05|
This course provides students with foundational knowledge for designing computer systems that can capture, represent, and automatically analyze the behavior of their users and interact with them natuarally, by generating multisensory feedback in real-time.
This course provides students with foundational conceptual knowledge, methodologies, and tools for designing, implementing, and evaluating computer systems that can capture, represent, and automatically analyze the behavior of their users (e.g., in terms of gesture, movement, facial expressions, speech) and interact with them by generating multisensory feedback (e.g., images, sounds, control of actuators) in real-time.
The course aims at introducing the foundational knowledge needed for designing and developing computer systems that can interact with their users naturally, by exploiting multiple sensory channels. This requires students to know and apply technologies for capturing, representing, and automatically analyzing the behavior of the users – e.g., algorithms for detecting and analyzing gesture, full-body movement, facial expressions, and speech – and for generating multisensory feedback (e.g., images, sounds, and control of actuators) in real-time. At the end of the course, students will:
None. It is useful, even if not strictly required, having some basic knowledge on human-machine interaction topics (design, development, and evaluation cycle of traditional user interfaces, interaction design methodologies). It is also useful having a basic programming experience.
The course includes theoretical and practical lectures (approximately 32h of theoretical lectures and 16h of practical lectures for a total of 48h of lectures). Theoretical lectures aim at introducing the concepts and the techniques the course focuses on. Practical lectures consist of hands-on and enable students to apply the presented concepts and technologies in specific case studies. Students can attend the practical lectures by using their laptops in the classroom. Practical lectures exploit tools for gesture and movement analysis and for audio processing. Lectures will be in English.
1. Introduction to multimodal systems
2. Visual modality
3. Auditory modality
4. Multimodal fusion
5. Case studies
Learning material includes pdf copies of the slides presented at the lectures, examples and exercises for the practical lectures, and a collection of scientific papers the teacher provides the students with. Learning material is made available on AulaWeb. Learning material is provided in English.
GUALTIERO VOLPE (President)
ANTONIO CAMURRI (President Substitute)
All class schedules are posted on the EasyAcademy portal.
The exam consists of a project and an oral discussion. The project is assigned by the teacher and concerns the design of a multimodal interactive system for a specific application. It may include the development of software modules for audio and video processing aiming at contributing to the analysis of the behavior of the users, a deeper analysis of specific topics presented during the lectures by means of a targeted bibliographical search, the analysis of existing solutions, including testing and assessment of algorithms and their performances. The oral discussion consists of the presentation of the results of the project and also includes questions on topics addressed during the lectures.
The project aims at assessing the extent to which the student is able to analyze a case study, to evaluate the opportunity of developing a multimodal interactive systems in such a context, to design a multimodal interactive system, and to apply the technologies presented during the lectures. The oral discussion aims at assessing the extent to which the student knows and understands the foundational theoretical aspects of multimodal systems (motivations, logical architectures, and guidelines for design and development) as well as the student’s understanding of the major data acquisition devices and of the techniques for representation and automated analysis of the behavior of the users. The teacher will assess the quality of the project and of the presentation of the results, the capability of critical reasoning on the developed project, the correct use of specialized lexicon, the deepness of the student’s knowledge and understanding of the topics addressed in the course, and the capability of presenting such a content properly.
|18/02/2022||09:30||GENOVA||Esame su appuntamento|
|16/09/2022||09:30||GENOVA||Esame su appuntamento|
Master theses are available concerning the topics presented in the course and addressing the design and development of multimodal interactive systems in areas of interest for the scientific and technological research carried out at the Casa Paganini – InfoMus research center of DIBRIS – University of Genoa (www.casapaganini.org). For students that are interested in a master thesis on these topics, the course provides the theoretical and practical knowledge, which is needed to carry out the work in the thesis.