OVERVIEW

This teaching unit introduces the theoretical foundations and practical application of state-of-the-art deep learning techniques for automated driving, with particular emphasis on environmental perception, scene understanding, and prediction. It covers models for visual and time-series data, attention mechanisms, Transformers, and language and vision-language models, alongside explainability and robustness.

A central theme is edge AI, which enables machine learning models to operate directly on vehicles and embedded devices, close to sensors and data sources. Students explore how computational resources, memory, energy consumption, and latency influence model design and deployment, including on microcontrollers. These considerations are essential to automated driving and are also relevant to robotics, industrial automation, smart sensing, and other embedded applications.

AIMS AND CONTENT

LEARNING OUTCOMES

This teaching unit provides students with advanced knowledge and solid understanding of the state of the art machine learning techniques used in automated driving, particularly with regard to context perception and prediction. The course also introduces to tiny machine learning firmware on microcontrollers for field deployment. The student will develop analytical and design skills through a project.

AIMS AND LEARNING OUTCOMES

The teaching unit aims to provide students the knowledge and practical skills needed to select, develop, and evaluate machine learning solutions for automated driving, particularly for perception and prediction. Theoretical foundations are complemented by application code and practical examples using Python and libraries such as Keras/TensorFlow, and PyTorch.

Students examine convolutional and recurrent networks, attention mechanisms, Transformers, and their application to object detection, semantic and panoptic segmentation, and time-series classification and forecasting. The unit also introduces large language models, retrieval-augmented generation, and vision-language models, considering their potential roles and limitations in automated driving.

Throughout the unit, students are encouraged to assess both predictive performance and deployment feasibility, relating computational load, memory requirements, and inference latency to the constraints of the target hardware. Explainability and robustness are addressed as essential aspects of evaluating machine learning systems. An examination project, agreed upon with the lecturer, provides an opportunity to integrate these skills in a practical application.

By the end of the teaching unit, students will be able to:

• Explain the principles of the main model families covered and justify their suitability for specific perception, scene understanding, and prediction tasks.
• Design and implement machine learning workflows, including data preparation, model training, and evaluation, using appropriate Python tools.
• Select suitable evaluation metrics and validation procedures, interpret results, and identify limitations in model performance and generalisation.
• Explain the roles and limitations of language models, retrieval-augmented generation, and vision-language models in automated driving applications.
• Apply appropriate methods to investigate model explainability and robustness.
• Analyse computational, memory, and latency requirements, and identify suitable model and hardware optimisation strategies for edge deployment, including the feasibility of deployment on microcontrollers.
• Document and justify design choices, presenting experimental evidence and critically discussing the results.

PREREQUISITES

Digital systems electronics

Fundamentals of programming

Tiny Machine learning

Cyberphysical systems

TEACHING METHODS

Lectures face-to-face, with use of slides, and examples/exercises carried out on the PC, mainly using the Keras/Tensorflow, PyTorch libraries, in python language. Student reception. Proposal, implementation and discussion of a project.

Students with valid certifications for Specific Learning Disorders (SLDs), disabilities or other educational needs are invited to contact the teacher and the School's contact person for disability at the beginning of teaching to agree on possible teaching arrangements that, while respecting the teaching objectives, take into account individual learning patterns. Contacts of the teacher and the School's disability contact person can be found at the following link 

University Committee for the Inclusion of Students with Disabilities or DSAs | UniGe | University of Genoa

SYLLABUS/CONTENT

• Introduction to machine learning for automated driving
• Convolutional Neural Networks architectures and training
• Object detection
• Semantic / panoptic segmentation
• Timeseries classification and prediction
• LSTM, 1D convolution
• Attention mechanism
• Transformers and Large Language Models
• Retrieval Augmented Generation (RAG)
• Vision Transformers and Detection Transformers
• Vision Language Models
• Multimodal Transformers for Driving
• Trustworthy Machine Learning (explainability and robustness)
• Deployment on microcontrollers

RECOMMENDED READING/BIBLIOGRAPHY

A. Geron, Hands-On Machine Learning With Scikit-Learn and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent, O’ Reilly

I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, The MIT Press

Documentation of the used libraries

Lecture notes and other material (typicaly, scientific papers) suggested by the lecturer during the course

Non-attending students may contact the lecturer to agree on the best arrangements

TEACHERS AND EXAM BOARD

LESSONS

LESSONS START

To access to the lesson calendar, click on the following link: https://corsi.unige.it/en/corsi/11780/studenti-orario

Class schedule

The timetable for this course is available here: Portale EasyAcademy

EXAMS

EXAM DESCRIPTION

Project work on an application example of machine learning for automated driving.

ASSESSMENT METHODS

Assessment is based on a project and its discussion, evaluating students’ theoretical understanding and their ability to apply the methods covered in the teaching unit. The project addresses a machine learning problem relevant to automated driving, particularly perception, scene understanding, or prediction, and may include TinyML deployment on a microcontroller.

Assessment takes place at three stages:

• Project definition: preliminary discussions assess the student’s ability to formulate a clear problem, identify suitable data and evaluation criteria, and justify the choice of methods in relation to the task and available resources.
• Design and implementation: progress reviews assess the student’s ability to prepare data, implement and train models, adopt appropriate validation procedures, and analyse experimental results. Computational requirements and deployment constraints are considered in relation to the project objectives. Where embedded deployment is included, assessment also considers measured resource use and inference performance on the target device.
• Written report and final discussion: these assess the student’s understanding of the theoretical foundations, ability to justify technical choices, and capacity to interpret results critically. The discussion connects the project to the broader course content, including alternative approaches, generalisation, explainability, robustness, and edge deployment where relevant.

Evaluation considers the soundness of the methodology, correctness of the implementation, quality of the experimental evidence, and clarity of the written and oral presentation. Participation in class discussions and practical activities provides additional evidence of the student’s understanding, technical reasoning, and ability to apply the concepts covered.

FURTHER INFORMATION

Ask the professor for other information not included in this description of the teaching unit.