OVERVIEW

The course introduces the theoretical and practical foundations of machine learning, showing how to develop systems that learn from data to perform prediction, classification and analysis tasks.

It covers the main supervised and unsupervised learning models, deep neural networks and techniques for processing images, sequences and time series. Particular attention is given to model optimisation and deployment on embedded devices, considering memory, computational and energy constraints.

AIMS AND CONTENT

LEARNING OUTCOMES

The module provides theoretical and practical foundations of machine learning paradigms: linear models, decision trees, random forests, and deep neural networks. Students learn to select the most suitable solutions, considering the memory and computational power constrains of the target electronic system. This prepares them to design solutions for real-world problems, such as data processing near sensors.

AIMS AND LEARNING OUTCOMES

By the end of the course, students will be able to:

• formulate a machine learning problem by identifying objectives, data, input variables and expected outputs;
• prepare and explore data, handling categorical variables, normalisation, feature engineering and splitting into training, validation and test sets;
• select and apply models for regression, classification, clustering and dimensionality reduction;
• explain the operation of the main algorithms and neural architectures covered, discussing their assumptions, advantages and limitations;
• implement and train models in Python using the main libraries for data processing, machine learning and deep learning;
• evaluate performance using appropriate metrics and procedures, recognise overfitting and underfitting, and apply regularisation and hyperparameter selection techniques;
• apply neural networks to images, sequences and time series, understanding the role of attention mechanisms and Transformer architectures;
• analyse model memory and computational requirements and apply compression and optimisation techniques for deployment on embedded devices;
• compare alternative solutions, justifying trade-offs between predictive performance, complexity, memory, latency and energy consumption;
• document their work and clearly communicate methods, implementation choices and experimental results.

PREREQUISITES

Basic knowledge of the following areas is required:

• Linear algebra: vectors, matrices and tensors; norms and products; linear transformations; matrix inversion; eigenvalues and eigenvectors.
• Differential calculus and optimisation: derivatives, partial derivatives, gradients, Jacobians and Hessians; function minimisation and gradient descent.
• Statistics and probability: fundamental concepts and tools for describing data and interpreting results.
• Python programming: language constructs and development of simple programs.
• Scientific computing and data handling: NumPy for arrays, matrix operations and vectorisation; Pandas for series and data frames; Matplotlib for data visualisation.

TEACHING METHODS

The course alternates between lectures and practical computer-based sessions. During lectures, the teaching staff introduce theoretical concepts and demonstrate their application through code examples executed in Jupyter Notebook. Students may use their own computers to carry out the proposed activities. Practical sessions involve data preparation, model implementation and training, performance evaluation and optimisation for target platforms. Comparison of alternative solutions and discussion of difficulties encountered will support understanding of the methods and the ability to justify design choices. Students will also be guided in interpreting results and presenting their work.

Students with valid certifications for Specific Learning Disabilities (SLD), disabilities, or other educational needs are invited to contact the teacher and the School Disability Coordinator at the beginning of the teaching unit to discuss possible teaching methods that, while respecting the teaching unit objectives, take into account individual learning styles. The contact details of the School Disability Coordinator are available at the following link: University Committee for the Inclusion of Students with Disabilities or SLD | UniGe | University of Genoa

SYLLABUS/CONTENT

Part 1 – Fundamentals and supervised learning

• Introduction to machine learning: learning from examples, models, parameters and hyperparameters; training and inference; model-based and instance-based approaches.
• Data and generalisation: data quantity, quality and representativeness; noise, overfitting, underfitting and the no-free-lunch theorem.
• Data preparation: statistical exploration, correlations, sampling, dataset splitting, categorical variable encoding, normalisation and feature engineering.
• Regression and classification: problem formulation and evaluation metrics; mean squared error, accuracy, precision, recall, F1 score and confusion matrices; binary, multiclass, multilabel and multioutput classification.
• Model selection: cross-validation, hyperparameter search and interpretation of learning curves.
• Linear models: linear and polynomial regression, the normal equation, batch, stochastic and mini-batch gradient descent; logistic and softmax regression; cross-entropy loss.
• Generalisation and regularisation: the bias–variance trade-off, penalty-based regularisation and early stopping.
• Decision trees and ensemble learning: entropy and the ID3 algorithm; interpretability and overfitting; voting, random forests, boosting and stacking.

Part 2 – Neural networks and advanced models

• Neural network fundamentals: perceptrons, multilayer networks, activation functions, forward propagation and backpropagation.
• Training deep networks: weight initialisation, vanishing and exploding gradients, gradient clipping, batch normalisation, optimisers and learning rate schedules.
• Regularisation and model reuse: penalty-based regularisation, dropout, max-norm constraints, early stopping and transfer learning.
• Convolutional neural networks: convolutions, filters, receptive fields, stride, padding and pooling; analysis of computational and memory requirements.
• Computer vision: main convolutional architectures, data augmentation, object detection and segmentation; encoder–decoder networks, transposed convolutions and skip connections.
• Neural Architecture Search: principles of automated architecture search and examples of evolutionary approaches.
• Time series and recurrent networks: single-step and multi-step forecasting, recurrent networks, LSTMs, GRUs and one-dimensional convolutional networks.
• Attention and Transformers: bidirectional networks, attention mechanisms, positional encoding, multi-head attention and an introduction to large language models.
• Dimensionality reduction: high-dimensional data, projection-based and manifold approaches, PCA and LLE.
• Clustering and anomaly detection: K-means and its variants, selection of the number of clusters, DBSCAN and Gaussian mixtures; applications to segmentation and semi-supervised learning.
• Autoencoders and generative models: deep, convolutional, recurrent, denoising, sparse and variational autoencoders; introduction to generative adversarial networks (GANs).

Part 3 – Optimisation and deployment on embedded devices

• Edge AI and TinyML: processing close to sensors and constraints of target platforms.
• Quantisation: post-training quantisation and quantisation-aware training.
• Binary neural networks: principles, training and implications for hardware implementation.
• Pruning: removal of network weights and structures; structured and unstructured approaches.
• Knowledge distillation: transferring knowledge from a teacher model to a student model.
• Embedded inference tools: TensorFlow Lite and TensorFlow Lite for Microcontrollers.
• FPGA implementation: introduction to the development workflow using Brevitas and FINN.
• Deployment on embedded platforms: practical activities using NVIDIA Jetson Nano and Arduino Nano BLE Sense.
• Vision-language models: an example of deployment on Raspberry Pi 5.

Practical activities use Python, Jupyter Notebook and libraries such as NumPy, Pandas, Matplotlib, scikit-learn, TensorFlow/Keras and PyTorch.

RECOMMENDED READING/BIBLIOGRAPHY

The materials and code examples used during lectures provide the reference resources for study and practical exercises.

The following books are recommended for further reading:

• Aurélien Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, O’Reilly Media.
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, Springer.

Additional resources include materials from Neural Networks for Machine Learning, scientific papers and documentation for the libraries used, as indicated by the teaching staff.

TEACHERS AND EXAM BOARD

LESSONS

LESSONS START

https://corsi.unige.it/en/corsi/11970/students-timetable

Class schedule

The timetable for this course is available here: Portale EasyAcademy

EXAMS

EXAM DESCRIPTION

The examination consists of an oral assessment covering theoretical topics and a discussion of the practical exercises completed during the course. Students may take the examination as a single assessment or split it into two oral assessments covering the topics and practical exercises of the first and second semesters, respectively. The first assessment may be taken as early as the end of the first semester. Students must demonstrate an understanding of how machine learning models work and the ability to apply them to practical problems, with particular attention to optimisation and deployment on embedded devices. The discussion of practical exercises will cover data preparation, model selection and training, performance evaluation and the solutions adopted to meet the constraints of the target platform.

Students with valid certifications for Specific Learning Disabilities (SLD), disabilities, or other educational needs may request compensatory and/or exemption measures for the exam. The procedures will be defined on a case-by-case basis, in accordance with the University Regulations, in consultation with the instructor and the School's Disability and SLD Coordinator.

ASSESSMENT METHODS

The oral examination assesses students’ understanding of the course content and their ability to apply it to the design, evaluation and implementation of machine learning systems.

Students must:

• explain the concepts and algorithms covered, describing how they work, their conditions of application, advantages and limitations;
• justify the choice of data, preparation procedures, models and evaluation metrics;
• interpret experimental results, identifying generalisation problems and potential training issues;
• compare alternative solutions in terms of predictive performance and computational requirements;
• explain the optimisation techniques and design choices adopted for deployment on embedded devices;
• present their work clearly, using appropriate technical language.

Assessment will take into account the accuracy and completeness of responses, the ability to connect theory and practice, independent judgement and clarity of technical communication.

FURTHER INFORMATION

For further information not included in the course description, please contact the lecturer.