OVERVIEW

The course presents algorthms for autonomous intelligent agents that interact with an unknown environment. In particular, the environment is simulated by a virtual world created through video games technology. The course will introduce the Reinforcement Learning e del Deep Reinforcement Learning approaches.

AIMS AND CONTENT

LEARNING OUTCOMES

The course provides algorithms and strategies to develop autonomous agents using a game engine.

AIMS AND LEARNING OUTCOMES

Regular attendance and active participation in the proposed educational activities (lectures and technical-practical labs), along with individual study, will allow you to:

• acquire the correct terminology to adequately describe the behavior of an autonomous agent.
• gain in-depth knowledge of the main reinforcement learning algorithms and be able to critically analyze their differences.
• be capable of quantitatively evaluating the performance of an autonomous agent in a given problem.
• formulate a real-world problem in terms of autonomous agents.
• apply the algorithms to real problems using the Python programming language

PREREQUISITES

To effectively engage with the content of the course, it is necessary to have the following foundational knowledge: programming skills with a high-level programming language, preferably Python (in order to understand and implement the algorithms and exercises covered in the course); and familiarity with the supervised machine learning topic (in order to understand the algorithms based on function approximators).

TEACHING METHODS

The course consists of both lectures and practical laboratory sessions. During the lectures, the instructor presents the topics and provides code examples that are demonstrated using OpenAI Gym environments. Students are encouraged to use their own computers during the lectures to practice the concepts taught by the instructor. In the practical laboratory sessions, the course instructor, along with laboratory tutors, guides students in using their computers to solve real-world problems by applying the algorithms covered in the lectures

SYLLABUS/CONTENT

The following list outlines the topics covered in the course. The slides and notebooks with notes used during the lectures can be downloaded from the provided link: https://github.com/riccardoberta/autonomous-agents

1. Introduction to autonomous agents and reinforcement learning
2. Markov Decision Processes (MDPs)
3. Dynamic Programming for reinforcement learning
4. Monte Carlo methods
5. Temporal Difference (TD) learning
6. Q-learning and SARSA algorithms
7. Function approximation in reinforcement learning
8. Policy gradient methods
9. Exploration and exploitation strategies
10. Deep Reinforcement Learning

By accessing the GitHub repository, you will have access to the lecture materials, including slides and notebooks, to further explore and study the course topics.

The United Nations has formulated a list of 17 goals under the 2030 Agenda for Sustainable Development, ranging from eradicating poverty and hunger worldwide to improving health and education, and addressing inequality and injustice. There are several challenges that the development of autonomous agents must address to be effectively deployed for social good, such as limited data for learning, geographical imbalances, partial data, and learning with limited memory and computational power. The course will adopt a responsible design perspective to provide students with the ability to develop socially intelligent systems. Specifically, students will learn how models can be designed without biases (Goal 10: Reduced Inequalities), how they can be made explainable (Goal 16: Peace, Justice, and Strong Institutions), and how to mitigate ethical risks posed by autonomous behaviors (Goal 16: Peace, Justice, and Strong Institutions). Furthermore, when presenting implementation examples, scenarios will be used to promote sustainable use of natural resources (Goal 13: Climate Action, Goal 15: Life on Land, Goal 14: Life Below Water), improving quality of life in cities (Goal 11: Sustainable Cities and Communities, Goal 3: Good Health and Well-being), and industries (Goal 9: Industry, Innovation, and Infrastructure). By the end of the course, students will be aware of the pitfalls that can make autonomous agent behavior irresponsible, and will know how to design responsible models that assist rather than harm people, while still being useful and accurate.

RECOMMENDED READING/BIBLIOGRAPHY

All the slides, source code, and other teaching materials used during the lectures will be available on the Aul@web platform. In general, the lecture notes and materials on Aul@web are sufficient for exam preparation. The following books are suggested as supplementary texts, but students can also use other recent university-level books on Reinforcement Learning:

• Miguel Morales, Grokking Deep Reinforcement Learning. Manning.
• Richard S. Sutton, Andrew G. Barto, Reinforcement Learning: An Introduction. MIT Press.

These books can provide additional insights and resources for further study on the topic of Reinforcement Learning.

TEACHERS AND EXAM BOARD

Exam Board

RICCARDO BERTA (President)

ALESSANDRO DE GLORIA

FRANCESCO BELLOTTI (President Substitute)

LESSONS

LESSONS START

https://corsi.unige.it/8732/p/studenti-orario

Class schedule

The timetable for this course is available here: Portale EasyAcademy

EXAMS

EXAM DESCRIPTION

The exam consists of a written test covering both the theoretical and practical topics presented in class. Specifically, students are expected to demonstrate a thorough understanding of the functioning of the algorithms underlying the development of autonomous agents and their ability to implement them in the Python programming language. The exam will assess the student's comprehension of the concepts, their ability to apply the algorithms, and their practical implementation skills.

ASSESSMENT METHODS

Details regarding the exam preparation methods and the level of depth required for each topic will be provided during the course of the lectures. The written exam will assess the actual acquisition of fundamental knowledge on Reinforcement Learning algorithms and their application for building autonomous agents. The presented problems and open-ended questions will allow the evaluation of students' ability to apply their knowledge in practical situations that may arise in real-world scenarios.

Exam schedule