OVERVIEW

This course introduces some of the main design issues for distributed systems (i.e., systems made of networked computers that communicate and coordinate by passing messages to each other).

Topics like performance, availability and security are discussed, and students make hands-on experience on simulated and real systems.

The course is presented in two versions: a 6 CFU version offered to the students of the Software Security & Engineering (SSE) curriculum, and a 9 CFU version offered to those of the Data Science & Engineering (DSE) curriculum. The 9 CFU version offers a more in-depth introduction to data-intensive systems such as Apache Hadoop and Spark

AIMS AND CONTENT

LEARNING OUTCOMES

Learning the fundamental design principles for distributed systems and getting more in-depth knowledge of a few chosen architectures relative to the students' curriculum; getting familiarity with the concepts of reliability, availability, and fault tolerance.

AIMS AND LEARNING OUTCOMES

For the 6 CFU version of the course:

• EXPLAIN the reasons for using (or not using) distributed systems
• DISCUSS the key motivations and ideas for solutions of the problem of distributed consensus
• SUMMARIZE some important results about scheduling for distributed systems
• DISCUSS the main motivations, issues and solutions related to the design of peer-to-peer architectures
• EXPLAIN the design of large-scale data processing tools
• EVALUATE the impact of some design choices through simulation

In addition, for the 9 CFU version of the course,

• SOLVE programming tasks efficiently using large-scale data processing tools

PREREQUISITES

To be successful in this course, students should already know:

• data structures and algorithms
• client-server architectures
• network protocols
• basic security topics

Programming skills are required for the practical activities, which will be carried out in Python.

TEACHING METHODS

Lectures and practical exercises.

SYLLABUS/CONTENT

• Transactions and Consensus
  • ACID properties and the CAP theorem
  • Consensus algorithms: Paxos, Raft
  • Concrete applications: Zookeeper, Google Cloud Spanner
• Queueing and Scheduling
  • Some theory on the problem of prioritizing and dispatching jobs to workers in a distributed system
• Erasure Coding
  • How to efficiently guarantee data availability
• Peer-to-Peer (P2P) Systems
  • Tor: a system for enabling anonymous communication
  • Searching in P2P systems: unstructured networks, superpeers, distributed hash tables
  • Incentives to cooperation: BitTorrent
  • Eventual consistency: the example of Amazon Dynamo
• Introduction to Large-Scale Data Processing
  • The MapReduce paradigm
  • The design of Hapace Hadoop and Apache Spark

For the 9 CFU version:

• Distributed data systems: overview, partitioning, replication
• Containerized architecture: Docker
• Practices with PySpark

RECOMMENDED READING/BIBLIOGRAPHY

Scientific papers covering the topics of the syllabus will be suggested during the course. The course material will be presented on Aulaweb.

TEACHERS AND EXAM BOARD

Exam Board

MATTEO DELL'AMICO (President)

GIORGIO DELZANNO (President Substitute)

LESSONS

LESSONS START

According to the calendar approved by the Degree Program Board: https://corsi.unige.it/en/corsi/11964/studenti-orario

Class schedule

The timetable for this course is available here: EasyAcademy

EXAMS

EXAM DESCRIPTION

6 CFU version:

Written exam and oral examination, with discussion of the written exam and of assignments.

9 CFU version:

Evaluation of assignments and written exam.

Guidelines for students with certified Specific Learning Disorders, disabilities, or other special educational needs are available at https://corsi.unige.it/en/corsi/11964/studenti-disabilita-dsa

ASSESSMENT METHODS

For each assignment, the quality of the code and the reports will indicate the technical skills acquired by the students.

The exam will allow the teachers to complete their evaluation with questions on the course program. 

FURTHER INFORMATION

For further information, please refer to the course’s AulaWeb module or contact the instructor.