CODICE 101799 ANNO ACCADEMICO 2020/2021 CFU 9 cfu anno 1 COMPUTER SCIENCE 10852 (LM-18) - GENOVA SETTORE SCIENTIFICO DISCIPLINARE INF/01 LINGUA Inglese SEDE GENOVA PERIODO 1° Semestre MATERIALE DIDATTICO AULAWEB PRESENTAZIONE We will study principles and algorithms, architectures and technologies, programming models and frameworks needed to support data-intensive applications OBIETTIVI E CONTENUTI OBIETTIVI FORMATIVI Learning the theoretical, methodological, and technological fundamentals of advanced data processing architectures, large-scale distributed environments, and data intensive programming including Docker, HDFS, Hadoop, Spark, and Cloud/IoT platforms. OBIETTIVI FORMATIVI (DETTAGLIO) E RISULTATI DI APPRENDIMENTO Labs will be focused on map-reduce architectures and libraries Technology: HDFS, Hadoop, Spark using Python and Java/Scala Intermediate lab activities (5/6 labs) Final project PREREQUISITI Sequential, Concurrent and Distributed Programming Database Theory and Practice Basic notions of Data Analysis MODALITA' DIDATTICHE Frontal lectures Lab sessions PROGRAMMA/CONTENUTO Distributed Systems and Distributed Programming Virtualization and containers Parallel Python Distributed data systems and shared nothing architectures Partitioning Replication Fault Tolerance CAP Theorem Map/Filter/Reduce and Generators in Python Map in Multiprocessing Introduction to Hadoop and HDFS Map Reduce Map Reduce: Simple Design Patterns and Relational Algebra Operators Hadoop Runtime System Apache Spark Apache Spark Internals PySpark, Java/Scala Spark Streaming Data Streaming Spark DOCENTI E COMMISSIONI GIORGIO DELZANNO Ricevimento: Su appuntamento in presenza o Teams BARBARA CATANIA Ricevimento: Su appuntamento, via email o Microsoft Teams Stanza: Valle Puggia – 327 GIOVANNA GUERRINI Commissione d'esame GIORGIO DELZANNO (Presidente) GIOVANNA GUERRINI BARBARA CATANIA (Presidente Supplente) FEDERICO DASSERETO (Supplente) LEZIONI Orari delle lezioni L'orario di questo insegnamento è consultabile all'indirizzo: Portale EasyAcademy ESAMI MODALITA' D'ESAME Final online test with open and closed questions Project presentation and discussion Bonusf for at least 70% attendance rate (if in presence) and lab assignments successfully delivered MODALITA' DI ACCERTAMENTO The proposed exercises, project and final test cover both conceptual and practical aspects presented in the course Calendario appelli Data appello Orario Luogo Tipologia Note 18/01/2021 09:00 GENOVA Esame su appuntamento 14/06/2021 09:00 GENOVA Esame su appuntamento 06/09/2021 09:00 GENOVA Esame su appuntamento 10/01/2022 09:00 GENOVA Esame su appuntamento