OVERVIEW

The teaching presents the theory of Markov chains, both discrete time and continuous ones, with particular regard to Poisson processes and queuing theory. The goal is to give the student the ability to translate concrete problems of stochastic evolution in terms of Markov chains (when possible), creating and analyzing probabilistic models associated with them.

AIMS AND CONTENT

LEARNING OUTCOMES

We want to introduce Markov chains and other simple stochastic processes in order to model and solve real problems of stochastic evolution.

AIMS AND LEARNING OUTCOMES

The goal is to let the student learn the language of Markov chains, so that he can be able of building an appropriate model from the real problems of stochastic evolutions taking values in a finite or countable set. The student will learn to classify these values and to determine the invariant laws with respect to these evolutions. We then will see as the study of the queueing theory can be translated, under some assumptions, in the language of Markov chains, and we will make the student able to study the efficiency of the model.

TEACHING METHODS

Teaching is done in the traditional way, with lectures held at the blackboard. Expected 2 theory lessons per week (4 hours) and 1 of exercises (2 hours).
At the end of the course there will be a guided full text-exercise in order to give students the opportunity to understand their degree of readiness and to clarify together possible doubts.

SYLLABUS/CONTENT

Discrete time Markov chains. Definition. Classification of states. Transience and recurrence criteria. Probability of absorbtion in recurrence classes. Invariant laws. Limit Theorems. Convergence to equilibrium. Applications: random walks.

Contnuous time Markov chains. Hitting time. Chapman-Kolmogorov equations. Invariant laws. Jumps chain. Born and death chains. Poisson processes.

Queueing theory.

RECOMMENDED READING/BIBLIOGRAPHY

P. Baldi, Calcolo delle Probabilità e Statistica Matematica

W. Feller, An introduction to Probability Theory and its Applications

S. Karlin, H.M. Taylor, A First Course in Stochastic Processes.

S. Karlin, H.M. Taylor, A Second Course in Stochastic Processes.

S.M. Ross, Introduction to Probability Models.

G. Grimmett, D. Stirzaker, (2001). Probability and Random Processes. 

J.R. Norris. Markov Chains.

P. Brémaud. Markov Chains: Gibbs Fields, Montecarlo Simulation, and Queues.

Notes

TEACHERS AND EXAM BOARD

Exam Board

VERONICA UMANITA' (President)

EVA RICCOMAGNO

EMANUELA SASSO

LESSONS

LESSONS START

The class will start according to the academic calendar.

Class schedule

STOCHASTIC PROCESSES

EXAMS

EXAM DESCRIPTION

Written test + oral test.
To participate in the written test you must register on the UNIGE site.
The written exam is only overcome by scoring greater than or equal to 16 points.

ASSESSMENT METHODS

The written test consists of 2 exercises, one on the discrete part and the other on the continuous one. The duration of the test is 3 hours and acces to the course notes (including exercises done in the classroom) and handouts is allowed.

The oral examination is aimed at assessing the general understanding of the subject, and it is required that the student knows how to properly expose the concepts seen in the course, to show the main results and to solve the exercises. The oral examination can be taken immediately after the written test or even in subsequent exam calls during the academic year in progress.

Exam schedule

FURTHER INFORMATION

Prerequisites:
Topics in Algebra, Probability.
More details at the "Course Teaching" web page on Aulaweb of the current academic year.