Nauczanie maszynowe

Uczenie maszynowe

Spring 2016

Instructor: Igor Podolak Time: Lecture Wed 12:00 – 14:00

Email: igor.podolak@uj.edu.pl

Place: 2171 Wydzial Matematyki

i Informatyki UJ

Instructors: Igor Podolak Time: 14.00 – 16.00

Agnieszka Pocha

Maciej Brzeski

Office Hours: After class, or in o.ce hours, or by appointment, or post your question on the ML forum.

Main References: This is a restricted list of various interesting and useful books that will be touched during the course. You need to consult them occasionally.

Christopher M. Bishop, Pattern Recognition and Machine Learning , Springer, 2006.

Sergios Theodoridis, Machine Learning, a Bayesian and optimization perspective , Academic Press 2015.

H Trevor Hastie, Robert Tibshiranie, Jerome Friedman, The elements of Statistical Learning, 2nd ed., Springer 2009.

Peter Flach, Machine Learning: The Art and Science of Algorithms that Make Sense of Data , Cam.bridge University Press, 2012.

Gavin, Hackeling, Mastering machine learning with scikit-learn , Packt Publishing 2014.

Daphne Koller, Nir Friedman, Probabilistic graphical models, MIT Press 2009.

Objectives: This course is primarily designed for graduate students for them to learn the basics of machine learning. We shall start with the methods for linear models’ building, the definition of a model itself, approaches for model evaluation and validation.

The classes accompanying the lectures shall dig deeper into the problems. Students are expected to interpret the mathematics and implement algorithms using Python programming language. Students shall build some implementations of learning models, build and evaluate solutions for given problems, find out how to cope with live problems using ML methods. Students shall gain knowledge of available ML libraries, mainly these in Python.

Prerequisites: An undergraduate-level understanding of probability, statistics, graph theory, algorithms, and linear algebra is assumed, as well as some calculus.

Python programming skills are greatly welcome.

Tentative Course Outline (this is very tentative and shall most probably be shorter):

1. What the hell is all that thing called machine learning about?

•

what is it to learn?

• regression and classification

• supervised vs unsupervised training

• model evaluation and validation, comparing models, statistical tests

2. Mean squares linear estimation

• geometric interpretation

• solutions for some problems: image deblurring by deconvolution, noise cancelation, Kalman filters

3. Least Mean Squares and gradient descent algorithms

• Least Mean Squares algorithm

• mean square error cost function, steepest descent approach

• convergence, step sizes

• preprocessing data and its impact on the algorithm

• the stochastic gradient descent

• regularization methods: Ockham’s razor, ridge regression and lasso methods,

4. Approaches to classification

• what is classification and what di.ers it from regression problems

• the Bayesian classifier model, the Na¨ıve Bayes classifier

• nearest neighbour model

• logistic regression

• classification trees, random forests

• combined models: bagging, stacking, etc.

• boosting models, AdaBoost

• Linear Discriminant Analysis LDA

• sparse solution search, .0, .1, .2 minimizers

5. Kernel methods

• Reproducing Kernel Hilbert Spaces RKHS

• Support Vector Machines SVM

• Support Vector Regression

• problem of choosing right parameters

6. Bayesian learning

• the maximum likelihood ML and the maximum a posteriori MAP estimators

• the expectation-maximization EM algorithm

7. Additive models and ensemble learning

• decision trees

• bagging and boosting trees, random forests

• Boosting methods, AdaBoost approach

• impact of boosting on bias and variance of models

• training other ensembles of models

8. Unsupervised learning

• Association rules

• clustering methods: K-means and hierarchical

• Self-organizing maps as example of clustering

9. Graphical models

• inference, causality

• separation, sigmoidal bayesian networks

• conditional random fields CRF

• Hidden Markov Models HMM

• learning graphical models

10. Dimensionality reduction

• intrinsic dimensionality of problems, curse of dimensionality

• PCA, SVD, ICA, etc.

• manifold learning

• eigenfaces generation

Grading Policy:

Homeworks and quizzes (30%), Midterm (20%), Three to four pro jects (40% = 4 * 10%), Attention and activ.ity (10%)

Final (30%)

Important Dates: ¯

Midterm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ab¯an 16, 1393

Final Exam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dey 18, 1393

Course Policy:

• Please sign up for MUM the usual way.

Class Policy:

• Regular attendance is essential and expected.

Academic Honesty: Lack of knowledge of the academic honesty policy is not a reasonable explanation for a violation.

Christopher M. Bishop, Pattern Recognition and Machine Learning , Springer, 2006.

Sergios Theodoridis, Machine Learning, a Bayesian and optimization perspective , Academic Press 2015.

H Trevor Hastie, Robert Tibshiranie, Jerome Friedman, The elements of Statistical Learning, 2nd ed., Springer 2009.

Peter Flach, Machine Learning: The Art and Science of Algorithms that Make Sense of Data , Cam.bridge University Press, 2012.

Gavin, Hackeling, Mastering machine learning with scikit-learn , Packt Publishing 2014.

Daphne Koller, Nir Friedman, Probabilistic graphical models, MIT Press 2009.