Syllabus

Review: Linear algebra, matrix calculus, probability and statistics. Supervised learning: linear regression (gradient descent, normal equations),Weighted Linear Regression (LWR), logistic regression, perceptron Newton's method, KL-divergence,(cross-) entropy, natural gradient, exponential family and generalized linear models, generative models (Gaussian discriminant analysis, Naive Bayes), Kernel Method (SVM, Gaussian processes), tree ensembles (decision trees, random forests, boosting and gradient boosting), learning theory, regularization, bias-variance decomposition and tradeoff, concentration inequalities, generalization and uniform convergence, VC-dimension.

Deep Learning: Neural networks, back propagation, deep architectures, unsupervised learning, K-means, Gaussian Mixture Model (GMM), Expectation Maximization (EM), Variational Auto-encoder (VAE), Factor Analysis, Principal Components Analysis (PCA), Independent Components Analysis (ICA), introduction to Reinforcement Learning (RL).

Application: Advice on structuring an ML project, evaluation metrics, missing data techniques and tracking.

Special Topic: Computer vision, NLP, machine listening and music information retrieval, speech, compressive sensing, array processing, beamforming, independent component analysis, MIMO/SIMO models, under- constrained separation, spectral factorizations.