Syllabus

Prerequisite: Linear algebra, Probability, and interest in programming

Description: Deep learning methods are now prevalent in the area of machine learning, and are now used invariably in many research areas. In recent years it received significant media attention as well. The influx of research articles in this area demonstrates that these methods are remarkably successful at a diverse range of tasks. Namely self-driving cars, new kinds of video games, AI, Automation, object detection and recognition, surveillance tracking etc. The proposed course aims at introducing the foundations of Deep learning to various professionals who are working in the area of automation, machine learning, artificial intelligence, mathematics, statistics, and neurosciences (both theory and applications). This is a proposed course to introduce Neural networks and Deep learning approaches (mainly Convolutional Neural networks) and give a few typical applications, where and how they are applied. The following topics will be explored in the proposed course. We will cover a range of topics from basic neural networks, convolutional and recurrent network structures, deep unsupervised and reinforcement learning, and applications to problem domains like speech recognition and computer vision.

Prerequisites: a strong mathematical background in calculus, linear algebra, and probability & statistics, as well as programming in Python and C/C++. There will be assignments and a final project.

1. Introduction to Visual Computing and Neural Networks

2. Basics of Multilayer Perceptron to Deep Neural Networks with Autoencoders

3. Unsupervised deep learning

4. Autoencoders for Representation Learning and MLP Initialization

5. Stacked, Sparse, Denoising Autoencoders and Ladder Training

6. Cost functions, Learning Rate Dynamics and Optimization

7. Introduction to Convolutional Neural Networks (CNN) and LeNet

8. Convolutional Autoencoders and Deep CNN (AlexNet, VGGNet)

9. Very Deep CNN architecture for Classification (GoogLeNet, ResNet, DenseNet)

10. Computational Complexity and Transfer Learning of a Network

11. Object Localization (RCNN) and Semantic Segmentation

12. Generative Models with Adversarial Learning

13. Recurrent Neural Networks (RNN) for Video Classification

14. Deep reinforcement learning

15. NLP/Vision Application