Tutorials

Survey Papers on Deep Learning

Yoshua Bengio, Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, 2(1), pp.1-127, 2009.

Yoshua Bengio, Aaron Courville, Pascal Vincent, Representation Learning: A Review and New Perspectives, Arxiv, 2012.

Deep Learning Code Tutorials

The Deep Learning Tutorials are a walk-through with code for several important Deep Architectures (in progress; teaching material for Yoshua Bengio’s IFT6266 course).

Unsupervised Feature and Deep Learning

Stanford’s Unsupervised Feature and Deep Learning tutorials has wiki pages and matlab code examples for several basic concepts and algorithms used for unsupervised feature learning and deep learning.

Videos

  • Deep Learning Representations
Yoshua Bengio’s Google tech talk on Deep Learning Representations at Google Montreal (Google Montreal, 11/13/2012)
  • Deep Learning with Multiplicative Interactions

Geoffrey Hinton’s talk at the Redwood Center for Theoretical Neuroscience (UC Berkeley, March 2010).

  • Recent developments on Deep Learning

Geoffrey Hinton’s GoogleTech Talk, March 2010.

  • Learning Deep Hierarchies of Representations 

general presentation done by Yoshua Bengio in September 2009, also at Google.

  • A New Generation of Neural Networks 

Geoffrey Hinton’s December 2007 Google TechTalk.

  • Deep Belief Networks

Geoffrey Hinton’s 2007 NIPS Tutorial [updated 2009] on Deep Belief Networks 3 hour video , ppt, pdf , readings

  • Training deep networks efficiently

Geoffrey Hinton’s talk at Google about dropout and “Brain, Sex and Machine Learning”.

  • Deep Learning and NLP
 Yoshua Bengio and Richard Socher’s talk, “Deep Learning for NLP(without magic)” at ACL 2012.
  • Tutorial on Learning Deep Architectures
Yoshua Bengio and Yann LeCun’s presentation at “ICML Workshop on Learning Feature Hiearchies” on June 18th 2009.

Energy-based Learning

[LeCun et al 2006]. A Tutorial on Energy-Based Learning, in Bakir et al. (eds) “Predicting Structured Outputs”, MIT Press 2006: a 60-page tutorial on energy-based learning, with an emphasis on structured-output models. The tutorial includes an annotated bibliography of discriminative learning, with a simple view of CRF, maximum-margin Markov nets, and graph transformer networks.

A 2006 Tutorial an Energy-Based Learning given at the 2006 CIAR Summer School: Neural Computation & Adaptive Perception.[Energy-Based Learning: Slides in DjVu (5.2MB), Slides in PDF (18.2MB)] [Deep Learning for Generic Object Recognition:Slides in DjVu (3.8MB), Slides in PDF (11.6MB)]

ECCV 2010 Tutorial

Feature learning for Image Classification (by Kai Yu and Andrew Ng): introducing a paradigm of feature learning from unlabeled images, with an emphasis on applications to supervised image classification.

NIPS 2010 Workshop

Deep Learning and Unsupervised Feature Learning: basic concepts about unsupervised feature learning and deep learning methods with links to papers and code.

Summer Schools

Graduate Summer School: Deep Learning, Feature Learning: IPAM summer school about deep learning.

 Online Courses

Geoffrey Hinton’s Online Neural networks Course on Coursera.