Survey Papers on Deep Learning
Yoshua Bengio, Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, 2(1), pp.1-127, 2009.
Deep Learning Code Tutorials
The Deep Learning Tutorials are a walk-through with code for several important Deep Architectures (in progress; teaching material for Yoshua Bengio’s IFT6266 course).
Videos
- Deep Learning with Multiplicative Interactions
Geoffrey Hinton’s talk at the Redwood Center for Theoretical Neuroscience (UC Berkeley, March 2010).
- Recent developments on Deep Learning
Geoff Hinton’s GoogleTech Talk, March 2010.
- Learning Deep Hierarchies of Representations
- A general presentation done by Yoshua Bengio in September 2009, also at Google.
- A New Generation of Neural Networks
- Geoffrey Hinton’s December 2007 Google TechTalk.
- Deep Belief Networks
- Geoffrey Hinton’s 2007 NIPS Tutorial [updated 2009] on Deep Belief Networks
3 hour video,ppt,pdf,readings
Energy-based Learning
[LeCun et al 2006]. A Tutorial on Energy-Based Learning, in Bakir et al. (eds) “Predicting Structured Outputs”, MIT Press 2006: a 60-page tutorial on energy-based learning, with an emphasis on structured-output models. The tutorial includes an annotated bibliography of discriminative learning, with a simple view of CRF, maximum-margin Markov nets, and graph transformer networks.
A 2006 Tutorial an Energy-Based Learning given at the 2006 CIAR Summer School: Neural Computation & Adaptive Perception.[Energy-Based Learning: Slides in DjVu (5.2MB), Slides in PDF (18.2MB)] [Deep Learning for Generic Object Recognition:Slides in DjVu (3.8MB), Slides in PDF (11.6MB)]
ECCV 2010 Tutorial
Feature learning for Image Classification (by Kai Yu and Andrew Ng): introducing a paradigm of feature learning from unlabeled images, with an emphasis on applications to supervised image classification.
NIPS 2010 Tutorial
Deep Learning and Unsupervised Feature Learning: basic concepts about unsupervised feature learning and deep learning methods with links to papers and code.