Blog « Deep Learning

LISA Lab Wins the Final Phase of UTLC Challenge

The Unsupervised and Transfer Learning Challenge is now officially over ! The LISA lab performed very well, winning top honours in Phase 2, which consisted in unsupervised learning of robust features that would transfer to a new distribution with new classes. Full contest results can be found here. Unsurprisingly, our strategy relied heavily on deep learning methods, notably the recent Contractive Auto-Encoder and Spike & Slab RBM.

A JMLR submission detailing our methodology is currently under review for publication in the fall. We will post an update on deeplearning.net when it is made available.

TPAMI’s Special Issue on Learning Deep Architectures.

Working on something you think might be of interest to the deep learning community? Consider submitting a manuscript to TPAMI’s Special Issue on Learning Deep Architectures.

See the Call for Papers for more details here: http://www.computer.org/cms/Computer.org/transactions/cfps/cfp_tp_lda.pdf

Deep Networks Advance State of Art in Speech

Deep Learning leads to breakthrough in speech recognition at MSR.

Speech Recognition Leaps Forward – Microsoft Research

Dong Yu, researcher at Microsoft Research Redmond, and Frank Seide, senior researcher and research manager with Microsoft Research Asia, have been spearheading this work, and their teams have collaborated on what has developed into a research breakthrough in the use of artificial neural networks for large-vocabulary speech recognition.

New Challenge Announced

The Unsupervised and Transfer Learning Challenge is being held until April 15. The goal of the challenge is to “[...] devise preprocessing algorithms to create good data representations. The algorithms can be trained with unlabeled data only during phase 1 (unsupervised learning). Some labels (from other classes than those used for evaluation) will be made available during phase 2 (transfer learning).”

There are prizes to be gained and the first phase submissions are being accepted until February 28, 2011.

Deep Learning Workshop at NIPS 2010

There is a workshop devoted to advances in deep learning at this year’s NIPS conference. The workshop schedule and list of accepted papers are available at:

http://deeplearningworkshopnips2010.wordpress.com/schedule/

New Events Page

We’ve added a new events page, containing links to workshops and meetings that are of interest to the deep learning community. Contact us if we missed any or if you’re organizing an event and want it added to the list.

Deep Learning papers at ICML 2010

From the list of accepted papers, judging by title only:

Learning Fast Approximations of Sparse Coding
Karol Gregor, Yann LeCun

Boosted Backpropagation Learning for Training Deep Modular Networks
Alexander Grubb, Drew Bagnell

Deep learning via Hessian-free optimization
James Martens

3D Convolutional Neural Networks for Human Action Recognition
Shuiwang Ji, Wei Xu, Ming Yang, Kai Yu

Restricted Boltzmann Machines are Hard to Approximately Evaluate or Simulate
Phil Long, Rocco Servedio

Deep Supervised T-Distributed Embedding
Renqiang Min, Zineng Yuan, Laurens van der Maaten, Anthony Bonner, Zhaolei Zhang

Deep networks for robust visual recognition
Yichuan Tang, Chris Eliasmith

Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair, Geoffrey Hinton

Learning Deep Boltzmann Machines using Adaptive MCMC
Ruslan Salakhutdinov

A theoretical analysis of feature pooling in vision algorithms
Y-Lan Boureau, Jean Ponce, Yann LeCun

Deep networks for robust visual recognition
Yichuan Tang, Chris Eliasmith

Deep learning papers at AISTATS 2010

Full proceedings

Learning the Structure of Deep Sparse Graphical Models
Ryan Adams, Hanna Wallach, Zoubin Ghahramani

Why Does Unsupervised Pre-training Help Deep Learning?
Dumitru Erhan, Aaron Courville, Yoshua Bengio, Pascal Vincent

Efficient Learning of Deep Boltzmann Machines
Ruslan Salakhutdinov, Hugo Larochelle

Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot, Yoshua Bengio

Factored 3-Way Restricted Boltzmann Machines For Modeling Natural Images
Marc’Aurelio Ranzato, Alex Krizhevsky, Geoffrey Hinton

Inductive Principles for Restricted Boltzmann Machine Learning.
Benjamin Marlin, Kevin Swersky, Bo Chen and Nando de Freitas.

Welcome

deeplearning.net is finally up!

Deep Learning

Recent Posts

Meta