Neural Nets
Purpose of this page
Here I list everything interesting that I encountered on my journey of learning neural nets, for my reference, and for anyone who could find such a list useful in their own learning ;).
Courses
- CS231n: Convolutional Neural Networks for Visual Recognition at Stanford
- Machine Learning at Oxford:
Resources for learning about Neural Networks
- Andrej Karpathy: Blog
- Andrej Karpathy: Hacker’s guide to Neural Networks
- Geoffrey Hinton (the father of NN) Google Tech Talk about The Next Generation of Neural Networks
- Yann LeCun: Obstacles on the path to AI
- What is the class of this image? Article comparing the current state of the cart algorithms in object classification.
Articles and motivation for studying Neural Networks
- Interview with Yann Lecun: I’m going to make Facebook’s AI predict what happens in videos
- Computer, respond to this email. (Google prototype NN for replying to your email (demo in Google Inbox)
- Journey and achievement of AI in Facebook. Deep from the history till the date!
- Helping people to see with DNs
- Visual Question and Answering Demo - MemNet (natural language understanding system) combined with image recognition technology in a system called Visual Q&A (VQA) that answers to questions about photos
- EmTech 2015
- Facebook’s AI can caption photos for the blind on its own
- 19 A.I. experts reveal the biggest myths about robots
- System that replaces human intuition with algorithms outperforms human teams
- Facebook’s Working On A Tool To Help The Blind “See” Images
- CIFAR-10 Competition Winners: Interviews with Dr. Ben Graham, Phil Culliton, & Zygmunt Zając
ConvNets Articles
ConvNets
- Andrej Karpathy: ConvNetJS - Deep Learning in your browser
- CS231n Convolutional Neural Networks for Visual Recognition
- CS231n Whole Course on GitHub
- [Deep Learning Oxford Course, Lecture 10: Convolutional Neural Networks]
Advanced
- Alex Krizhevsky NN homepage
- Alex Krizhevsky: Convolutional Deep Belief Networks on CIFAR-10
- 2005. G. Hinton and V. Nair. Inferring Motor Programs from Images of Handwritten Digits.
BackProp
Deep Learning
Books
Deep Belief Networks
Slides
Deep-Structured Learning
Restricted Boltzmann Machines (RBMs)
- April 2009. Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images.
- Restricted Boltzmann Machines on Multi-Core Processors
Natural Language Processing (NLP)
- 2014. D. Tang, F. Wei, B. Qi, T. Liu, M. Zhou. Coooolll: A Deep Learning System for Twitter Sentiment Classification∗
- 2014. A Convolutional Neural Network for Modelling Sentences
- 2011. R. Collobert, J. Weston, L. Bottou M. Karlen, K. Kavukcuoglu, P. Kuksa. Natural Language Processing (Almost) from Scratch
- 2008. R. Collbert, J. Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning.
- 1989! A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K. Lang. Phoneme Recognition using Time-Delay Neural Networks
Homepages
Proceedings
Tools
Torch
Basics
Torch Installation in short ```bash
install dependencies
curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-deps | bash
Clone with dependencies into desired directory
git clone https://github.com/torch/distro.git /opt/torch –recursive
cd the install directory and run installation script
cd /opt/torch && ./install.sh
Update current session
source ~/.bashrc
Run Torch REPL
th ```
- Torch Cheatsheet
- Torch reference for numpy users
- Torch reference for Matlab users
- Lua in 15 minutes
- Torch Tutorials
Articles
- Torch vs Theano comparison
- Torch vs Theano in a context of deep learning
- Loading data in Torch (is a mess)
Slides
- Torch7 - scientific computing for Lua(JIT)
- Deep learning with Torch - the good, the bad, the ugly since 2002.
Advanced
- Torch Autograd by Twitter for automatically differentiating native Torch code
- Autograd for Torch by Twitter.
TensorFlow
TensorFlow (Apache 2.0 source license) is the tool behind automatic labelling of items in YouTube videos and photos, improving speech recognition in Google apps (e.g. when it is noisy), and used across different products such as Google Calendar events creation based on email (e.g. itinerary from ticket receipts), or even the automatic email reply prototype.
It scales on multiple CPUs and GPUs…
Articles:
TensorFlow source code on GitHub