Skip to main content

My first deep learning steps with Google and Udacity

I did my first steps in deep learning by taking the deep learning course at Udacity.

Deep learning is a hot topic. Deep neural networks can classify images, describe scenes, translate text and do so much more. It's great that Google and Udacity offer this course which helped me getting started with learning about deep learning.


Image originally shows me hiking in Switzerland.
Deep neural networks at 
 Deep Dream Generator turned the image into a dreamy scene.

How does the course work?

The course consists of dozens 1-2 minute videos and assignments accompanying the videos.

Well, actually it's the other way round: The assignments are the heart of the course and the videos just give you the basic understanding you need to get started building networks. There are no exams.

The course covers basic neural networks, softmax, stochastic gradient descent, backpropagation, ReLU units, hidden layers, regularization, dropout, convolutional networks, recurrent networks, LSTM cells and more. Building deep neural networks is a bit like playing Legos and the course shows you the building bricks and teaches you how to use them.

In the assignments you build and optimize deep neural networks that read hand-written letters from images, learn the meaning of words (a la word2vec), can produce texts and flip words in sentences. The assignments are all based on Google's Tensorflow open source library.

The course is extremely hands-on and not self contained. The videos are short with a length of only 1-2 minutes. This concept is very different from the 10-15 minute videos you have in most coursera MOOCs covering all the needed materials. Udacity's deep learning course just gives you an intuitive understanding of the concepts. I found myself reading a lot of other blogs and tutorials to get a better understanding of the course contents and to know enough for the assignments.  I encourage you to read the papers that are linked in the assignments. They are well written and help you to get the assignments done.

I found the explanations in the videos of high quality. They often visualize the concepts and they did a good job at abstracting the concepts.

The assignments can be a bit difficult and you should expect to work a lot on them. Fortunately you can get help in the course forum and also help your peers. I found the community to be very friendly and helpful.

What prior knowledge is needed?

I agree with the prerequisites listed by Udacity: You should have at least 2 years of programming experience, ideally in Python, and you should be able to fork and pull the TensorFlow git repository from Github. The course requires some knowledge in mathematics (matrix multiplication, differentiation, integration and partial derivatives), basic statistics (mean, variance, standard deviation) and basic machine learning concepts. Prior experience with TensorFlow isn't needed. Experience in working with image or text data is also not necessary.

I think you should feel comfortable to read scientific papers, since the assignments encourage you to do so.

All deep neural networks in the assignments can be done with CPUs of most newer laptops (I used an MacBook Pro 13', 2014 model).

The course is self-paced and took me roughly 2.5 months and  4 - 8 hours per week to finish.

Should you take the course?

Yes, given you want to get first hands-on experience in deep learning and/or want to learn TensorFlow. 

Overall the course was fun and I learned a lot.  Even if I am not a big fan of the short videos concept. I wished the videos were a bit longer and had more content.

If you want to do a hands-on MOOC in deep learning there are no other options. For learning the fundamentals you can take coursera Machine Learning class or Neural Networks for Machine Learning.

For me attending the deep learning course lowered the barrier to get into deep learning and I am excited to start building my own deep neural networks.

Comments

Popular posts from this blog

Explaining the decisions of machine learning algorithms

Being both statistician and machine learning practitioner, I have always been interested in combining the predictive power of (black box) machine learning algorithms and the interpretability of statistical models.

I thought the only way to combine predictive power and interpretability is by using methods that are somewhat in the middle between 'easy to understand' and 'flexible enough', like decision trees or the RuleFit algorithm or, additionally, by using techniques like partial dependency plots to understand the influence of single features. Then I read the paper "Why Should I Trust You" Explaining the Predictions of Any Classifier [1], which offers a really decent alternative for explaining decisions made by black boxes.


What is LIME? The authors propose LIME, an algorithm for Local Interpretable Model-agnostic Explanations. LIME can explain why a black box algorithm assigned a specific classification/prediction to one datapoint (image/text/tabular data) b…

Statistical modeling: two ways to see the world.

This a machine-learning-vs-traditional-statistics kind of blog post inspired by Leo Breiman's "Statistical Modeling: The Two Cultures". If you're like: "I had enough of this machine learning vs. statistics discussion,  BUT I would love to see beautiful beamer-slides with an awesome font.", then jump to the bottom of the post and for my slides on this subject plus source code.

I prepared presentation slides about the paper for a university course. Leo Breiman basically argued, that there are two cultures of statistical modeling:
Data modeling culture: You assume to know the underlying data-generating process and model your data accordingly. For example if you choose to model your data with a linear regression model you assume that the outcome y is normally distributed given the covariates x. This is a typical procedure in traditional statistics. Algorithmic modeling culture:  You treat the true data-generating process as unkown and try to find a model that is…