The first 2 books in this series focused on word embeddings using 2 novel techniques: Word2Vec and GLoVe.
In this book, we return to a classic NLP problem: sentiment analysis. Classification performance on the sentiment analysis task had plateaued for many years, due to not being able to handle negation, which is essentially because existing models failed to account for the structure of language.
The bag-of-words vectors for “I love this movie”, “I don’t love this movie”, and “Don’t you love this movie” are very similar.
In this book, we return to the fundamentals of language - the parse tree - and structure our neural networks to mirror the tree.
It makes sense that a neural network created to classify language would have the same structure as language.
These neural networks are called “recursive neural networks” and I will show you how they work both mathematically and with a full implementation in Theano.
A naive solution to recursive neural networks would be to use recursion to implement them. This is however, a very poor solution because both Theano and TensorFlow require you to compile a graph of the neural network. If every sentence is a different tree, then every sentence will require a different neural network graph, which would be very inefficient for both Theano or TensorFlow to compute. No one would blame you for attempting this solution first. In fact, I will demonstrate why it’s bad by having you run code that implements it.
Once you understand why recursion is not ideal for recursive neural networks, I will show you a “trick” that will help you implement them more efficiently. We will then run the recursive neural net on our sentiment analysis data and achieve state-of-the-art performance.
Finally, we discuss a modification to the vanilla recursive neural network called the recursive neural tensor network or RNTN. It was invented by the guys at Stanford, who have created and published many NLP tools throughout the years that are now considered standard.
I will show you how the model is structured mathematically and then I will show you how to implement it in Theano. You’ll see that it’s just a simple modification to our recursive neural network.
Amazingly, all the technologies we discuss in this book can be downloaded and installed for FREE. That means all you need to invest after purchasing this book is your effort and your time. The only prerequisites are that you are comfortable with Python , Numpy, and Theano coding and you know the basics of deep learning.
If you are completely new to deep learning, you might want to check out my earlier books and courses on the subject, since they are required in order to understand this book:
Deep Learning in Python https://www.udemy.com/data-science-deep-learning-in-python
Deep Learning in Python Prerequisities https://www.udemy.com/data-science-logistic-regression-in-python
Much like how IBM’s Deep Blue beat world champion chess player Garry Kasparov in 1996, Google’s AlphaGo recently made headlines when it beat world champion Lee Sedol in March 2016.
What was amazing about this win was that experts in the field didn’t think it would happen for another 10 years. The search space of Go is much larger than that of chess, meaning that existing techniques for playing games with artificial intelligence were infeasible. Deep learning was the technique that enabled AlphaGo to correctly predict the outcome of its moves and defeat the world champion.
Deep learning progress has accelerated in recent years due to more processing power (see: Tensor Processing Unit or TPU), larger datasets, and new algorithms like the ones discussed in this book.
Book 1 in the series can be found at: https://www.amazon.com/dp/B01KQ0ZN0A
Book 2 in the series can be found at: https://www.amazon.com/dp/B01KRBOO4Y
In this book, we return to a classic NLP problem: sentiment analysis. Classification performance on the sentiment analysis task had plateaued for many years, due to not being able to handle negation, which is essentially because existing models failed to account for the structure of language.
The bag-of-words vectors for “I love this movie”, “I don’t love this movie”, and “Don’t you love this movie” are very similar.
In this book, we return to the fundamentals of language - the parse tree - and structure our neural networks to mirror the tree.
It makes sense that a neural network created to classify language would have the same structure as language.
These neural networks are called “recursive neural networks” and I will show you how they work both mathematically and with a full implementation in Theano.
A naive solution to recursive neural networks would be to use recursion to implement them. This is however, a very poor solution because both Theano and TensorFlow require you to compile a graph of the neural network. If every sentence is a different tree, then every sentence will require a different neural network graph, which would be very inefficient for both Theano or TensorFlow to compute. No one would blame you for attempting this solution first. In fact, I will demonstrate why it’s bad by having you run code that implements it.
Once you understand why recursion is not ideal for recursive neural networks, I will show you a “trick” that will help you implement them more efficiently. We will then run the recursive neural net on our sentiment analysis data and achieve state-of-the-art performance.
Finally, we discuss a modification to the vanilla recursive neural network called the recursive neural tensor network or RNTN. It was invented by the guys at Stanford, who have created and published many NLP tools throughout the years that are now considered standard.
I will show you how the model is structured mathematically and then I will show you how to implement it in Theano. You’ll see that it’s just a simple modification to our recursive neural network.
Amazingly, all the technologies we discuss in this book can be downloaded and installed for FREE. That means all you need to invest after purchasing this book is your effort and your time. The only prerequisites are that you are comfortable with Python , Numpy, and Theano coding and you know the basics of deep learning.
“Hold up... what’s deep learning and all this other crazy stuff you’re talking about?”
If you are completely new to deep learning, you might want to check out my earlier books and courses on the subject, since they are required in order to understand this book:
Deep Learning in Python https://www.udemy.com/data-science-deep-learning-in-python
Deep Learning in Python Prerequisities https://www.udemy.com/data-science-logistic-regression-in-python
Much like how IBM’s Deep Blue beat world champion chess player Garry Kasparov in 1996, Google’s AlphaGo recently made headlines when it beat world champion Lee Sedol in March 2016.
What was amazing about this win was that experts in the field didn’t think it would happen for another 10 years. The search space of Go is much larger than that of chess, meaning that existing techniques for playing games with artificial intelligence were infeasible. Deep learning was the technique that enabled AlphaGo to correctly predict the outcome of its moves and defeat the world champion.
Deep learning progress has accelerated in recent years due to more processing power (see: Tensor Processing Unit or TPU), larger datasets, and new algorithms like the ones discussed in this book.
Book 1 in the series can be found at: https://www.amazon.com/dp/B01KQ0ZN0A
Book 2 in the series can be found at: https://www.amazon.com/dp/B01KRBOO4Y
