Word2Vec
Word2Vec is a set neural network algorithms that have gotten a lot of attention in recent years as part of the re-emergence of deep learning in AI.
The idea that one can represent words and concepts as vectors is not new. The ability to do it effectively and generate noteworthy results is.
Word2Vec algorithms are especially interesting because they allow us to perform arithmetic on the word vectors that yield both surprising and satisfying results. We call these “word analogies”.
Some popular word analogies Word2Vec is capable of finding:
“King” is to “Man” as “Queen” is to “Woman”.
“France” is to “Paris” as “Italy” is to “Rome”.
“December” is to “November” as “July” is to “June”.
Not only can we cluster similar words together, we can make all these clusters have the same “structure”, all by using Word2Vec.
Word2Vec was created by a team led by Tomas Mikolov at Google and has many advantages over earlier algorithms that attempt to do similar things, like Latent Semantic Analysis (LSA) or Latent Semantic Indexing (LSI).
In this book we cover various popular flavors of the Word2Vec algorithm, including CBOW (continuous bag-of-words), skip-gram, and negative sampling.
I show you both their derivations in math (you’ll see that if you already are familiar with deep learning concepts, there is no new math to be learned), and how to implement them in code.
Whereas implementation in Numpy is just the straightforward application of the equations in code, Theano is a bit more complex because it requires new array-slicing techniques, namely running gradient descent on only a part of a matrix. It’s not straightforward, but I walk you through all the bits and pieces required to understand the full implementation.
Amazingly, all the technologies we discuss in this book can be downloaded and installed for FREE. That means all you need to invest after purchasing this book is your effort and your time. The only prerequisites are that you are comfortable with Python , Numpy, and Theano coding and you know the basics of deep learning.
“Hold up... what’s deep learning and all this other crazy stuff you’re talking about?”
If you are completely new to deep learning, you might want to check out my earlier books and courses on the subject, since they are required in order to understand this book:
Deep Learning in Python https://www.udemy.com/data-science-deep-learning-in-python
Deep Learning in Python Prerequisities https://www.udemy.com/data-science-logistic-regression-in-python
Much like how IBM’s Deep Blue beat world champion chess player Garry Kasparov in 1996, Google’s AlphaGo recently made headlines when it beat world champion Lee Sedol in March 2016.
What was amazing about this win was that experts in the field didn’t think it would happen for another 10 years. The search space of Go is much larger than that of chess, meaning that existing techniques for playing games with artificial intelligence were infeasible. Deep learning was the technique that enabled AlphaGo to correctly predict the outcome of its moves and defeat the world champion.
Deep learning progress has accelerated in recent years due to more processing power (see: Tensor Processing Unit or TPU), larger datasets, and new algorithms like the ones discussed in this book.
Book 2 in the series can be found at: https://www.amazon.com/dp/B01KRBOO4Y
Book 3 in the series can be found at: https://www.amazon.com/dp/B01KS5AEXO
