RNN’s and their use in machine learning

Neural networks are a fascinating field of computer science that attempts to model the brain in a mathematical sense. Unfortunately, they are nowhere near the level of complexity as the human brain, and will most likely not be in the near future. However, while there is no single network that can match the brain in its complexity and accuracy, there are specific networks that can rival the accuracy and speed of the human mind, though they often lag in the accuracy department.

One such example of this is the recurrent neural network, typically shortened to RNN. The RNN is significantly more accurate than a typical neural network, specifically when it comes to data that enter in sequences, such as sound and videos. To discuss why we need to know what an RNN is and how it is different than a neural network.

A typical neural network is relatively simple. For simplicity, this neural network will have two input features, one hidden layer, and one output neuron. The way it learns is simple. Feed some data forward through the neural network, it will come up with some output, and then check the prediction it made against the actual prediction. Based on the accuracy of the prediction, it will go back through the network and change specific values for each in hopes of getting a closer prediction. After a certain amount of iterations, it will come up with a model.

An RNN is different. Rather than only using the data it has once, it uses previous data to make a decision about the current one. This is why it is so good with sequences, like sentences. A sentence needs context and thus depends on previous words to interpret what is going on. Like I talked about in my previous blog about the Winograd schema, context is important to determine what each pronoun in a sentence is related to.

For example:

“I put the cloth on the table to protect it.”

While this is a very hard statement for a network to interpret, it still serves as an example of the benefit of RNNs. To determine what “it” is, we need to know what it means in the context of the statement, which means that we need to previous words in the sentence to know what exactly “it” means (in this case, it is the table).

Conclusion

RNN’s are a powerful tool in neural networks, and as I learn more about the world of neural networks, it is easy to see a connection to computer science. Both neural networks and data structures are actually fairly simple to implement, thanks to extensive libraries either built into a language or via 3rd parties. However, the real challenge is in interpreting which network, or data structure, to use in the case of any real-life applications.