Why it’s great to master Recurrent Neural Networks (RNN)?

Mastering Recurrent Neural Networks (RNN) offers several advantages in the field of machine learning and time-series analysis:

1. Effective modeling of sequential data: RNNs are specifically designed to handle sequential data, making them powerful tools for analyzing and predicting time-series data, natural language, speech, and other sequential data types.
2. Capturing long-term dependencies: RNNs excel at capturing long-term dependencies in sequential data, allowing them to retain information from previous time steps and make informed predictions based on context.
3. Versatile applications: RNNs have a wide range of applications, including language modeling, machine translation, sentiment analysis, speech recognition, stock market prediction, and weather forecasting.

What will be covered in this tutorial+

In this tutorial on Recurrent Neural Networks (RNN), we will cover the following topics:

• Understanding RNN: Exploring the concept of RNN and its architecture, which enables the propagation of information across time steps, facilitating the modeling of sequential data.
• Building an RNN on a time-series: Implementing an RNN model on a time-series dataset to analyze patterns, trends, and make predictions based on historical data.
• Theory of RNN (LSTM cells): Diving into the theoretical foundations of RNN, focusing on Long Short-Term Memory (LSTM) cells, which help RNNs overcome the vanishing gradient problem and capture long-term dependencies effectively.
• Data preprocessing with MinMaxScaler: Preparing the time-series data for RNN modeling by using the MinMaxScaler from the sklearn library to scale the values within a specific range, ensuring optimal model performance.
• Creating an RNN model with TensorFlow: Implementing an RNN model using the TensorFlow framework, a popular deep learning library, to train and evaluate the model on the time-series data.
• Applying Dropout techniques: Utilizing Dropout, a regularization technique, to prevent overfitting in the RNN model by randomly disabling a portion of the neurons during training, enhancing generalization capabilities.
• Predicting stock prices and making weather forecasts using RNN: Applying the learned concepts and techniques to real-world scenarios, specifically predicting stock prices and making weather forecasts, showcasing the practical applications of RNN in time-series analysis.

By mastering these concepts and techniques, you will gain valuable skills in understanding and implementing Recurrent Neural Networks, enabling you to effectively model and analyze sequential data, make accurate predictions, and unlock a wide range of applications in various domains.

Step 1: Feed-forward vs Recurrent Neural Network

Neural Network that has a connection only in one direction is called Feed-Forward Neural Network (Examples: Artificial Neural Network, Deep Neural Network, and Convolutional Neural Network).

A Recurrent Neural Network is a Neural Network that generates output that feeds back into its own inputs. This enables it to do one-to-many and many-to-many relationships (not possible for feed-forward neural networks).

An example of one-to-many is a network that can generate sentences (while feed-forward neural network can only generate “words” or fixed sets of outputs).

Another example is working with time-series data, which we will explore in this tutorial.

A Recurrent Neural Network can be illustrated as follows.

Examples of Recurrent Neural Network includes also.

• Google translate
• Voice recognition
• Video copy right violation

Step 2: Is RNN too complex to understand?

Recurrent Neural Network (RNN) is complex – but luckily – it is not needed to understand in depth.

“You don’t need to understand everything about the specific architecutre of an LSTM cell […] just that LSTM cell is meant to allow past information to be reinjected at a later time.“

Quote of the author of Keras (Francios Chollet)

Let’s just leave it that and get started.

Step 3: RNN predicting stock price

For the purpose of this tutorial we will use Apple stock price and try to make a RNN to predict stock stock price the day after.

For that we will use this file of historic Apple stock prices here. You do not need to download it, we will use it directly in the code.

import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Dropout
import matplotlib.pyplot as plt

file_url = 'https://raw.githubusercontent.com/LearnPythonWithRune/MachineLearningWithPython/main/files/aapl.csv'
data = pd.read_csv(file_url, parse_dates=True, index_col=0)

# Create a train and test set
data_train = data.loc['2000':'2019', 'Adj Close'].to_numpy()
data_test = data.loc['2020', 'Adj Close'].to_numpy()

# Use the MinMaxScaler to scale the data
scaler = MinMaxScaler()
data_train = scaler.fit_transform(data_train.reshape(-1, 1))
data_test = scaler.transform(data_test.reshape(-1, 1))

# To divide data into x and y set
def data_preparation(data):
    x = []
    y = []
    
    for i in range(40, len(data)):
        x.append(data[i-40:i, 0])
        y.append(data[i])
        
    x = np.array(x)
    y = np.array(y)
    
    x = x.reshape(x.shape[0], x.shape[1], 1)
    
    return x, y

x_train, y_train = data_preparation(data_train)
x_test, y_test = data_preparation(data_test)

# Create the model
model = Sequential()
model.add(LSTM(units=45, return_sequences=True, input_shape=(x_train.shape[1], 1)))
model.add(LSTM(units=45, return_sequences=True))
model.add(LSTM(units=45))
model.add(Dense(units=1))

# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])

# Train the model
model.fit(x_train, y_train, epochs=5, batch_size=32)

# Predict with the model
y_pred = model.predict(x_test)

# Unscale it
y_unscaled = scaler.inverse_transform(y_pred)

# See the prediction accuracy
fig, ax = plt.subplots()
y_real = data.loc['2020', 'Adj Close'].to_numpy()
ax.plot(y_real[40:])
ax.plot(y_unscaled)
alt.show()

Resulting in.

This looks more like a moving average of the price and does not to a particular good job.

I am not surprised, as predicting stock prices is not anything easy. If you could do it with a simple model like this, then you would become rich really fast.

Want to learn more?

In the next lesson you will learn How to use Natural Language Processing for Trigrams.

This is part of a FREE 10h Machine Learning course with Python.

• 15 video lessons – which explain Machine Learning concepts, demonstrate models on real data, introduce projects and show a solution (YouTube playlist).
• 30 JuPyter Notebooks – with the full code and explanation from the lectures and projects (GitHub).
• 15 projects – with step guides to help you structure your solutions and solution explained in the end of video lessons (GitHub).