Get Started with Recurrent Neural Network (RNN) with Tensorflow

What will we cover?

  • Understand Recurrent Neural Network (RNN)
  • Build a RNN on a timeseries
  • Hover over the theory of RNN (LSTM cells)
  • Use the MinMaxScaler from sklearn.
  • Create a RNN model with tensorflow
  • Applying the Dropout techniques.
  • Predict stock prices and make weather forecast using RNN.

Step 1: Feed-forward vs Recurrent Neural Network

Neural Network that has connection only in one direction is called Feed-Forward Neural Network (Examples: Artificial Neural Network, Deep Neural Network, and Convolutional Neural Network).

A Recurrent Neural Network is a Neural Network that generates output that feeds back into its own inputs. This enables it to do one-to-many and many-to-many relationship (not possible for feed-forward neural networks).

An example of one-to-many is a network that can generate sentences (while feed-forward neural network can only generate “words” or fixed sets of outputs).

Another example is working with time-series data, which we will explore in this tutorial.

A Recurrent Neural Network can be illustrated as follows.

Examples of Recurrent Neural Network includes also.

  • Google translate
  • Voice recognition
  • Video copy right violation

Step 2: Is RNN too complex to understand?

Recurrent Neural Network (RNN) is complex – but luckily – it is not needed to understand in depth.

You don’t need to understand everything about the specific architecutre of an LSTM cell […] just that LSTM cell is meant to allow past information to be reinjected at a later time.

Quote of the author of Keras (Francios Chollet)

Let’s just leave it that and get started.

Step 3: RNN predicting stock price

For the purpose of this tutorial we will use Apple stock price and try to make a RNN to predict stock stock price the day after.

For that we will use this file of historic Apple stock prices here. You do not need to download it, we will use it directly in the code.

import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Dropout
import matplotlib.pyplot as plt

file_url = 'https://raw.githubusercontent.com/LearnPythonWithRune/MachineLearningWithPython/main/files/aapl.csv'
data = pd.read_csv(file_url, parse_dates=True, index_col=0)

# Create a train and test set
data_train = data.loc['2000':'2019', 'Adj Close'].to_numpy()
data_test = data.loc['2020', 'Adj Close'].to_numpy()

# Use the MinMaxScaler to scale the data
scaler = MinMaxScaler()
data_train = scaler.fit_transform(data_train.reshape(-1, 1))
data_test = scaler.transform(data_test.reshape(-1, 1))

# To divide data into x and y set
def data_preparation(data):
    x = []
    y = []
    
    for i in range(40, len(data)):
        x.append(data[i-40:i, 0])
        y.append(data[i])
        
    x = np.array(x)
    y = np.array(y)
    
    x = x.reshape(x.shape[0], x.shape[1], 1)
    
    return x, y

x_train, y_train = data_preparation(data_train)
x_test, y_test = data_preparation(data_test)

# Create the model
model = Sequential()
model.add(LSTM(units=45, return_sequences=True, input_shape=(x_train.shape[1], 1)))
model.add(LSTM(units=45, return_sequences=True))
model.add(LSTM(units=45))
model.add(Dense(units=1))

# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])

# Train the model
model.fit(x_train, y_train, epochs=5, batch_size=32)

# Predict with the model
y_pred = model.predict(x_test)

# Unscale it
y_unscaled = scaler.inverse_transform(y_pred)

# See the prediction accuracy
fig, ax = plt.subplots()
y_real = data.loc['2020', 'Adj Close'].to_numpy()
ax.plot(y_real[40:])
ax.plot(y_unscaled)
alt.show()

Resulting in.

This looks more like a moving average of the price and does not to a particular good job.

I am not surprised, as predicting stock prices is not anything easy. If you could do it with a simple model like this, then you would become rich really fast.

Want to learn more?

This is part of a FREE 10h Machine Learning course with Python.

  • 15 video lessons – which explain Machine Learning concepts, demonstrate models on real data, introduce projects and show a solution (YouTube playlist).
  • 30 JuPyter Notebooks – with the full code and explanation from the lectures and projects (GitHub).
  • 15 projects – with step guides to help you structure your solutions and solution explained in the end of video lessons (GitHub).

Leave a Reply Cancel reply

Exit mobile version