Machine Learning

Get Started with Recurrent Neural Network (RNN) with Tensorflow

What will we cover?

  • Understand Recurrent Neural Network (RNN)
  • Build a RNN on a timeseries
  • Hover over the theory of RNN (LSTM cells)
  • Use the MinMaxScaler from sklearn.
  • Create a RNN model with tensorflow
  • Applying the Dropout techniques.
  • Predict stock prices and make weather forecast using RNN.

Step 1: Feed-forward vs Recurrent Neural Network

Neural Network that has connection only in one direction is called Feed-Forward Neural Network (Examples: Artificial Neural Network, Deep Neural Network, and Convolutional Neural Network).

A Recurrent Neural Network is a Neural Network that generates output that feeds back into its own inputs. This enables it to do one-to-many and many-to-many relationship (not possible for feed-forward neural networks).

An example of one-to-many is a network that can generate sentences (while feed-forward neural network can only generate “words” or fixed sets of outputs).

Another example is working with time-series data, which we will explore in this tutorial.

A Recurrent Neural Network can be illustrated as follows.

Examples of Recurrent Neural Network includes also.

  • Google translate
  • Voice recognition
  • Video copy right violation

Step 2: Is RNN too complex to understand?

Recurrent Neural Network (RNN) is complex – but luckily – it is not needed to understand in depth.

You don’t need to understand everything about the specific architecutre of an LSTM cell […] just that LSTM cell is meant to allow past information to be reinjected at a later time.

Quote of the author of Keras (Francios Chollet)

Let’s just leave it that and get started.

Step 3: RNN predicting stock price

For the purpose of this tutorial we will use Apple stock price and try to make a RNN to predict stock stock price the day after.

For that we will use this file of historic Apple stock prices here. You do not need to download it, we will use it directly in the code.

import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Dropout
import matplotlib.pyplot as plt

file_url = 'https://raw.githubusercontent.com/LearnPythonWithRune/MachineLearningWithPython/main/files/aapl.csv'
data = pd.read_csv(file_url, parse_dates=True, index_col=0)

# Create a train and test set
data_train = data.loc['2000':'2019', 'Adj Close'].to_numpy()
data_test = data.loc['2020', 'Adj Close'].to_numpy()

# Use the MinMaxScaler to scale the data
scaler = MinMaxScaler()
data_train = scaler.fit_transform(data_train.reshape(-1, 1))
data_test = scaler.transform(data_test.reshape(-1, 1))

# To divide data into x and y set
def data_preparation(data):
    x = []
    y = []
    
    for i in range(40, len(data)):
        x.append(data[i-40:i, 0])
        y.append(data[i])
        
    x = np.array(x)
    y = np.array(y)
    
    x = x.reshape(x.shape[0], x.shape[1], 1)
    
    return x, y

x_train, y_train = data_preparation(data_train)
x_test, y_test = data_preparation(data_test)

# Create the model
model = Sequential()
model.add(LSTM(units=45, return_sequences=True, input_shape=(x_train.shape[1], 1)))
model.add(LSTM(units=45, return_sequences=True))
model.add(LSTM(units=45))
model.add(Dense(units=1))

# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])

# Train the model
model.fit(x_train, y_train, epochs=5, batch_size=32)

# Predict with the model
y_pred = model.predict(x_test)

# Unscale it
y_unscaled = scaler.inverse_transform(y_pred)

# See the prediction accuracy
fig, ax = plt.subplots()
y_real = data.loc['2020', 'Adj Close'].to_numpy()
ax.plot(y_real[40:])
ax.plot(y_unscaled)
alt.show()

Resulting in.

This looks more like a moving average of the price and does not to a particular good job.

I am not surprised, as predicting stock prices is not anything easy. If you could do it with a simple model like this, then you would become rich really fast.

Want to learn more?

This is part of a FREE 10h Machine Learning course with Python.

  • 15 video lessons – which explain Machine Learning concepts, demonstrate models on real data, introduce projects and show a solution (YouTube playlist).
  • 30 JuPyter Notebooks – with the full code and explanation from the lectures and projects (GitHub).
  • 15 projects – with step guides to help you structure your solutions and solution explained in the end of video lessons (GitHub).
Rune

Share
Published by
Rune

Recent Posts

Learn Python FREE Online

Why learn Python? There are many reasons to learn Python, and that is the power…

3 days ago

How to Check if a Number is Even or Odd with Python

What will you learn? How to use the modulo operator to check if a number…

1 week ago

The Truth About Being a Python Software Contractor

There are a lot of Myths out there There are lot of Myths about being…

2 months ago

Do This and 10X Your Salary as a Software Engineer

To be honest, I am not really a great programmer - that is not what…

2 months ago

Ultimate Guide to the Data Science Career Path

What does it take to become a Data Scientist? Data Science is in a cross…

2 months ago

How to Setup a MySQL Server in Docker for Your Python Project

What will you learn? Need to setup a SQL server? You don’t need to install…

4 months ago