What will we cover?
- Understand Recurrent Neural Network (RNN)
- Build a RNN on a timeseries
- Hover over the theory of RNN (LSTM cells)
- Use the MinMaxScaler from sklearn.
- Create a RNN model with tensorflow
- Applying the Dropout techniques.
- Predict stock prices and make weather forecast using RNN.
Step 1: Feed-forward vs Recurrent Neural Network
A Recurrent Neural Network is a Neural Network that generates output that feeds back into its own inputs. This enables it to do one-to-many and many-to-many relationship (not possible for feed-forward neural networks).
An example of one-to-many is a network that can generate sentences (while feed-forward neural network can only generate “words” or fixed sets of outputs).
Another example is working with time-series data, which we will explore in this tutorial.
A Recurrent Neural Network can be illustrated as follows.
Examples of Recurrent Neural Network includes also.
- Google translate
- Voice recognition
- Video copy right violation
Step 2: Is RNN too complex to understand?
Recurrent Neural Network (RNN) is complex – but luckily – it is not needed to understand in depth.
“You don’t need to understand everything about the specific architecutre of an LSTM cell […] just that LSTM cell is meant to allow past information to be reinjected at a later time.“Quote of the author of Keras (Francios Chollet)
Let’s just leave it that and get started.
Step 3: RNN predicting stock price
For the purpose of this tutorial we will use Apple stock price and try to make a RNN to predict stock stock price the day after.
For that we will use this file of historic Apple stock prices here. You do not need to download it, we will use it directly in the code.
import numpy as np import pandas as pd from sklearn.preprocessing import MinMaxScaler from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.layers import LSTM from tensorflow.keras.layers import Dropout import matplotlib.pyplot as plt file_url = 'https://raw.githubusercontent.com/LearnPythonWithRune/MachineLearningWithPython/main/files/aapl.csv' data = pd.read_csv(file_url, parse_dates=True, index_col=0) # Create a train and test set data_train = data.loc['2000':'2019', 'Adj Close'].to_numpy() data_test = data.loc['2020', 'Adj Close'].to_numpy() # Use the MinMaxScaler to scale the data scaler = MinMaxScaler() data_train = scaler.fit_transform(data_train.reshape(-1, 1)) data_test = scaler.transform(data_test.reshape(-1, 1)) # To divide data into x and y set def data_preparation(data): x =  y =  for i in range(40, len(data)): x.append(data[i-40:i, 0]) y.append(data[i]) x = np.array(x) y = np.array(y) x = x.reshape(x.shape, x.shape, 1) return x, y x_train, y_train = data_preparation(data_train) x_test, y_test = data_preparation(data_test) # Create the model model = Sequential() model.add(LSTM(units=45, return_sequences=True, input_shape=(x_train.shape, 1))) model.add(LSTM(units=45, return_sequences=True)) model.add(LSTM(units=45)) model.add(Dense(units=1)) # Compile the model model.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy']) # Train the model model.fit(x_train, y_train, epochs=5, batch_size=32) # Predict with the model y_pred = model.predict(x_test) # Unscale it y_unscaled = scaler.inverse_transform(y_pred) # See the prediction accuracy fig, ax = plt.subplots() y_real = data.loc['2020', 'Adj Close'].to_numpy() ax.plot(y_real[40:]) ax.plot(y_unscaled) alt.show()
This looks more like a moving average of the price and does not to a particular good job.
I am not surprised, as predicting stock prices is not anything easy. If you could do it with a simple model like this, then you would become rich really fast.
Want to learn more?
This is part of a FREE 10h Machine Learning course with Python.
- 15 video lessons – which explain Machine Learning concepts, demonstrate models on real data, introduce projects and show a solution (YouTube playlist).
- 30 JuPyter Notebooks – with the full code and explanation from the lectures and projects (GitHub).
- 15 projects – with step guides to help you structure your solutions and solution explained in the end of video lessons (GitHub).