Deep Neural Network: A Guide to Deep Learning

Why it’s great to master Deep Neural Networks (DNN)?

Mastering Deep Neural Networks (DNNs) provides several benefits in the field of machine learning and artificial intelligence:

1. Powerful modeling capabilities: DNNs have demonstrated remarkable performance in complex tasks such as image recognition, natural language processing, and speech recognition. By understanding and mastering DNNs, you can leverage their powerful modeling capabilities to tackle a wide range of challenging problems and achieve state-of-the-art results.
2. Representation learning: DNNs are capable of automatically learning hierarchical representations from raw data, which enables them to extract meaningful and high-level features. This ability to learn useful representations from raw inputs is crucial in various domains where feature engineering is difficult or time-consuming.
3. Versatility and adaptability: DNNs can be applied to diverse domains and tasks, making them a versatile tool for solving a wide range of problems. Moreover, they can be adapted and fine-tuned for specific tasks through transfer learning, enabling faster development and improved performance with limited labeled data.

What will be covered in this tutorial?

In this tutorial on Deep Neural Networks (DNNs), we will cover the following topics:

• Understanding DNNs: Exploring the concepts and principles underlying Deep Neural Networks, including their architecture, activation functions, and backpropagation algorithm. This foundational knowledge will provide you with a solid understanding of how DNNs work and their potential for solving complex problems.
• Weight calculation in DNNs: Delving into the algorithms used to calculate the weights in Deep Neural Networks, such as gradient descent and its variants (e.g., stochastic gradient descent). Understanding these weight optimization techniques is crucial for training DNNs effectively and improving their performance.
• Visualizing DNN capabilities: Introducing tools and techniques to visually understand what Deep Neural Networks can solve. This includes visualizing learned representations, feature maps, and activation patterns to gain insights into how DNNs interpret and process data. These visualization methods enhance your understanding of the inner workings of DNNs and provide valuable insights for model analysis and improvement.

By mastering Deep Neural Networks and completing this tutorial, you will gain the knowledge and skills to effectively apply DNNs to solve complex problems, understand their training process, and visually analyze their behavior. This expertise will enable you to leverage the power of DNNs in various domains and contribute to cutting-edge advancements in machine learning and artificial intelligence.

Step 1: What is Deep Neural Network?

Be sure to read the Artificial Neural Network Guide.

The adjective “deep” in deep learning refers to the use of multiple layers in the network (Wiki).

Usually having two or more hidden layers counts as deep.

Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning.

Step 2: How to train and difficulties in training DNN

Training an Artificial Neural Network only relies on finding weights from input to output nodes. In a Deep Neural Network (DNN) become a bit more complex and requires more techniques.

To do that we need backpropagation, which is an algorithm for training Neural Networks with hidden layers (DNN).

• Algorithm
  • Start with a random choice of weights
  • Repeat
    • Calculate error for output layer
    • For each layer – starting with output layer
      • Propagate error back one layer
      • Update weights

A problem you will encounter is overfitting. Which means to fit too close to training data and not generalize well.

That is, you fit the model to the training data, but the model will not predict well on data not coming from your training data.

To deal with that, dropout is a common technique.

• Temporarily remove units – selectat random – from a neural network to prevent over reliance on certain units
• Dropout value of 20%-50%
• Better performance when dropout is used on a larger network
• Dropout at each layer of the network has shown good results.
• Original Paper

Step 3: Play around with it

To learn more about fitting check out the playground at tensorflow.

• https://playground.tensorflow.org/

Ideas to check that

• If you have no hidden layers then you can only fit with straight lines.
• If you add hidden layers you can model the XOR function.

Step 4: A DNN model of XOR

Let’s go crazy and fit an XOR dataset with a DNN model.

import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow.keras.layers import Dense, Dropout
from tensorflow.keras.models import Sequential
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
data = pd.read_csv('https://raw.githubusercontent.com/LearnPythonWithRune/MachineLearningWithPython/main/files/xor.csv')
fig, ax = plt.subplots()
ax.scatter(x=data['x'], y=data['y'], c=data['class id'])
plt.show()

This is the data we want to fit.

Then let’s create it.

Remember to insert the dropout and play around with it.

X_train, X_test, y_train, y_test = train_test_split(data[['x', 'y']], data['class id'], random_state=42)
accuracies = []
for i in range(5):
    tf.random.set_seed(i)
    
    model = Sequential()
    model.add(Dense(6, input_dim=2, activation='relu'))
    # model.add(Dropout(.2))
    model.add(Dense(4, activation='relu'))
    # model.add(Dropout(.2))
    model.add(Dense(1, activation='sigmoid'))
   
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    
    model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=0)
    _, accuracy = model.evaluate(X_test, y_test)
    accuracies.append(accuracy*100)
    
sum(accuracies)/len(accuracies)

Resulting in accuracy of 98%.

Want to learn more?

In the next lesson you learn Convolutional Neural Network to Detect Handwriting.

This is part of a FREE 10h Machine Learning course with Python.

• 15 video lessons – which explain Machine Learning concepts, demonstrate models on real data, introduce projects and show a solution (YouTube playlist).
• 30 JuPyter Notebooks – with the full code and explanation from the lectures and projects (GitHub).
• 15 projects – with step guides to help you structure your solutions and solution explained in the end of video lessons (GitHub).

Python Circle

Do you know what the 5 key success factors every programmer must have?

How is it possible that some people become programmer so fast?

While others struggle for years and still fail.

Not only do they learn python 10 times faster they solve complex problems with ease.

What separates them from the rest?

I identified these 5 success factors that every programmer must have to succeed:

1. Collaboration: sharing your work with others and receiving help with any questions or challenges you may have.
2. Networking: the ability to connect with the right people and leverage their knowledge, experience, and resources.
3. Support: receive feedback on your work and ask questions without feeling intimidated or judged.
4. Accountability: stay motivated and accountable to your learning goals by surrounding yourself with others who are also committed to learning Python.
5. Feedback from the instructor: receiving feedback and support from an instructor with years of experience in the field.

I know how important these success factors are for growth and progress in mastering Python.

That is why I want to make them available to anyone struggling to learn or who just wants to improve faster.

With the Python Circle community, you can take advantage of 5 key success factors every programmer must have.

Be part of something bigger and join the Python Circle community.