Introduction

My primary focus is reinforcement learning, but some reinforcement learning techniques involve deep learning (deep reinforcement learning). I don’t want to focus on deep learning too much, and this is why I’m “speedrunning” the basics of deep learning and PyTorch.

Preparation

Satisfy the requirements for PyTorch

For NVIDIA users, setup CUDA using the following CUDA Toolkit:

CUDA Toolkit 11.8 Downloads

Neural Network Fundamentals

Perceptron

A perceptron is essentially just a function that does the following things:

  1. multiply each input with numbers called weights
  2. sum them up
  3. put them through a non-linear activation function, which is just like a filter
    1. common types of non-linear activation functions are sigmoid functions and rectified linear unit (ReLU)

Perceptron visualized from MIT Introduction to Deep Learning | 6.S191

Perceptron visualized from MIT Introduction to Deep Learning | 6.S191

Perceptron visualized (simplified) from MIT Introduction to Deep Learning | 6.S191

Perceptron visualized (simplified) from MIT Introduction to Deep Learning | 6.S191

Neural Network

A neural network is just perceptrons stacked up.

A simple one hidden layer neural network from MIT Introduction to Deep Learning | 6.S191

A simple one hidden layer neural network from MIT Introduction to Deep Learning | 6.S191

If we look at each node in the hidden layer, we can see that they are going through the same process (the perceptron steps), but with different sets of weights.