First AI – Not Just Another Article

torch – This imports the core library of PyTorch.

torch.nn – This submodule of PyTorch provides tools for building and training neural networks. It contains various pre-defined layers, loss functions, and other components needed for constructing neural network architectures.

torch.optim – This submodule contains various optimization algorithms that can be used to update the weights of the neural network during training.

torchvision – This is a package provided by PyTorch that offers dataset load and preprocessing of the MNIST dataset.

torchvision.transforms – This submodule provides a set of data transformations that can be applied to images. It’s used to compose the transformations applied to the MNIST images before feeding them into the neural network.

transform=transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.5,),(0.5,))]),This code defines a sequence of transformations to be applied to the input images.

transforms.ToTensor() – Converts the images from the dataset into PyTorch tensors.

transforms.Normalize((0.5,), (0.5,)) – Normalize the pixel values of the images. The first tuple specifies the mean to subtract from each channel, and the second tuple specifies the standard deviation to divide by. This normalization process is often done to make the data have a mean of 0 and a standard deviation of 1.

full_train_dataset=torchvision.datasets.MNIST(root=’./data’,train=True, transform=transform, download=True), This code loads the MNIST dataset with the specified settings.

root=’./data’ – Specifies the directory where the dataset will be stored.

train=True – This indicates that you’re loading the training split of the dataset.

transform=transform – Applies the transformations you defined earlier to the loaded images.

download=True – Specifies that if the dataset isn’t already present in the specified directory, it should be downloaded from the internet.

The above lines of code prepare the MNIST dataset for training by applying the defined transformations and making it compatible with PyTorch’s data handling. The dataset will be normalized and converted to tensors, and these preprocessed samples will be used to train our neural network.

Next, we have to filter the MNIST dataset to get only the digit classes 1 to 5:

Here list called selected_indices now only contain images of number 0 to 5. Now we should shuffle selected_indices randomly. In PyTorch we use torch.utils.data.sampler.SubsetRandomSampler() to shuffle our data. Then we can use the shaffled data to create a data loader for training. In PyTorch, we do that by using torch.utils.data.DataLoader(dataset=full_train_dataset, batch_size=batch_size, sampler=train_sampler)

Here we created a custom class called Net, which is a subclass of nn.Module(PyTorch’s base class for all neural network modules).

def __init__(self) Under this constructor method, we define the layers and components of your neural network.

super(Net, self).__init__() – This calls the constructor of the parent class(nn.Module) to properly initialize our custom neural network class.

“It is important to note that when building a neural network structure in PyTorch, Instead of defining individual layers of neurons, we define the connections between neuron layers. In simple terms, we don’t categorize them as input, hidden, or output layers; instead, we consider them as the links or nodes connecting one another.”

For example, in our neural network with four layers (input layer, hidden layer 1, hidden layer 2, and output layer), we have three significant connections between these layers. These connections are from the output layer to hidden layer 1, from hidden layer 1 to hidden layer 2, and from hidden layer 2 to the output layer. It’s these connections that we primarily define when designing our neural network in PyTorch.

self.fc1 = nn.Linear(784, 128) – This defines the neuron’s connection between the input layer and the first hidden layer where 784 neurons(in the input layer) are fully connected to 128 neurons(the first hidden layer).

self.fc2 = nn.Linear(128, 64) – This defines the neuron’s connection between the first hidden layer and the second hidden layer where 128 neurons(in the first hidden layer) are fully connected to 64 neurons(the second hidden layer).

self.fc3 = nn.Linear(64, 5) – This defines the neuron’s connection between the second hidden layer and the output layer where 64 neurons(in the second hidden layer) are fully connected to 5 neurons(the output layer).

torch.nn – automatically creates relevant weight and biases tensors for neurons in those connections and assigns suitable random values to them.

x = x.view(-1, 784) – This line reshapes the input data x to a flat tensor of shape (batch_size,784), effectively flattening the 2D image into a 1D vector.

x = torch.relu(self.fc1(x)) -This applies a ReLU activation function to the output of the first neuron’s connection(the input layer and the first hidden layer)

x = torch.relu(self.fc2(x)) – Similarly, this applies ReLU activation to the output of the second neuron connection(the first hidden layer and the second hidden layer)

x = self.fc3(x)) – This generates the final output logits for the third neuron’s connection(the second layer and the output layer) connected layer.

return x – This returns the final output

Now we can initialize the defined model:

Next, we should implement our loss function, learning rate, and Optimization method. here we use the cross-entropy loss function and Adam adaptive optimization method.

total_step = len(train_loader) – This calculates the total number of batches in our training data loader. It’s used to track progress during training.

for epoch in range(num_epochs): – This is the outer loop that iterates through each epoch during training. The number of epochs is determined by the value of `num_epochs’

for i, (images, labels) in enumerate(train_loader): – This is the inner loop that iterates through each batch of the training data loader. `i` is the batch index, and `(images, labels)` represents the batch of images and their corresponding labels.

labels = labels – 1 – This subtracts 1 from each label to make the labels 0-indexed. In our dataset, we’ve subtracted 1 from labels to represent classes 1 to 5 as indices 0 to 4.

outputs = model(images) – This passes the batch of images through your neural network model to get the raw output logits for each class.

probabilities = softmax_activation(outputs) – This applies the softmax activation function to the logits to obtain the class probabilities.

loss = cross_entropy_loss(probabilities, labels) – This computes the cross-entropy loss between the predicted probabilities and the ground truth labels.

optimizer.zero_grad() – This zeros out the gradients of the model’s parameters before calculating new gradients in the backward pass.

loss.backward() – This computes gradients of the loss with respect to the model’s parameters using backpropagation.

optimizer.step() – This updates the model’s parameters using the computed gradients and the optimization algorithm (in this case, Adam).

(i+1) % 100 == 0: – This checks if the current batch index is a multiple of 100. If true, it prints the progress and loss for that batch. (This gives the loss value of one training data after every 100 training data)

print(“Training finished”) – After the training loop completes, this message is printed to indicate that the training process has finished.

In summary, this code iterates through the training data, feeds it through the model, calculates the loss, and updates the model’s parameters using backpropagation and optimization. The training loop repeats for the specified number of epochs(here 10 times), and the progress and loss are printed periodically.

This line of code saves the learned parameters of our trained neural network model to a file named ‘digit_recognition_model_with_softmax_cross_entropy.pth’. This saved state_dict can later be loaded into another instance of the same model architecture to replicate the trained model’s parameters without the need for retraining.

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
# Hyperparameters
batch_size = 64
learning_rate = 0.001
num_epochs = 10
# Load and preprocess MNIST dataset
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])
full_train_dataset = torchvision.datasets.MNIST(root='./data', train=True, transform=transform, download=True)
# Filter the dataset to include only digits 1 to 5
selected_indices = []
for idx, (image, label) in enumerate(full_train_dataset):
    if label in [1, 2, 3, 4, 5]:  # Include only labels 1 to 5 (digits 1 to 5)
        selected_indices.append(idx)
train_sampler = torch.utils.data.sampler.SubsetRandomSampler(selected_indices)
train_loader = torch.utils.data.DataLoader(dataset=full_train_dataset, batch_size=batch_size, sampler=train_sampler)
# Define the neural network
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(784, 128)
        self.fc2 = nn.Linear(128, 64)
        self.fc3 = nn.Linear(64, 5)
    def forward(self, x):
        x = x.view(-1, 784)  # Flatten the input
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x
# Instantiate the model
model = Net()
# Loss and optimizer
softmax_activation = nn.Softmax(dim=1)
cross_entropy_loss = nn.CrossEntropyLoss()  # Cross-Entropy loss
optimizer = optim.Adam(model.parameters(), lr=learning_rate)
# Training loop
total_step = len(train_loader)
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        labels = labels - 1  # Subtract 1 to make labels 0-indexed
        outputs = model(images)
        probabilities = softmax_activation(outputs)
        loss = cross_entropy_loss(probabilities, labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        if (i+1) % 100 == 0:
            print(f'Epoch [{epoch+1}/{num_epochs}], Step [{i+1}/{total_step}], Loss: {loss.item():.4f}')
print("Training finished")
# Save the model
torch.save(model.state_dict(), 'digit_recognition_model_with_softmax_cross_entropy.pth')

In the following code, I have selected an image of the number 4 from the MNIST dataset. I will pass this image data through our model, and then display the predicted class label alongside the selected image using the matplotlib library.

First of all, we should import the necessary libraries.

Next, we should load our saved neural network architecture and train neural network parameters for it.

model.load_state_dict(torch.load(‘digit_recognition_model_with_softmax_cross_entropy.pth’)) – This will load the trained parameter values from its saved location.(in our case, ‘digit_recognition_model_with_softmax_cross_entropy.pth’)

model.eval() – This sets the model to evaluation mode. When a model is in evaluation mode, it affects the behavior of certain layers in the network, such as dropout and batch normalization layers. The primary purpose of using model.eval() is to ensure consistent behavior between the training and evaluation phases. During training, dropout layers introduce randomness by randomly ‘dropping out’ units during forward propagation, which helps prevent overfitting. Likewise, batch normalization layers compute statistics based on the batch being processed.

Now we can load our data sample.

In the code above, similar to the training process, we load the entire MNIST dataset. Then, we filter and save only the images of the number 4 using the following code: [idx for idx, (_, label) in enumerate(mnist_dataset) if label == 4], and store them in the digit_4_indices variable. If you want images of another number, such as number 2, you can simply change if label == 4 to if label == 2. This modification will provide you with only the images of the number 2.

To obtain a single data sample, we select one image of the number 4 by indexing digits_4_indices, which contains images of the number 4. In this case, we are using the first image. You can select a different image of the number 4 by changing the index number.

Next, we can pass the selected data sample through our trained model like this:

The output_probabilities variable contains the model’s predicted probabilities for each class corresponding to the input image. In simpler terms, it provides probabilities for each class, indicating how likely the image belongs to each class. The class with the highest probability value is considered the model’s predicted class for that image.

We can display the model-predicted class with the selected image of the number 4 like this:

Our model’s output will be something like this:

“You can see that our model correctly predicted that image as number 4.”