Multiclass task

Multiclass task#

This page describes how to build a multiclass classification network using pytorch.

import pandas as pd

from sklearn import datasets

import torch
from torch import nn, optim
from torch.utils.data import TensorDataset, DataLoader

Create sample#

Generate some data with a certain number of features and classes.

n_features = 5
n_classes = 3

X, Y = datasets.make_classification(
    n_features = 5, 
    n_classes = n_classes,
    n_clusters_per_class = 1,
    n_informative = 2,
    n_redundant = 0,
    n_samples = 10000,
    random_state = 10
)

X, Y = torch.tensor(X, dtype = torch.float), torch.tensor(Y)

Model#

In the next cell, we need to define a model where the number of inputs corresponds to the number of features of the input data and the number of outputs corresponds to the number of classes for the sample generated below.

network = nn.Sequential(
    nn.Linear(in_features = n_features, out_features = 30),
    nn.ReLU(),
    nn.Linear(in_features = 30, out_features = n_classes)
)

The output of the model for each object will be the number of floats corresponding to the number of classes in the input data. The index of the maximum value we will understand as the predicted class.

The following cell show how a whole complex of transformations can be performed in code:

output = network(X)

show_df = pd.DataFrame(
    output.detach().numpy(),
    columns = [f"out {i}" for i in range(n_classes)]
)
show_df["predicted class"] = output.max(axis = 1).indices
show_df["real_class"] = Y

show_df.head(5)
out 0 out 1 out 2 predicted class real_class
0 -0.131736 0.335459 -0.036521 1 1
1 0.123889 -0.031864 -0.228509 0 2
2 -0.137551 0.272115 -0.049151 1 1
3 -0.376087 0.127549 -0.022875 1 2
4 -0.129013 0.456219 -0.003062 1 1

And, of course, we can use operations from the previous cell to calculate a fraction of correct predictions - accuracy:

accyracy = (output.max(axis = 1).indices == Y).numpy().mean()
print(f"accuracy - {accyracy}")

accuracy - 0.3848

Optimization#

Now we can do classic burner model optimisation pipline and print accuracy after each epoch - we get better and better results.

data_loder = DataLoader(
    TensorDataset(X, Y), batch_size = 100
)

optimizer = optim.Adam(network.parameters(), lr = 1e-2)
loss_fn = nn.CrossEntropyLoss()

for epoch in range(10):
    for x, y in data_loder:
        optimizer.zero_grad()
        output = network(x)
        loss_val = loss_fn(output, y)
        loss_val.backward()
        optimizer.step()
    predict = network(X).max(axis = 1).indices
    print(f"epoch - {epoch}, accuracy - {((predict == Y)).numpy().mean()}")

epoch - 0, accuracy - 0.9231
epoch - 1, accuracy - 0.924
epoch - 2, accuracy - 0.9243
epoch - 3, accuracy - 0.9242
epoch - 4, accuracy - 0.9243
epoch - 5, accuracy - 0.9245
epoch - 6, accuracy - 0.9239
epoch - 7, accuracy - 0.9244
epoch - 8, accuracy - 0.9252
epoch - 9, accuracy - 0.9254
Contents