线性回归和softmax回归的实现

发表于 | 分类于 |
字数统计: 747 | 阅读时长 ≈ 3

背景

今天的主题是 线性回归（回归问题）和SoftMax回归（分类问题）

开始着手准备秋招了
一些基础的代码实现是在面试中经常出现的
麻雀虽小 五脏俱全
基础回归包含了在深度学习训练过程中的
数据准备、网络模型、损失函数、优化器、训练及验证的完整流程
需要着重掌握

线性回归

  1. 引入基础包

    1
    2
    3
    import torch
    from torch import nn
    from torch.utils import data

  2. 设置基础的参数

    1
    2
    3
    4
    5
    6
    true_w = torch.Tensor([1.5, -2.2, 0.8])
    true_b = 2.7
    batch_size = 10
    epochs = 5
    lr = 1e-2
    train_size = 1000

  3. 生成训练数据

    1
    2
    3
    4
    5
    6
    7
    8
    def synthetic_data(w, b, num_examples):
        X = torch.rand((num_examples, len(w)))
        y = torch.matmul(X, w) + b
        y += torch.rand(1)
        return X, y.reshape(-1, 1)
    features, labels = synthetic_data(true_w, true_b, train_size)
    print(features.shape, labels.shape)
    # (torch.Size([1000, 3]), torch.Size([1000, 1]))

  4. 准备数据，选择模型、损失以及优化器

    1
    2
    3
    4
    5
    net = nn.Sequential(nn.Linear(3, 1))
    loss = nn.MSELoss()
    optim = torch.optim.SGD(net.parameters(), lr=lr)
    dataset = data.TensorDataset(feature, labels)
    dataIter = data.DataLoader(dataset, batch_size, shuffle=True)

  5. 开始训练

    1
    2
    3
    4
    5
    6
    7
    8
    for epoch in range(epochs):
        for X, y in dataIter:
            l = loss(net(X), y)
            optim.zero_grad() # 将上一次的梯度清零
            l.backward() # 计算梯度
            optim.step() # 梯度下降
        l = loss(net(features), labels)
        print(f'epoch{epoch}: loss{l: f}')

  6. 查看训练结果（模型参数）

    1
    2
    3
    for param in net.parameters():
        print(param)
    print(true_w, true_b)

softmax回归

用softmax回归来解决分类问题
使用单层的全连接层来解决图片分类问题
和线性回归大同小异 只是需要最后处理一下类别
用softmax + cross_entropy 来作为损失函数进行模型的训练

  1. 引入基础包

    1
    2
    3
    4
    import torch
    import torchvision
    from torchvision import transform
    from torch import nn

  2. 设置基础参数

    1
    2
    3
    batch_size = 256
    lr = 0.1
    num_epochs = 10

  3. 生成训练数据

    1
    2
    3
    4
    5
    6
    7
    def load_data_fashion_mnist(batch_size, resize=None):
        trans = transforms.ToTensor()
        mnist_train = torchvision.datasets.FashionMNIST(root='../data', train=True, transform=trans, download=True) # dataset
        mnist_test = torchvision.datasets.FashionMNIST(root='../data', train=False, transform=trans, download=True)
        return (data.DataLoader(mnist_train, batch_size, shuffle=True),
                data.DataLoader(mnist_test, batch_size, shuffle=True)) #dataIter
    train_iter, test_iter = load_data_fashion_mnist(batch_size)

  4. 设置模型、损失以及优化器

    1
    2
    3
    4
    5
    6
    7
    def init_weights(w):
        if type(w) == nn.Linear:
            nn.init.normal_(w.weight, std=0.01)
    net = nn.Sequential(nn.Linear(nn.Flatten(), nn.Linear(784, 10)))
    net.apply(init_weights)
    loss = nn.CrossEntropyLoss(reduction='none')
    trainer = torch.optim.SGD(net.parameters(), lr=lr)

  5. 开始训练

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38

    def accuracy(y_hat, y): # 计算准确率
        if len(y_hat.shape) > 1 and y_hat.shape[1] > 1:
            y_hat = y_hat.argmax(axis=1)
        cmp = y.hat.type(y.d_type) == y
        return float(cmp.type(y.dtype).sum())

    def train_epoch(net, loss, trainer, train_iter):
        net.train()
        loss_sum = 0
        acc_sum = 0
        y_sum = 0
        for X, y in train_iter:
            y_hat = net(X)
            l = loss(y_hat, y)
            trainer.zero_grad()
            l.mean().backward()
            trainer.step()
            loss_sum += l
            acc_sum += accuracy(y_hat, y)
            y_sum += y.numel()
        return loss_sum / y_sum, accuracy_sum / y_sum

    for epoch in num_epochs:
        _loss, _acc = train_epoch(net, loss, trainer, train_iter)
        print('epoch {}: loss: {:.6f} acc: {:.4f}'.format(epoch, _loss, _acc))
    """
    epoch 0: loss: 0.784160 acc: 0.7510
    epoch 1: loss: 0.569403 acc: 0.8135
    epoch 2: loss: 0.524297 acc: 0.8260
    epoch 3: loss: 0.501037 acc: 0.8328
    epoch 4: loss: 0.484912 acc: 0.8363
    epoch 5: loss: 0.473458 acc: 0.8403
    epoch 6: loss: 0.463434 acc: 0.8437
    epoch 7: loss: 0.457866 acc: 0.8452
    epoch 8: loss: 0.452482 acc: 0.8459
    epoch 9: loss: 0.446812 acc: 0.8478
    """

  6. 进行预测

    1
    2
    3
    4
    5
    6
    7
    8
    9
    for X, y in test_iter:
        break
    trues = y
    preds = net(X).argmax(axis=1)
    cmp = (trues==preds).sum()
    print('acc', len(cmp) / len(trues))
    """
    acc 0.859375
    """