PyTorch modelassessment and 保存

Learningsuch as何assessmentmodelperformance, 以及model 保存 and 加载method, 确保model 可重现性 and 可deployment性

PyTorch modelassessment and 保存

1. modelassessment

modelassessment is 深度Learning流程in important 环节, 它可以helping我们Understandmodel performance, 判断model is 否过拟合 or 欠拟合, 并 for modelimprovementproviding依据.

1.1 assessment模式

in PyTorchin, model has 两种模式: 训练模式 (train) and assessment模式 (eval) . assessment模式会关闭Dropout and Batch Normalizationetc. in 训练 and assessment阶段behavior不同 层. 

# 切换 to assessment模式
model.eval()

# 切换 to 训练模式
# model.train()

1.2 常用assessment指标

不同 task需要using不同 assessment指标, 以 under is 一些常用 assessment指标:

1.2.1 classificationtask
  • 准确率 (Accuracy) : 正确classification 样本数占总样本数 比例
  • 精确率 (Precision) : 预测 for 正class 样本inpractical for 正class 比例
  • 召回率 (Recall) : practical for 正class 样本in被预测 for 正class 比例
  • F1分数: 精确率 and 召回率 调 and 平均值
  • 混淆矩阵 (Confusion Matrix) : 展示classification结果 矩阵
  • Roc曲线 and AUC分数: 用于assessment二classificationmodel performance
1.2.2 回归task
  • 均方误差 (MSE) : 预测值 and 真实值之差 平方 平均值
  • 平均绝 for 误差 (MAE) : 预测值 and 真实值之差 绝 for 值 平均值
  • R²分数: 决定系数, 衡量model for data方差 解释程度
  • 平均绝 for 百分比误差 (MAPE) : 预测值 and 真实值之差 绝 for 值 and 真实值 百分比 平均值

1.3 implementationmodelassessment

让我们以classificationtask for 例, implementationmodelassessment: 

def evaluate_model(model, dataloader, criterion, device):
    """assessmentmodel"""
    model.eval()  # 切换 to assessment模式
    run_loss = 0.0
    correct = 0
    total = 0
    
    with torch.no_grad():  # 禁用梯度计算
        for inputs, targets in dataloader:
            inputs, targets = inputs.to(device), targets.to(device)
            
            #  before 向传播
            outputs = model(inputs)
            loss = criterion(outputs, targets)
            
            # statistics损失
            run_loss += loss.item() * inputs.size(0)
            
            # statistics准确率
            _, predicted = torch.max(outputs.data, 1)
            total += targets.size(0)
            correct += (predicted == targets).sum().item()
    
    # 计算平均损失 and 准确率
    avg_loss = run_loss / len(dataloader.dataset)
    accuracy = correct / total
    
    return avg_loss, accuracy

# usingexample
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Net().to(device)
criterion = torch.nn.CrossEntropyLoss()

test_loss, test_accuracy = evaluate_model(model, test_loader, criterion, device)
print(f"test损失: {test_loss:.4f}, test准确率: {test_accuracy:.4f}")

1.4 混淆矩阵

混淆矩阵可以更详细地展示model classification结果: 

from sklearn.metrics import confusion_matrix
import matplotlib.pyplot as plt
import numpy as np

def plot_confusion_matrix(model, dataloader, classes, device):
    """绘制混淆矩阵"""
    model.eval()
    y_true = []
    y_pred = []
    
    with torch.no_grad():
        for inputs, targets in dataloader:
            inputs, targets = inputs.to(device), targets.to(device)
            outputs = model(inputs)
            _, predicted = torch.max(outputs.data, 1)
            
            y_true.extend(targets.cpu().numpy())
            y_pred.extend(predicted.cpu().numpy())
    
    # 计算混淆矩阵
    cm = confusion_matrix(y_true, y_pred)
    
    # 绘制混淆矩阵
    plt.figure(figsize=(10, 8))
    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)
    plt.title('混淆矩阵')
    plt.colorbar()
    
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)
    
    #  in 矩阵in添加数值
    thresh = cm.max() / 2.
    for i, j in np.ndindex(cm.shape):
        plt.text(j, i, format(cm[i, j], 'd'),
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")
    
    plt.tight_layout()
    plt.ylabel('真实class别')
    plt.xlabel('预测class别')
    plt.show()

# usingexample
classes = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']  # MNISTdata集class别
plot_confusion_matrix(model, test_loader, classes, device)

2. model保存

model训练completion after , 我们需要将model保存 to disk, 以便 after 续using. PyTorchproviding了两种主要 model保存方式:

  • 保存完整model: 保存model structure and parameter
  • 仅保存modelparameter: 仅保存model 权重 and 偏置

2.1 仅保存modelparameter

仅保存modelparameter is 最常用 方式, 它只保存model 权重 and 偏置, 不保存model structure. 这种方式更flexible, 允许我们 in 加载model时using不同 modelstructure. 

# 保存modelparameter
torch.save(model.state_dict(), 'model_params.pth')

# 保存最佳model
if val_loss < best_val_loss:
    best_val_loss = val_loss
    torch.save(model.state_dict(), 'best_model_params.pth')

2.2 保存完整model

保存完整model会保存model structure and parameter, 这种方式更 simple , 但flexible性较差. 

# 保存完整model
torch.save(model, 'complete_model.pth')

2.3 保存check点

in 训练过程in, 我们通常会保存check点, package含modelparameter, optimization器status, 训练轮数etc.information, 以便 after 续restore训练. 

# 保存check点
torch.save({
    'epoch': epoch,
    'model_state_dict': model.state_dict(),
    'optimizer_state_dict': optimizer.state_dict(),
    'loss': loss,
    'accuracy': accuracy,
    'learning_rate': scheduler.get_last_lr()
}, 'checkpoint.pth')

3. model加载

model加载 is model保存 逆过程, 根据保存方式 不同, 加载方式也 has 所不同.

3.1 加载modelparameter

加载modelparameter时, 我们需要先creationmodelinstance, 然 after 将保存 parameter加载 to modelin. 

# 加载modelparameter
model = Net()  # creationmodelinstance
model.load_state_dict(torch.load('model_params.pth'))
model.eval()  # 切换 to assessment模式

3.2 加载完整model

加载完整model时, 我们直接usingtorch.loadfunction即可. 

# 加载完整model
model = torch.load('complete_model.pth')
model.eval()  # 切换 to assessment模式

注意

加载完整model时, 需要确保model 定义 in 当 before environmentin可用. such as果model定义发生了变化, 可能会导致加载失败. 因此, 推荐using仅保存modelparameter 方式.

3.3 加载check点

加载check点时, 我们需要先creationmodel and optimization器instance, 然 after 将保存 parameter加载 to 相应 instancein. 

# 加载check点
checkpoint = torch.load('checkpoint.pth')

# creationmodel and optimization器instance
model = Net()
optimizer = torch.optim.Adam(model.parameters())

# 加载modelparameter
model.load_state_dict(checkpoint['model_state_dict'])

# 加载optimization器status
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])

# 获取otherinformation
epoch = checkpoint['epoch']
loss = checkpoint['loss']
accuracy = checkpoint['accuracy']
learning_rate = checkpoint['learning_rate']

# 继续训练
epochs = 100
for i in range(epoch, epochs):
    # 训练code...

3.4 跨设备加载model

in 不同设备 on 加载model时, 我们需要usingmap_locationparameter指定设备. 

#  from GPU加载 to CPU
model = Net()
model.load_state_dict(torch.load('model_params.pth', map_location=torch.device('cpu')))

#  from CPU加载 to GPU
model = Net()
model.load_state_dict(torch.load('model_params.pth', map_location=torch.device('cuda')))
model.to('cuda')

4. model 可重现性

for 了确保model 可重现性, 我们需要固定随机种子. 

import torch
import numpy as np
import random

# 固定随机种子
def set_seed(seed):
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)  # such as果using many 个GPU
    np.random.seed(seed)
    random.seed(seed)
    torch.backends.cudnn.deterministic = True  # 确保卷积operation 确定性
    torch.backends.cudnn.benchmark = False  # 禁用自动寻找最优卷积algorithms

# 设置随机种子
set_seed(42)

5. modelexport

除了保存PyTorchmodel out , 我们还可以将modelexport for other格式, 以便 in 不同 平台 on deployment.

5.1 export for TorchScript

TorchScript is PyTorchmodel 一种in间表示, 可以 in C++etc.environmentinrun. 

# export for TorchScript
model = Net()
model.load_state_dict(torch.load('model_params.pth'))
model.eval()

# method1: 跟踪 (Tracing) 
example = torch.randn(1, 1, 28, 28)  # example输入
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save('model_traced.pt')

# method2: 脚本 (Scripting) 
scripted_module = torch.jit.script(model)
scripted_module.save('model_scripted.pt')

5.2 export for ONNX

ONNX (Open Neural Network Exchange) is a开放 model格式, support不同framework之间 model转换. 

# export for ONNX
model = Net()
model.load_state_dict(torch.load('model_params.pth'))
model.eval()

# example输入
example = torch.randn(1, 1, 28, 28)

# exportmodel
torch.onnx.export(
    model,
    example,
    'model.onnx',
    export_params=True,
    opset_version=11,
    do_constant_folding=True,
    input_names=['input'],
    output_names=['output'],
    dynamic_axes={'input': {0: 'batch_size'}, 'output': {0: 'batch_size'}}  # 动态批次 big  small 
)

实践练习

练习1: modelassessment

usingMNISTdata集训练一个 simple 卷积神经network, 然 after implementationmodelassessment, 计算准确率, 精确率, 召回率 and F1分数, 并绘制混淆矩阵.

练习2: model保存 and 加载

训练一个model, 保存modelparameter and check点, 然 after 尝试加载model并继续训练.

练习3: modelexport

将训练 good modelexport for TorchScript and ONNX格式, 然 after 尝试usingONNX Runtime加载 and runONNXmodel.

6. summarized

本tutorial介绍了PyTorchmodelassessment and 保存 techniques, including:

  • modelassessment 常用指标 and implementationmethod
  • model保存 两种方式: 保存完整model and 仅保存modelparameter
  • model加载 method and Notes
  • modelcheck点 保存 and 加载
  • model 可重现性
  • modelexport for TorchScript and ONNX格式

Master这些techniques可以helping你更 good 地assessmentmodelperformance, 保存 and 加载model, 确保model 可重现性 and 可deployment性. in practicalapplicationin, 我们需要根据具体requirements选择合适 model保存 and 加载方式.

on 一课: PyTorchmodel训练 and optimization 返回tutoriallist under 一课: PyTorch卷积神经network(CNN)