TensorFlow 构建 simple model

1. model构建 basic流程

in TensorFlowin构建 and 训练model通常遵循以 under basic流程:

1. data准备: 收集, 清洗 and 预processingdata, 将data划分 for 训练集, verification集 and test集.
2. model定义: 定义model structure, including输入层, 隐藏层 and 输出层.
3. 损失function选择: 根据taskclass型选择合适 损失function, such as均方误差 (回归task) or 交叉熵 (classificationtask) .
4. optimization器configuration: 选择optimizationalgorithms and Learning率, such as随机梯度 under 降 (SGD) , Adametc..
5. model训练: using训练data训练model, through反向传播updatemodelparameter.
6. modelassessment: usingverification集assessmentmodelperformance, 调整超parameter.
7. model预测: using训练 good model for new datafor预测.

2. 线性回归model

线性回归 is 最 simple 机器Learningmodel之一, 用于预测连续值. 它fake设输入特征 and 输出之间存 in 线性relationships, 可以用公式 y = w dot x + b 表示, 其in w is 权重, b is 偏置.

2.1 准备data

首先, 我们生成一些mockdata用于训练线性回归model:

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

# 设置随机种子, 确保结果可重现
tf.random.set_seed(42)
np.random.seed(42)

# 生成100个样本
X = 2 * np.random.rand(100, 1)
# 生成真实tag, 添加一些噪声
y = 4 + 3 * X + np.random.randn(100, 1)

# 绘制data点
plt.scatter(X, y, color='b', alpha=0.7)
plt.xlabel('X')
plt.ylabel('y')
plt.title('mockdata')
plt.show()

2.2 usingtf.keras构建线性回归model

TensorFlow 2.x推荐usingKeras API构建model. Keras is a advanced神经networkAPI, providing了 simple 易用 interface.

# usingSequential API构建model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(1,))  # 输出层, 1个神经元
])

# 查看model摘要
model.summary()

model摘要会显示model 层次structure, parameter数量etc.information:

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 1)                 2         
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________

2.3 编译model

in 训练model之 before , 我们需要编译model, 指定损失function and optimization器:

# 编译model
model.compile(
    optimizer='sgd',  # 随机梯度 under 降optimization器
    loss='mse'  # 均方误差损失function
)

2.4 训练model

usingfit()method训练model:

# 训练model
history = model.fit(
    X, y,  # 训练data
    epochs=100,  # 训练轮数
    batch_size=10,  # 批次 big  small 
    verbose=1  # 显示训练进度
)

2.5 visualization训练过程

我们可以visualization训练过程in 损失变化:

# 绘制损失曲线
plt.plot(history.history['loss'])
plt.xlabel('Epoch')
plt.ylabel('Loss (MSE)')
plt.title('model训练损失')
plt.show()

2.6 model预测

using训练 good modelfor预测:

# 生成testdata
X_test = np.array([[0], [2]])

# 预测
y_pred = model.predict(X_test)

print("testdata预测结果: ")
print(f"当X=0时, 预测y={y_pred[0][0]:.4f}")
print(f"当X=2时, 预测y={y_pred[1][0]:.4f}")

# 绘制model拟合曲线
plt.scatter(X, y, color='b', alpha=0.7, label='真实data')
plt.plot(X, model.predict(X), color='r', label='model预测')
plt.xlabel('X')
plt.ylabel('y')
plt.title('线性回归model拟合结果')
plt.legend()
plt.show()

2.7 获取modelparameter

我们可以查看训练 good modelparameter:

# 获取modelparameter
w, b = model.layers[0].get_weights()
print(f"权重 w = {w[0][0]:.4f}")
print(f"偏置 b = {b[0]:.4f}")

理想circumstances under , 我们expectation得 to w ≈ 3 and b ≈ 4, 这 and 我们生成data时using 真实parameter一致.

3. 逻辑回归model

逻辑回归 is a用于二classificationtask model, 它将线性回归 输出throughsigmoidfunction转换 for 0 to 1之间 概率值, 表示样本属于正class 概率.

3.1 准备二classificationdata

# 生成二classificationdata
from sklearn.datasets import make_classification

# 生成100个样本, 2个特征, 2个class别
X, y = make_classification(
    n_samples=100,  # 样本数量
    n_features=2,  # 特征数量
    n_informative=2,  #  has 效特征数量
    n_redundant=0,  # 冗余特征数量
    n_classes=2,  # class别数量
    random_state=42
)

# 将y转换 for 列向量
y = y.reshape(-1, 1)

# 绘制二classificationdata
plt.scatter(X[y[:, 0]==0, 0], X[y[:, 0]==0, 1], color='r', label='class别 0')
plt.scatter(X[y[:, 0]==1, 0], X[y[:, 0]==1, 1], color='b', label='class别 1')
plt.xlabel('特征 1')
plt.ylabel('特征 2')
plt.title('二classificationdata')
plt.legend()
plt.show()

3.2 构建逻辑回归model

# 构建逻辑回归model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(2,), activation='sigmoid')  # 输出层, usingsigmoid激活function
])

# 查看model摘要
model.summary()

3.3 编译model

for 于classificationtask, 我们通常using交叉熵serving as损失function:

# 编译model
model.compile(
    optimizer='adam',  # Adamoptimization器
    loss='binary_crossentropy',  # 二classification交叉熵损失function
    metrics=['accuracy']  # assessment指标: 准确率
)

3.4 训练model

# 训练model
history = model.fit(
    X, y,  # 训练data
    epochs=100,  # 训练轮数
    batch_size=10,  # 批次 big  small 
    verbose=1  # 显示训练进度
)

3.5 visualization训练过程

# 绘制损失曲线 and 准确率曲线
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# 绘制损失曲线
ax1.plot(history.history['loss'])
ax1.set_xlabel('Epoch')
ax1.set_ylabel('Loss (Binary Crossentropy)')
ax1.set_title('model训练损失')

# 绘制准确率曲线
ax2.plot(history.history['accuracy'])
ax2.set_xlabel('Epoch')
ax2.set_ylabel('Accuracy')
ax2.set_title('model训练准确率')

plt.tight_layout()
plt.show()

3.6 modelassessment

我们可以usingevaluate()methodassessmentmodel in 训练data on performance:

# assessmentmodel
loss, accuracy = model.evaluate(X, y, verbose=0)
print(f"model损失: {loss:.4f}")
print(f"model准确率: {accuracy:.4f}")

3.7 visualization决策edge界

# 绘制决策edge界
def plot_decision_boundary(model, X, y):
    # creation网格
    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.01),
                         np.arange(y_min, y_max, 0.01))
    
    # 预测网格点
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    
    # 绘制决策edge界
    plt.contourf(xx, yy, Z, alpha=0.3)
    plt.scatter(X[y[:, 0]==0, 0], X[y[:, 0]==0, 1], color='r', label='class别 0')
    plt.scatter(X[y[:, 0]==1, 0], X[y[:, 0]==1, 1], color='b', label='class别 1')
    plt.xlabel('特征 1')
    plt.ylabel('特征 2')
    plt.title('逻辑回归决策edge界')
    plt.legend()
    plt.show()

# 绘制决策edge界
plot_decision_boundary(model, X, y)

4. usingfunction式API构建model

除了Sequential API, Keras还providing了function式API, 允许我们构建更 complex model, such as many 输入, many 输出model and 具 has 共享层 model.

4.1 构建线性回归model

# usingfunction式API构建线性回归model
inputs = tf.keras.Input(shape=(1,))  # 输入层
outputs = tf.keras.layers.Dense(1)(inputs)  # 输出层

model = tf.keras.Model(inputs=inputs, outputs=outputs)

# 查看model摘要
model.summary()

4.2 构建具 has 隐藏层 model

usingfunction式API可以easily构建具 has many 个隐藏层 model:

# 构建具 has 隐藏层 model
inputs = tf.keras.Input(shape=(2,))  # 输入层
x = tf.keras.layers.Dense(10, activation='relu')(inputs)  # 隐藏层, 10个神经元, ReLU激活function
outputs = tf.keras.layers.Dense(1, activation='sigmoid')(x)  # 输出层

model = tf.keras.Model(inputs=inputs, outputs=outputs)

# 查看model摘要
model.summary()

5. using子classAPI构建model

Keras还providing了子classAPI, 允许我们throughinheritancetf.keras.Modelclass来构建model, providing了最 big flexible性.

5.1 构建线性回归model

# using子classAPI构建线性回归model
class LinearRegressionModel(tf.keras.Model):
    def __init__(self):
        super(LinearRegressionModel, self).__init__()
        self.dense = tf.keras.layers.Dense(1)  # 输出层
    
    def call(self, inputs):
        return self.dense(inputs)

# creationmodelinstance
model = LinearRegressionModel()

# 编译model
model.compile(
    optimizer='sgd',
    loss='mse'
)

# 训练model
history = model.fit(X, y, epochs=100, batch_size=10, verbose=1)

6. 保存 and 加载model

我们可以保存训练 good model, 以便 after 续using:

6.1 保存完整model

# 保存完整model
model.save('linear_regression_model.h5')

6.2 加载model

# 加载model
loaded_model = tf.keras.models.load_model('linear_regression_model.h5')

# using加载 modelfor预测
y_pred = loaded_model.predict(X_test)
print(y_pred)

6.3 只保存model权重

我们也可以只保存model权重:

# 保存model权重
model.save_weights('model_weights.h5')

# creation new modelinstance
new_model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(1,))
])

# 加载model权重
new_model.load_weights('model_weights.h5')

# 编译model
new_model.compile(optimizer='sgd', loss='mse')

# using new modelfor预测
y_pred = new_model.predict(X_test)
print(y_pred)

7. 超parameter调整

超parameter is 指 in 训练开始 before 设置 parameter, 而不 is through训练Learning得 to parameter. common 超parameterincluding:

• Learning率
• 批次 big small
• 训练轮数
• 隐藏层数量
• 每个隐藏层 神经元数量
• 激活function
• optimization器

7.1 调整Learning率

Learning率 is 最 important 超parameter之一, 它决定了modelparameterupdate 步 long . 我们可以尝试不同 Learning率:

# 尝试不同 Learning率
learning_rates = [0.001, 0.01, 0.1]

for lr in learning_rates:
    # 构建model
    model = tf.keras.Sequential([
        tf.keras.layers.Dense(1, input_shape=(1,))
    ])
    
    # 编译model
    model.compile(
        optimizer=tf.keras.optimizers.SGD(learning_rate=lr),
        loss='mse'
    )
    
    # 训练model
    history = model.fit(X, y, epochs=100, batch_size=10, verbose=0)
    
    # 绘制损失曲线
    plt.plot(history.history['loss'], label=f'lr={lr}')

plt.xlabel('Epoch')
plt.ylabel('Loss (MSE)')
plt.title('不同Learning率 under  损失曲线')
plt.legend()
plt.show()

7.2 usingLearning率scheduling器

Learning率scheduling器允许我们 in 训练过程in动态调整Learning率:

# usingLearning率scheduling器
lr_scheduler = tf.keras.callbacks.LearningRateScheduler(
    lambda epoch: 0.1 * (0.1 ** (epoch // 30))  # 每30个epochLearning率乘以0.1
)

# 构建model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(1,))
])

# 编译model
model.compile(
    optimizer='sgd',
    loss='mse'
)

# 训练model
history = model.fit(
    X, y,
    epochs=100,
    batch_size=10,
    callbacks=[lr_scheduler],  # 添加Learning率scheduling器
    verbose=1
)

8. 练习