在训练 pytorch 期间,MSE 损失以非常小的数量减少

社会演员多 pytorch 473

原文标题MSE Loss is reducing with a very small amount during training pytorch

我正在尝试我的第一个使用理想比率掩码进行语音分离的深度学习程序。它是一个 BLSTM 模型。训练时获得的 mse 损失正在以非常小的数量减少。关于它为什么会发生以及如何修复它的任何想法??谢谢我的学习率是 0.0003,亚当优化器。Train Epoch: 1 Loss: 0.027980val Epoch: 1 Loss: 0.025925Train Epoch: 2 Loss: 0.027975val Epoch: 2 Loss: 0.025961Train Epoch: 3 Loss: 0.027973val Epoch: 3 Loss : 0.025987

这是我的模型

import torch.nn as nn
import torch

print("Is the GPU available:", torch.cuda.is_available())  # True
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

class Base_model(nn.Module):
    def __init__(self):
        super(Base_model, self).__init__()
        self.lstm = nn.LSTM(input_size=322, hidden_size=300, num_layers=4, bidirectional = True, batch_first=True)
        self.rel = nn.ReLU()
        self.fc = nn.Linear(in_features=600, out_features=161)
        self.activation = nn.Sigmoid()
        
        
    def forward(self, x):
        # print(x.size(0))

        out, _ = self.lstm(x)
        out = self.fc(out)
        out = self.activation(out)
        return out 

我的训练代码

start_epoch = 0
    if args.model_name:
        print("Load the model:", args.checkpoints_dir + args.model_name)
        checkpoint = torch.load(args.checkpoints_dir + args.model_name)
        model.load_state_dict(checkpoint["model"])
        optimizer.load_state_dict(checkpoint["optimizer"])
        start_epoch = checkpoint['epoch']
        # lr_schedule.load_state_dict(checkpoint['lr_schedule'])  # load lr_scheduler

    for epoch in range(start_epoch, args.epochs):
        model.train()  # train model
        for batch_idx, (train_X, train_mask, train_nearend_mic_magnitude, train_nearend_magnitude) in enumerate(
                train_loader):
            train_X = torch.reshape(train_X, (train_X.shape[0],train_X.shape[2],train_X.shape[1]))
            train_mask = torch.reshape(train_mask, (train_mask.shape[0],train_mask.shape[2],train_mask.shape[1]))
            train_nearend_mic_magnitude = torch.reshape(train_nearend_mic_magnitude, (train_nearend_mic_magnitude.shape[0],train_nearend_mic_magnitude.shape[2],train_nearend_mic_magnitude.shape[1]))
            train_nearend_magnitude = torch.reshape(train_nearend_magnitude, (train_nearend_magnitude.shape[0],train_nearend_magnitude.shape[2],train_nearend_magnitude.shape[1]))
            
            train_X = train_X.to(device) 
            train_mask = train_mask.to(device)  # IRM [batch_size 161, 999]
            train_nearend_mic_magnitude = train_nearend_mic_magnitude.to(device)
            train_nearend_magnitude = train_nearend_magnitude.to(device)

            # forward propagation
            optimizer.zero_grad()  # zero out the gradient
            pred_mask = model(train_X)  # [batch_size, 322, 999]--> [batch_size, 161, 999]
            train_loss = criterion(pred_mask, train_mask)
            
            # backpropagation
            train_loss.backward()  # backpropagation
            optimizer.step()  # update parameters

            # ###########    Visual printing   ############
        print('Train Epoch: {} Loss: {:.6f} '.format(epoch + 1, train_loss.item()))



原文链接:https://stackoverflow.com//questions/71553939/mse-loss-is-reducing-with-a-very-small-amount-during-training-pytorch

回复

我来回复