RuntimeError: element 0 of tensors does not require grad and does not have a grad_

今天在跑代码的过程中,因为要训练一个模型然后在测试阶段使用PGD来生成相应的adv_image来测试这个模型,结果运行到测试阶段出现下面的问题。

报错如下:
RuntimeError: element 0 of tensors does not require grad and does not have a grad_

我的代码如下:

def validate_roubst(val_loader, model, criterion, epoch, args, log=None, tf_writer=None, flag='roubst_val'):
    batch_time = AverageMeter('Time', ':6.3f')
    losses = AverageMeter('Loss', ':.4e')
    top1 = AverageMeter('Acc@1', ':6.2f')
    top5 = AverageMeter('Acc@5', ':6.2f')

    model.eval()
    all_preds = []
    all_targets = []

    with torch.no_grad():
        end = time.time()
        for i, (input, target) in enumerate(val_loader):
            if args.gpu is not None:

                print('............')

            input = input.cuda(args.gpu, non_blocking=True)
            target = target.cuda(args.gpu, non_blocking=True)

            attack_method = PGD(model, args.device)
            adv_example = attack_method.generate(input, target, epsilon = 8/255, num_steps = 20, step_size = 0.01, clip_max = 1.0, clip_min = 0.0, print_process = False, bound = 'linf')

            output = model(adv_example)
            loss = criterion(output, target)

            acc1, acc5 = accuracy(output, target, topk=(1, 5))
            losses.update(loss.item(), input.size(0))
            top1.update(acc1[0], input.size(0))
            top5.update(acc5[0], input.size(0))

            batch_time.update(time.time() - end)
            end = time.time()

            _, pred = torch.max(output, 1)
            all_preds.extend(pred.cpu().numpy())
            all_targets.extend(target.cpu().numpy())

            if i % args.print_freq == 0:
                output = ('Test: [{0}/{1}]\t'
                            'Time {batch_time.val:.3f} ({batch_time.avg:.3f})\t'
                            'Loss {loss.val:.4f} ({loss.avg:.4f})\t'
                            'Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
                            'Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
                    i, len(val_loader), batch_time=batch_time, loss=losses,
                    top1=top1, top5=top5))
                print(output)
        cf = confusion_matrix(all_targets, all_preds).astype(float)
        cls_cnt = cf.sum(axis=1)
        cls_hit = np.diag(cf)
        cls_acc = cls_hit / cls_cnt
        output = ('{flag} Results: Prec@1 {top1.avg:.3f} Prec@5 {top5.avg:.3f} Loss {loss.avg:.5f}'
                .format(flag=flag, top1=top1, top5=top5, loss=losses))
        out_cls_acc = '%s Class Accuracy: %s'%(flag,(np.array2string(cls_acc, separator=',', formatter={'float_kind':lambda x: "%.3f" % x})))
        print(output)
        print(out_cls_acc)
        if log is not None:
            log.write(output + '\n')
            log.write(out_cls_acc + '\n')
            log.flush()

        tf_writer.add_scalar('loss/test_'+ flag, losses.avg, epoch)
        tf_writer.add_scalar('acc/test_' + flag + '_top1', top1.avg, epoch)
        tf_writer.add_scalar('acc/test_' + flag + '_top5', top5.avg, epoch)
        tf_writer.add_scalars('acc/test_' + flag + '_cls_acc', {str(i):x for i, x in enumerate(cls_acc)}, epoch)

     return top1.avg

如果出了问题,当然你必须找到解决方案:

[En]

If something goes wrong, of course you have to find a solution:

2.1 方案1

大多数人都想加上这句话:

[En]

Most people mean to add this sentence:

loss.requires_grad_(True) #加入此句就行了

具体做法就是:

loss = criterion(output, target)
loss.requires_grad_(True) # 加入此句在这个位置

loss.backward()

但是经过本人尝试,还是没有什么用,因为我在train阶段不会 出现错误,只有在test阶段就报错。

2.2 方案2

回到本质,或者从错误报告的角度来看,错误提示大致意味着元素不需要渐变。

[En]

To return to the essence, or from the point of view of error reporting, the error hint roughly means that the element does not require a gradient.

然后我仔细瞅了瞅我那段代码,发现了一个可疑之处: with torch.no_grad()
最后,仔细看看这个东西的一些使用规则。1):

[En]

Finally, take a closer look at some rules for the use of this thing (Ref. 1):

with torch.no_grad()则主要是 用于停止autograd模块的工作,以起到加速和节省显存的作用,具体行为就是停止gradient计算,从而节省了GPU算力和显存,但是并不会影响dropout和batchnorm层的行为。

看到我上面加粗的字体了吧,原来使用with torch.no_grad()就不会自动求梯度了,因为我们使用PGD生成adv_image需要求梯度,所以加上with torch.no_grad()就导致了我们无法求梯度,最终出现了下面的错误。

故解决方案为:

将 with torch.no_grad() 去掉

Original: https://blog.csdn.net/wyf2017/article/details/123156380
Author: 流年若逝
Title: RuntimeError: element 0 of tensors does not require grad and does not have a grad_

原创文章受到原创版权保护。转载请注明出处:https://www.johngo689.com/496084/

转载文章受原作者版权保护。转载请注明原作者出处!

(0)

大家都在看

亲爱的 Coder【最近整理,可免费获取】👉 最新必读书单  | 👏 面试题下载  | 🌎 免费的AI知识星球