# pytorch 深度学习之自动微分

import torch

x = torch.arange(4.0)
x

tensor([0., 1., 2., 3.])


[En]

A place is needed to store gradients. It is important that we do not allocate new memory each time we derive a parameter. Because we often update the same parameters thousands of times, allocating new memory each time can quickly run out of memory. Note that the gradient of a scalar function with respect to the vector (x) is a vector and has the same shape as (x).

x.requires_grad_(True)  # 等价于x=torch.arange(4.0,requires_grad=True)

y = 2 * torch.dot(x,x)
y

tensor(28., grad_fn=<mulbackward0>)
</mulbackward0>


(x) 是一个长度为 4 的向量，计算 (x) 和 (x) 的点积，得到了我们赋值给y的标量输出。 接下来，我们通过调用反向传播函数来自动计算 (y) 关于 (x) 每个分量的梯度，并打印这些梯度：

y.backward()

tensor([ 0.,  4.,  8., 12.])


x.grad == 4 * x

tensor([True, True, True, True])

tensor([1., 1., 1., 1.])


# 对非标量调用backward需要传入一个gradient参数，该参数指定微分函数关于self的梯度。
# 在我们的例子中，我们只想求偏导数的和，所以传递一个1的梯度是合适的
y = x * x
# 等价于y.backward(torch.ones(len(x)))
y.sum().backward()

tensor([0., 2., 4., 6.])


x.grad.zero_()
y = x * x
u = y.detach();
z = u * x

z.sum().backward()

tensor([True, True, True, True])


x.grad.zero_()
y.sum().backward()

tensor([True, True, True, True])


def f(a):
b = a * 2
while b.norm() < 1000:
b *= 2
if b.sum() > 0:
c = b
else:
c = 100 * b

return c

d = f(a)
d.backward()


tensor(True)


Original: https://www.cnblogs.com/xiaojianliu/p/16160210.html
Author: 6小贱
Title: pytorch 深度学习之自动微分

(0)