Deep Learning with PyTorch: A 60 Minute Blitz 要点整理

常用工具类

torch.Tensor

创建

• Construct a 5x3 matrix, uninitialized

x = torch.Tensor(5, 3)

• Construct a randomly initialized matrix

x = torch.rand(5, 3)

CUDA Tensors

Tensors can be moved onto GPU using the .cuda function.

if torch.cuda.is_available():    x = x.cuda()    y = y.cuda()    x + y

Operation

实现两个tensor的加法：x+y
1.

x + y

2.

torch.add(x, y)

3.giving an output tensor

result = torch.Tensor(5, 3)torch.add(x, y, out=result)

4.in-place

y.add_(x)

torch.autograd.Variable

• .backward()
• .data
• .grad: autograd.Variable
• .creator: autograd.Function. Each variable has a .creator attribute that references a Function that has created the Variable (except for Variables created by the user - their creator is None).

Numpy Bridge

The torch Tensor and numpy array will share their underlying memory locations, and changing one will change the other.

Converting torch Tensor to numpy Array

a = torch.ones(5)b = a.numpy()

Converting numpy Array to torch Tensor

a = np.ones(5)b = torch.from_numpy(a)

搭建神经网络的一般代码架构

import torchfrom torch.autograd import Variableimport torch.nn as nnimport torch.nn.functional as Fimport torch.optim as optim# 1.搭建网络class Net(nn.Module):    def __init__(self):        super(Net, self).__init__()        # 1 input image channel, 6 output channels, 5x5 square convolution        # kernel        self.conv1 = nn.Conv2d(1, 6, 5)        self.conv2 = nn.Conv2d(6, 16, 5)        # an affine operation: y = Wx + b        self.fc1 = nn.Linear(16 * 5 * 5, 120)        self.fc2 = nn.Linear(120, 84)        self.fc3 = nn.Linear(84, 10)    def forward(self, x):        # Max pooling over a (2, 2) window        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))        # If the size is a square you can only specify a single number        x = F.max_pool2d(F.relu(self.conv2(x)), 2)        x = x.view(-1, self.num_flat_features(x))        x = F.relu(self.fc1(x))        x = F.relu(self.fc2(x))        x = self.fc3(x)        return x    def num_flat_features(self, x):        size = x.size()[1:]  # all dimensions except the batch dimension        num_features = 1        for s in size:            num_features *= s        return num_featuresnet = Net()# 2.input & outputinput = Variable(torch.randn(1, 1, 32, 32))output = net(input)     # 执行了nn.Module里的__call__()# 3.loss & bptarget = Variable(torch.arange(1, 11))  # a dummy target, for examplecriterion = nn.MSELoss()loss = criterion(output, target)    # 执行了nn.Module里的__call__()loss.backward()         # 从loss(scalar)开始做反向传播# 4.create optimizeroptimizer = optim.SGD(net.parameters(), lr=0.01)# 5.每个step(loop)optimizer.zero_grad()   # zero the gradient buffers.                        # zero_grad中对每个参数执行param.grad.data.zero_()optimizer.step()    # Does the update