pytorch
是一个动态的建图的工具。不像Tensorflow
那样,先建图,然后通过feed
和run
重复执行建好的图。相对来说,pytorch
具有更好的灵活性。
编写一个深度网络需要关注的地方是:
1. 网络的参数应该由什么对象保存
2. 如何构建网络
3. 如何计算梯度和更新参数
数据放在什么对象中
pytorch
中有两种变量类型,一个是Tensor
,一个是Variable
。
Tensor
: 就像ndarray
一样,一维Tensor
叫Vector
,二维Tensor
叫Matrix
,三维及以上称为Tensor
Variable
:是Tensor
的一个wrapper
,不仅保存了值,而且保存了这个值的creator
,需要BP
的网络都是Variable
参与运算
import torchx = torch.Tensor(2,3,4) x
(0 ,.,.) = 1.00000e-37 * 1.5926 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000(1 ,.,.) = 1.00000e-37 * 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000[torch.FloatTensor of size 2x3x4]
x.size()
torch.Size([2, 3, 4])
a = torch.rand(2,3,4)b = torch.rand(2,3,4)_=torch.add(a,b, out=x) x
(0 ,.,.) = 0.9815 0.0833 0.8217 1.1280 0.7810 1.2586 1.0243 0.7924 1.0200 1.0463 1.4997 1.0994(1 ,.,.) = 0.8031 1.4283 0.6245 0.9617 1.3551 1.9094 0.9046 0.5543 1.2838 1.7381 0.6934 0.8727[torch.FloatTensor of size 2x3x4]
a.add_(b) torch.cuda.is_available()
True
自动求导
pytorch
的自动求导工具包在torch.autograd
中
from torch.autograd import Variablex = torch.rand(5)x = Variable(x,requires_grad = True)y = x * 2grads = torch.FloatTensor([1,2,3,4,5])y.backward(grads)x.grad
Variable containing: 2 4 6 8 10[torch.FloatTensor of size 5]
neural networks
使用torch.nn
包中的工具来构建神经网络
构建一个神经网络需要以下几步:
- 定义神经网络的
权重
,搭建网络结构 - 遍历整个数据集进行训练
- 将数据输入神经网络
- 计算loss
- 计算网络权重的梯度
- 更新网络权重
- weight = weight + learning_rate * gradient
import torch.nn as nnimport torch.nn.functional as Fclass Net(nn.Module): def __init__(self): super(Net, self).__init__() self.conv1 = nn.Conv2d(1, 6, 5) self.conv2 = nn.Conv2d(6, 16, 5) self.fc1 = nn.Linear(16*5*5, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84, 10) def forward(self, x): x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) x = F.max_pool2d(F.relu(self.conv2(x)), 2) x = x.view(-1, self.num_flat_features(x)) x = F.relu(self.fc1(x)) x = F.relu(self.fc2(x)) x = self.fc3(x) return x def num_flat_features(self, x): size = x.size()[1:] num_features = 1 for s in size: num_features *= s return num_featuresnet = Net()net
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
Net ( (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1)) (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1)) (fc1): Linear (400 -> 120) (fc2): Linear (120 -> 84) (fc3): Linear (84 -> 10))
len(list(net.parameters()))
10
input = Variable(torch.randn(1, 1, 32, 32))out = net(input) out out.backward(torch.randn(1, 10))
使用loss criterion 和 optimizer训练网络
torch.nn
包下有很多loss标准。同时torch.optimizer
帮助完成更新权重的工作。这样就不需要手动更新参数了
learning_rate = 0.01for f in net.parameters(): f.data.sub_(f.grad.data * learning_rate)
import torch.optim as optimoptimizer = optim.SGD(net.parameters(), lr = 0.01)optimizer.zero_grad() output = net(input) loss = criterion(output, target)loss.backward()optimizer.step()
整体NN结构
import torch.nn as nnimport torch.nn.functional as Fclass Net(nn.Module): def __init__(self): super(Net, self).__init__() self.conv1 = nn.Conv2d(1, 6, 5) self.conv2 = nn.Conv2d(6, 16, 5) self.fc1 = nn.Linear(16*5*5, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84, 10) def forward(self, x): x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) x = F.max_pool2d(F.relu(self.conv2(x)), 2) x = x.view(-1, self.num_flat_features(x)) x = F.relu(self.fc1(x)) x = F.relu(self.fc2(x)) x = self.fc3(x) return x def num_flat_features(self, x): size = x.size()[1:] num_features = 1 for s in size: num_features *= s return num_featuresnet = Net()optimizer = optim.SGD(net.parameters(), lr = 0.01)for i in range(num_iteations): optimizer.zero_grad() output = net(input) loss = criterion(output, target) loss.backward() optimizer.step()
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
其它
- 关于求梯度,只有我们定义的Variable才会被求梯度,由
creator
创造的不会去求梯度 - 自己定义Variable的时候,记得Variable(Tensor, requires_grad = True),这样才会被求梯度,不然的话,是不会求梯度的
import numpy as npa = np.ones(5)b = torch.from_numpy(a)np.add(a, 1, out=a)print(a) print(b)
a = np.ones(5)b = torch.from_numpy(a)a_ = b.numpy() np.add(a, 1, out=a)
if torch.cuda.is_available(): x = x.cuda() y = y.cuda() x + y
torch.Tensor(1,2,3) torch.Tensor([1,2,3])
import torchfrom torch.autograd import Variableimport numpy as npn1 = np.array([1., 2.]).astype(np.float32)t1 = torch.from_numpy(n1)n1[0] = 2.print(t1)
如遇无法下载pytorch安装包问题
[链接:http://pan.baidu.com/s/1c2cSoX6 密码:ckf8]