What is batch_size?
来源:互联网 发布:sql数据库存储过程 编辑:程序博客网 时间:2024/05/22 09:03
深度学习中经常看到epoch、 iteration和batchsize,下面按自己的理解说说这三个的区别:
(1)batchsize:批大小。在深度学习中,一般采用SGD训练,即每次训练在训练集中取batchsize个样本训练;
(2)iteration:1个iteration等于使用batchsize个样本训练一次;
(3)epoch:1个epoch等于使用训练集中的全部样本训练一次;
举个例子,训练集有1000个样本,batchsize=10,那么:
训练完整个样本集需要:100次iteration,1次epoch。
Batch size defines number of samples that going to be propagated through the network.
For instance, let's say you have 1050 training samples and you want to set up
batch_size
equal to 100. Algorithm takes first 100 samples (from 1st to 100th) from the training dataset and trains network. Next it takes second 100 samples (from 101st to 200th) and train network again. We can keep doing this procedure until we will propagate through the networks all samples. The problem usually happens with the last set of samples. In our example we've used 1050 which is not divisible by 100 without remainder. The simplest solution is just to get final 50 samples and train the network.Advantages:
It requires less memory. Since you train network using less number of samples the overall training procedure requires less memory. It's especially important in case if you are not able to fit dataset in memory.
Typically networks trains faster with mini-batches. That's because we update weights after each propagation. In our example we've propagated 11 batches (10 of them had 100 samples and 1 had 50 samples) and after each of them we've updated network's parameters. If we used all samples during propagation we would make only 1 update for the network's parameter.
Disadvantages:
- The smaller the batch the less accurate estimate of the gradient. In the figure below you can see that mini-batch (green color) gradient's direction fluctuates compare to the full batch (blue color).
Stochastic is just a mini-batch with batch_size
equal to 1. Gradient changes its direction even more often than a mini-batch.
- What is batch_size?
- What is what ?!
- what is what?
- What is portal? & What is portlet?
- what is game? what is good game?
- What Is 'SSL'? What Is 'SSH'?
- WHAT IS C#
- What is System Administration?
- What is EDI?
- What Is Wiki
- What is Struts Resume?
- what is blog??
- What is AspectJ
- what is grid ?
- What is MSN Messenger
- What is Plone?
- What is Mono?™
- What is AspectJ
- 将model转化为字典
- 二叉排序树创建、查找、删除
- ZSH使用
- Java中关于“unreachble code”的报错
- 事务和锁(共享锁,排他锁,乐观,悲观,死锁)
- What is batch_size?
- linux下载ncurses库去使用make menuconfig
- 前台HTML页面显示XML内容
- WIFI和蓝牙无线模块的应用小结
- HTML课堂讲义(7)
- IOS
- HDU1196 Lowest Bit
- nefuoj1109-状态压缩dp-游戏争霸赛
- redis Key