Torch中多GPU运行代码学习

Main.lua  

 

-- Copyright (c) 2014, Facebook, Inc.

 

-- All rights reserved.

 

--

 

-- This source code is licensed under the BSD-style license found in the

 

-- LICENSE file in the root directory of this source tree. An additional grant

 

-- of patent rights can be found in the PATENTS file in the same directory.

 

--

 

require 'torch'

 

require 'cutorch'

 

require 'paths'

 

require 'xlua'

 

require 'optim'

 

require 'nn'

 

 

 

torch.setdefaulttensortype('torch.FloatTensor')

 

 

 

local opts= paths.dofile('opts.lua')

 

 

 

opt = opts.parse(arg)

 

 

 

nClasses = opt.nClasses

 

 

 

paths.dofile('util.lua')

 

paths.dofile('model.lua')

 

opt.imageSize = model.imageSizeor opt.imageSize

 

opt.imageCrop = model.imageCropor opt.imageCrop

 

 

 

print(opt)

 

 

 

cutorch.setDevice(opt.GPU)-- by default, use GPU 1

 

torch.manualSeed(opt.manualSeed)

 

 

 

print('Saving everything to: ' .. opt.save)

 

os.execute('mkdir -p ' .. opt.save)

 

 

 

paths.dofile('data.lua')

 

paths.dofile('train.lua')

 

paths.dofile('test.lua')

 

 

 

epoch = opt.epochNumber

 

 

 

for i=1,opt.nEpochsdo

 

train()

 

test()

 

epoch = epoch+ 1

 

end

 

The training scripts come with several options which can be listed by running the script with the flag --help

th main.lua --help

To run the training, simply run main.lua By default, the script runs 1-GPU AlexNet with the CuDNN backend and 2 data-loader threads.

th main.lua -data [imagenet-folder with train and val folders]

For 2-GPU model parallel AlexNet + CuDNN, you can run it this way:

th main.lua -data [imagenet-folder with train and val folders] -nGPU 2 -backend cudnn -netType alexnet

Similarly, you can switch the backends to 'cunn' to use a different set of CUDA kernels.

You can also alternatively train OverFeat using this following command:

th main.lua -data [imagenet-folder with train and val folders] -netType overfeat

# multi-GPU overfeat (let's say 2-GPU)

th main.lua -data [imagenet-folder with train and val folders] -netType overfeat -nGPU 2