NVIDIA DIGITS 5.1-dev学习笔记之安装过程记录：Windows10 x64位系统 、 MicroSoft Caffe Master、CUDA 8.0 、Python 2.7

今天成功在windows下配置成功了英伟达的DIGITS，记录一下问题解决过程。 
环境简介： Windows10_x64 CUDA 8.0 / CUDA 7.5 Python2.7 Microsoft-Caffe-master 
Github DIGITS: https://github.com/NVIDIA/DIGITS/blob/master/docs/BuildDigitsWindows.md 点击打开链接 
完全按照此步骤操作不会出现问题，尤其是 关于Python package的版本问题，详见DIGITS-master目录下的requirements.txt：

Pillow>=2.3.0,<=3.1.2 
numpy>=1.8.1,<=1.11.0 
scipy>=0.13.3,<=0.17.0 
protobuf>=2.5.0,<=2.6.1 
six>=1.5.2,<=1.10.0 
requests>=2.2.1,<=2.9.1 
gevent>=1.0,<=1.1.0 
gevent-websocket==0.9.3 
Flask==0.10.1 
Flask-WTF>=0.11,<=0.12 
wtforms>=2.0,<=2.1 
Flask-SocketIO==2.6 
setuptools>=3.3,<=20.7.0 
lmdb==0.87 
h5py>=2.2.1,<=2.6.0 
pydot>=1.0.28,<=1.0.29 
psutil>=1.2.1,<=3.4.2 
matplotlib>=1.3.1,<=1.5.1 
scikit-fmm>=0.0.9

版本号一定确保和上述一致。

**

BUG 1 : 与google.protobuf 有关

** 
我记得是在加载from google.protobuf import _symbol 指令时（大概就是这个指令吧），问题是由于我原来安装的时ptotobuf 2.5.0，版本有点低，然后我更新到2.6.1后，就没问题了。ptotobuf在windows下的python支持编译方法请自行google。

BUG 2 : 当我运行时

I:\DIGITS-master>python -m digits  ___ ___ ___ ___ _____ ___ |   \_ _/ __|_ _|_   _/ __| | |) | | (_ || |  | | \__ \ |___/___\___|___| |_| |___/ 5.1-devA valid Caffe installation was not found on your system.Use the envvar CAFFE_ROOT to indicate a valid installation.Traceback (most recent call last):  File "I:\Python27\lib\runpy.py", line 174, in _run_module_as_main    "__main__", fname, loader, pkg_name)  File "I:\Python27\lib\runpy.py", line 72, in _run_code    exec code in run_globals  File "I:\DIGITS-master\digits\__main__.py", line 70, in <module>    main()  File "I:\DIGITS-master\digits\__main__.py", line 53, in main    import digits.config  File "digits\config\__init__.py", line 7, in <module>    from . import (  # noqa  File "digits\config\caffe.py", line 230, in <module>    executable, version, flavor = load_from_path()  File "digits\config\caffe.py", line 58, in load_from_path    version, flavor = get_version_and_flavor(executable)  File "digits\config\caffe.py", line 157, in get_version_and_flavor    version = parse_version(version_string)  File "digits\utils\__init__.py", line 160, in parse_version    return pkg_resources.SetuptoolsVersion(v)  File "I:\Python27\lib\site-packages\pkg_resources\_vendor\packaging\version.py", line 202, in __init__    raise InvalidVersion("Invalid version: '{0}'".format(version))pkg_resources._vendor.packaging.version.InvalidVersion: Invalid version: 'CAFFE_VERSION'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

然后，我看了下caffe.py这个文件。 
caffe.py： 
关键信息我都红色标记了。

from __future__ import absolute_importimport impimport osimport platformimport reimport subprocessimport sysfrom . import option_listfrom digits import device_queryfrom digits.utils import parse_versiondef load_from_envvar(envvar):    """    Load information from an installation indicated by an environment variable    """    value = os.environ[envvar].strip().strip("\"' ")#此处需要修改路径，于CAFFE_HOME对应    if platform.system() == 'Windows':        #executable_dir = os.path.join(value, 'install', 'bin')        executable_dir = os.path.join(value)        #python_dir = os.path.join(value, 'install', 'python')        python_dir = os.path.join(value, 'pycaffe')    else:        executable_dir = os.path.join(value, 'build', 'tools')        python_dir = os.path.join(value, 'python')    try:        executable = find_executable_in_dir(executable_dir)        if executable is None:            raise ValueError('Caffe executable not found at "%s"'                             % executable_dir)        if not is_pycaffe_in_dir(python_dir):            raise ValueError('Pycaffe not found in "%s"'                             % python_dir)        import_pycaffe(python_dir)        version, flavor = get_version_and_flavor(executable)    except:        print ('"%s" from %s does not point to a valid installation of Caffe.'               % (value, envvar))        print 'Use the envvar CAFFE_ROOT to indicate a valid installation.'        raise    return executable, version, flavordef load_from_path():    """    Load information from an installation on standard paths (PATH and PYTHONPATH)    """    try:        executable = find_executable_in_dir()        if executable is None:            raise ValueError('Caffe executable not found in PATH')        if not is_pycaffe_in_dir():            raise ValueError('Pycaffe not found in PYTHONPATH')        import_pycaffe()        version, flavor = get_version_and_flavor(executable)    except:        print 'A valid Caffe installation was not found on your system.'        print 'Use the envvar CAFFE_ROOT to indicate a valid installation.'        raise    return executable, version, flavordef find_executable_in_dir(dirname=None):    """    Returns the path to the caffe executable at dirname    If dirname is None, search all directories in sys.path    Returns None if not found    """    if platform.system() == 'Windows':        exe_name = 'caffe.exe'    else:        exe_name = 'caffe'    if dirname is None:        dirnames = [path.strip("\"' ") for path in os.environ['PATH'].split(os.pathsep)]    else:        dirnames = [dirname]    for dirname in dirnames:        path = os.path.join(dirname, exe_name)        if os.path.isfile(path) and os.access(path, os.X_OK):            return path    return Nonedef is_pycaffe_in_dir(dirname=None):    """    Returns True if you can "import caffe" from dirname    If dirname is None, search all directories in sys.path    """    old_path = sys.path    if dirname is not None:        sys.path = [dirname]  # temporarily replace sys.path    try:        imp.find_module('caffe')    except ImportError:        return False    finally:        sys.path = old_path    return Truedef import_pycaffe(dirname=None):    """    Imports caffe    If dirname is not None, prepend it to sys.path first    """    if dirname is not None:        sys.path.insert(0, dirname)        # Add to PYTHONPATH so that build/tools/caffe is aware of python layers there        os.environ['PYTHONPATH'] = '%s%s%s' % (            dirname, os.pathsep, os.environ.get('PYTHONPATH'))    # Suppress GLOG output for python bindings    GLOG_minloglevel = os.environ.pop('GLOG_minloglevel', None)    # Show only "ERROR" and "FATAL"    os.environ['GLOG_minloglevel'] = '2'    # for Windows environment, loading h5py before caffe solves the issue mentioned in    # https://github.com/NVIDIA/DIGITS/issues/47#issuecomment-206292824    import h5py  # noqa    try:        import caffe    except ImportError:        print 'Did you forget to "make pycaffe"?'        raise    # Strange issue with protocol buffers and pickle - see issue #32    sys.path.insert(0, os.path.join(        os.path.dirname(caffe.__file__), 'proto'))    # Turn GLOG output back on for subprocess calls    if GLOG_minloglevel is None:        del os.environ['GLOG_minloglevel']    else:        os.environ['GLOG_minloglevel'] = GLOG_minlogleveldef get_version_and_flavor(executable):    """    Returns (version, flavor)    Should be called after import_pycaffe()    """    version_string = get_version_from_pycaffe()    if version_string is None:        version_string = get_version_from_cmdline(executable)    if version_string is None:        version_string = get_version_from_soname(executable)    if version_string is None:        raise ValueError('Could not find version information for Caffe build ' +                         'at "%s". Upgrade your installation' % executable)    #这部分代码没用，但是会出现bug，我就注释了    #version = parse_version(version_string)    #if parse_version(0, 99, 0) > version > parse_version(0, 9, 0):    #    flavor = 'NVIDIA'    #    minimum_version = '0.11.0'    #    if version < parse_version(minimum_version):    #        raise ValueError(    #            'Required version "%s" is greater than "%s". Upgrade your installation.'    #            % (minimum_version, version_string))    #else:    #    flavor = 'BVLC'    flavor = 'BVLC'    return version_string, flavordef get_version_from_pycaffe():    try:        from caffe import __version__ as version        return version    except ImportError:        return Nonedef get_version_from_cmdline(executable):    command = [executable, '-version']    p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)    if p.wait():        print p.stderr.read().strip()        raise RuntimeError('"%s" returned error code %s' % (command, p.returncode))    pattern = 'version'    for line in p.stdout:        if pattern in line:            return line[line.find(pattern) + len(pattern) + 1:].strip()    return Nonedef get_version_from_soname(executable):    command = ['ldd', executable]    p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)    if p.wait():        print p.stderr.read().strip()        raise RuntimeError('"%s" returned error code %s' % (command, p.returncode))    # Search output for caffe library    libname = 'libcaffe'    caffe_line = None    for line in p.stdout:        if libname in line:            caffe_line = line            break    if caffe_line is None:        raise ValueError('libcaffe not found in linked libraries for "%s"'                         % executable)    # Read the symlink for libcaffe from ldd output    symlink = caffe_line.split()[2]    filename = os.path.basename(os.path.realpath(symlink))    # parse the version string    match = re.match(r'%s(.*)\.so\.(\S+)$' % (libname), filename)    if match:        return match.group(2)    else:        return None#看这里，看这里，一个路径问题#我们需要在环境变量里声明一下，CAFFE_ROOT 或者 CAFFE_HOME都可以，指向caffe编译后的 ./Build/x64/Releaseif 'CAFFE_ROOT' in os.environ:    executable, version, flavor = load_from_envvar('CAFFE_ROOT')elif 'CAFFE_HOME' in os.environ:    executable, version, flavor = load_from_envvar('CAFFE_HOME')else:    executable, version, flavor = load_from_path()option_list['caffe'] = {    'executable': executable,    'version': version,    'flavor': flavor,    'multi_gpu': (flavor == 'BVLC' or parse_version(version) >= parse_version(0, 12)),    'cuda_enabled': (len(device_query.get_devices()) > 0),}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239

然后就跑起来吧！！！

贴图看效果： 
1. 训练ing

 
2. 测试，可以看特征图，参数啊啊啊（Nvidia DIGITS大法好）

下面是安装步骤！！！

Build DIGITS on Windows

Limitation

DIGITS for Windows depends on Windows branch of BVLC Caffe. The following layers, required for DetectNet feature, are not implemented in that branch.

detectnet_transform_layer 
l1_loss_layer 
As a result, DIGITS for Windows does not support DetectNet. To run DIGITS with DetectNet, please use NV-Caffe 0.15 or above on Ubuntu.

Prerequisites

Python2 
CUDA 7.5 
CuDNN 5.1 
Caffe 
Graphviz

Installing prerequisites

Python2

Download and install Python 2.7.11 64bit from Python’s official site (https://www.python.org/ftp/python/2.7.11/python-2.7.11.amd64.msi). Please select Add Python Path during installation.

Download numpy, scipy, matplotlib, scikit-image, h5py from Unofficial Windows Binaries for Python Extension Packages webpage at (http://www.lfd.uci.edu/~gohlke/pythonlibs/). Remember to download correct version (2.7) and architecture (64-bit).

Additionally, download gevent v1.0.2 at the same site. Run command prompt (cmd.exe) as administrator, and issue the following commands.

python -m pip install cython 
python -m pip install numpy-1.11.0+mkl-cp27-cp27m-win_amd64.whl 
python -m pip install scipy-0.17.0-cp27-none-win_amd64.whl 
python -m pip install matplotlib-1.5.1-cp27-none-win_amd64.whl 
python -m pip install scikit_image-0.12.3-cp27-cp27m-win_amd64.whl 
python -m pip install h5py-2.6.0-cp27-cp27m-win_amd64.whl

If the installation process complains compiler not found, you need to install Microsoft Visual C++ Compiler for Python 2.7, downloaded at (https://www.microsoft.com/en-us/download/details.aspx?id=44266). We recommend installing it by

msiexec /i VCForPython27.msi ALLUSERS=1 
After that compiler is installed, finish the above python -m pip install commands.

At this moment, do not install gevent yet. We need to install it after installing DIGITS.

CUDA 7.5

CUDA 7.5 can be obtained at NVIDIA CUDA (https://developer.nvidia.com/cuda-downloads). Please select Windows 7 to download.

CuDNN 5.1

Download CuDNN 5.1 at NVIDIA website (https://developer.nvidia.com/cudnn). Please select CuDNN 5.1 for CUDA 7.5.

Caffe

Caffe can be obtained at (https://github.com/bvlc/caffe/tree/windows). Note you need to install Visual Studio 2013 to build Caffe. Before building it, enable Python support, CUDA and CuDNN by following instructions on the same page. Because we are using Official CPython, please change the value of PythonDir tag from C:\Miniconda2\ to C:\PYTHON27\ (assume your CPython installation is the default C:\PYTHON27). After building it, configure your Python environment to include pycaffe, which is described at (https://github.com/bvlc/caffe/tree/windows#remark). Your caffe.exe will be inside Build\x64\Release directory (if you made release build).

Graphviz

Graphviz is available at (www.graphviz.org/Download.PHP). Please note this site is not always available online. The installation directory can not contain space, so don’t install it under the regular ‘c:\Program Files (x86)’ directory. Try something like ‘c:\graphviz’ instead. When the installation directory contains space, pydot could not launch the dot.exe file, even it has no problem finding it. Add the c:\graphviz\bin directory to your PATH.

Installing DIGITS

Clone DIGITS from github.com (https://github.com/nvidia/digits). From the command prompt (run as administrator) and cd to DIGITS directory. Then type

python -m pip install -r requirements.txt

You may see error about Pillow, like ValueError: jpeg is required unless explicitly disabled using –disable-jpeg, aborting If this happens, download Pillow Windows Installer (Pillow-3.1.1.win-amd64-py2.7.exe) at https://pypi.python.org/pypi/Pillow/3.1.1 and run the exectuables. After installing Pillow in the above way, run

python -m pip install -r requirements.txt

again.

After the above command, check if all required Python dependencies are met by comparing requirements.txt and output of the following command.

python -m pip list

If gevent is not v1.0.2, install it from the whl file, downloaded previously from (http://www.lfd.uci.edu/~gohlke/pythonlibs/).

python -m pip install gevent-1.0.2-cp27-none-win_amd64.whl

It should uninstall the gevent you had, and install gevent 1.0.2.

Because readline is not available in Windows, you need to install one additional Python package.

python -m pip install pyreadline

Running DIGITS

First, check if caffe executable is included in your PATH environment variable. If not, add it.

set PATH=%PATH%;MY_CAFFE_ROOT\Build\x64\Release

Replace MY_CAFFE_ROOT with your local caffe directory.

Launch DIGITS devserver with the following command:

python digits-devserver

Point your browser to localhost:5000. You should be able to see DIGITS.

Troubleshooting

DIGITS crashes when trying to classify images with * Show visualizations and statistics *

This issue should have been resolved. However, if you still encounter this issue, this seems related to different hdf5 DLL binding between pycaffe and h5py. The DLL used by pycaffe was pulled from nuget, and its version is 1.8.15.2. Slightly older than the DLL in h5py. A temporary solution is to load h5py before pycaffe. To force loading h5py before pycaffe, you can either add one line at the beginning of digits-devserver file, or import h5py just before import caffe in digits/config/caffe_option.py.

import readline causes ImportError

Change import readline in digits\config\prompt.py to

try:    import readlineexcept ImportError:    import pyreadline as readline
1
2
3
4
1
2
3
4

DIGITS complains Torch binary not found in PATH

Currently, DIGITS does not support Torch on Windows platform.