Jetson TX1配置 TensorFlow1.2.1

我的JetsonTX1刷的是JetPack-L4T-2.3.1（JetPack-L4T-2.1在这里会有各种问题）
（注：如果你是TX1请先看最后。另外，如果你懒得自己编译，我这有编译好的.whl文件，但CSDN上文件大小有限制，若需要请私信）

1. 编译安装protobuf 3.3.0

先装一些依赖库

$ sudo apt-get install git autoconf automake libtool curl make g++ unzip maven$ sudo apt-get install openjdk-8-jdk   # 这里需要装Java 1.8

安装 protobuf 3.3.0

$ git clone https://github.com/google/protobuf.git  $ cd protobuf  $ ./autogen.sh$ ./configure$ make -j2$ make check$ sudo make install$ sudo ldconfig

生成protobuf-java-util-3.3.0.jar 和 protobuf-java-3.3.0.jar，为后面编译bazel使用

$ cd protobuf/java$ mvn package

2. 编译grpc-java 0.15.0

这个的版本是需要和下面要编译的bazel相配的，比较新的版本会在编译bazel的时候会出错（具体要用什么版本看你需要的bazel版本的情况）

$ git clone https://github.com/grpc/grpc-java.git$ cd grpc-java/$ git checkout v0.15.0

修改 ~/grpc-java/compiler/build.gradle 文件

model {  toolChains {    // If you have both VC and Gcc installed, VC will be selected, unless you    // set 'vcDisable=true'    if (!vcDisable) {      visualCpp(VisualCpp) {      }    }    gcc(Gcc) {+      target("unknown")    }    clang(Clang) {    }  }  platforms {    x86_32 {      architecture "x86"    }    x86_64 {      architecture "x86_64"    }+    unknowm {}  }  components {    java_plugin(NativeExecutableSpec) {      if (arch in ['x86_32', 'x86_64', 'ppcle_64']) {        // If arch is not within the defined platforms, we do not specify the        // targetPlatform so that Gradle will choose what is appropriate.        targetPlatform arch-      } +      } else {+        targetPlatform "unknown"+      }      baseName "$protocPluginBaseName"    }  }

$ cd grpc-java/compiler/$ ../gradlew java_pluginExecutable

在 ~/grpc-java/compiler/build/exe/java_plugin/下会生成一个 protoc-gen-grpc-java

3. 编译Bazel 0.4.5

因为编译最新版本0.5.2有问题，而TensorFlow1.2.1要求最低0.4.5，所以这里编译这个版本

$ git clone https://github.com/bazelbuild/bazel.git$ cd bazel/$ git checkout 0.4.5$ cd third_party/protobuf/3.0.0/$ cp ~/protobuf/java/core/target/protobuf-java-3.3.0.jar protobuf-java-3.0.0.jar$ cp ~/protobuf/java/util/target/protobuf-java-util-3.3.0.jar protobuf-java-util-3.0.0.jar### 这里的话你如果怕出问题可以回头重新编译protobuf3.0.0，我这边是直接把3.3.0生成的拿过来用了，后面其实也没出什么问题

修改 ～/bazel/src/main/java/com/google/devtools/build/lib/util/CPU.java

public enum CPU {  X86_32("x86_32", ImmutableSet.of("i386", "i486", "i586", "i686", "i786", "x86")),  X86_64("x86_64", ImmutableSet.of("amd64", "x86_64", "x64")),  PPC("ppc", ImmutableSet.of("ppc", "ppc64", "ppc64le")),-  ARM("arm", ImmutableSet.of("arm", "armv7l")),+  ARM("arm", ImmutableSet.of("arm", "armv7l", "aarch64")),  S390X("s390x", ImmutableSet.of("s390x", "s390")),  UNKNOWN("unknown", ImmutableSet.<String>of());

编译bazel：

$ cd bazel$ export PROTOC=/usr/bin/protoc$ export GRPC_JAVA_PLUGIN=~/grpc-java/compiler/build/exe/java_plugin/protoc-gen-grpc-java$ ./compile.sh  

编译成功会显示：Build successful! Binary is here: /home/ubuntu/bazel/output/bazel；
最后将生成的可执行文件拷贝到/usr/local/bin下

sudo cp /home/ubuntu/bazel/output/bazel /usr/local/bin

4. 编译TensorFlow1.2.1

$ git clone https://github.com/tensorflow/tensorflow.git$ cd tensorflow/$ git checkout v1.2.1$ sudo apt-get install python-numpy python-dev python-pip python-wheel$ ./configure    # 注意！！！在./configure这一步中的Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:-march=armv8-a# 其它是选择默认选项，记得在选择cuda的时候选y$ bazel build --config=opt --config=cuda --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --local_resources 2048,.5,1.0 //tensorflow/tools/pip_package:build_pip_package # 注意！！！因为GCC版本大于5，所以要设置--cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"选项；另外，由于TX1的内存比较小，直接编译会炸，所以要设置--local_resources 2048,.5,1.0选项

出现了以下几个错误：
错误1：

tensorflow/stream_executor/cuda/cuda_blas.cc:1916:9: error: ‘CUBLAS_GEMM_ALGO5’ was not declared in this scope
CUBLAS_GEMM_ALGO5, CUBLAS_GEMM_ALGO6, CUBLAS_GEMM_ALGO7}) {
^
tensorflow/stream_executor/cuda/cuda_blas.cc:1916:28: error: ‘CUBLAS_GEMM_ALGO6’ was not declared in this scope
CUBLAS_GEMM_ALGO5, CUBLAS_GEMM_ALGO6, CUBLAS_GEMM_ALGO7}) {
^
tensorflow/stream_executor/cuda/cuda_blas.cc:1916:47: error: ‘CUBLAS_GEMM_ALGO7’ was not declared in this scope
CUBLAS_GEMM_ALGO5, CUBLAS_GEMM_ALGO6, CUBLAS_GEMM_ALGO7}) {
^
tensorflow/stream_executor/cuda/cuda_blas.cc:1916:64: error: unable to deduce ‘std::initializer_list<_Tp>&&’ from ‘{CUBLAS_GEMM_DFALT, CUBLAS_GEMM_ALGO0, CUBLAS_GEMM_ALGO1, CUBLAS_GEMM_ALGO2, CUBLAS_GEMM_ALGO3, CUBLAS_GEMM_ALGO4, , , }’
CUBLAS_GEMM_ALGO5, CUBLAS_GEMM_ALGO6, CUBLAS_GEMM_ALGO7}) {

修改/usr/local/cuda/targets/aarch64-linux/include/cublas_api.h

/*For different GEMM algorithm */typedef enum{    CUBLAS_GEMM_DFALT         = -1,    CUBLAS_GEMM_ALGO0         =  0,    CUBLAS_GEMM_ALGO1         =  1,    CUBLAS_GEMM_ALGO2         =  2,    CUBLAS_GEMM_ALGO3         =  3,    CUBLAS_GEMM_ALGO4         =  4,+    CUBLAS_GEMM_ALGO5         =  5,+    CUBLAS_GEMM_ALGO6         =  6,+    CUBLAS_GEMM_ALGO7         =  7,} cublasGemmAlgo_t;

错误2：

external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:359:55: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:360:55: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstore(py, psub(pcj.pmul(pc,yi),pm.pmul(ps,xi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:360:55: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:374:56: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstoreu(px, padd(pm.pmul(pc,xi),pcj.pmul(ps,yi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:374:56: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:375:69: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstoreu(px+PacketSize, padd(pm.pmul(pc,xi1),pcj.pmul(ps,yi1)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:375:69: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:376:56: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstore (py, psub(pcj.pmul(pc,yi),pm.pmul(ps,xi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:376:56: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:377:69: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstore (py+PacketSize, psub(pcj.pmul(pc,yi1),pm.pmul(ps,xi1)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:377:69: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:385:66: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstoreu(x+peelingEnd, padd(pm.pmul(pc,xi),pcj.pmul(ps,yi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:385:66: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:386:66: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstore (y+peelingEnd, psub(pcj.pmul(pc,yi),pm.pmul(ps,xi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:386:66: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:415:53: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstore(px, padd(pm.pmul(pc,xi),pcj.pmul(ps,yi)));
^
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:415:53: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
external/eigen_archive/Eigen/src/Jacobi/Jacobi.h:416:53: error: ‘struct Eigen::internal::conj_helper<__vector(4) __builtin_neon_sf, Eigen::internal::Packet2cf, false, false>’ has no member named ‘pmul’
pstore(py, psub(pcj.pmul(pc,yi),pm.pmul(ps,xi)));

这些错误是eigen的arm部分有问题，解决方法：

https://github.com/tensorflow/tensorflow/issues/9697
https://github.com/samjabrahams/tensorflow-on-raspberry-pi/issues/101
https://bitbucket.org/eigen/eigen/commits/d781c1de9834/
可以手工修改下，或者去下一个上述对arm修改过的3.3版本的Eigen去替换掉
~/.cache/bazel/_bazel_ubuntu/5a48b4b8cc5de8b183f660553d84cc0b/external/eigen_archive/Eigen（那一长串的目录具体看你自己的）下的Eigen文件夹（下载地址：https://bitbucket.org/eigen/eigen/get/1ff051b7a3ae.zip）

eigen的错误没了，但是报了其它错：

ERROR: /home/ubuntu/tensorflow/tensorflow/core/kernels/BUILD:2713:1: output
‘tensorflow/core/kernels/_objs/batch_norm_op_gpu/tensorflow/core/kernels/batch_norm_op_gpu.cu.pic.o’ was not created.
ERROR: /home/ubuntu/tensorflow/tensorflow/core/kernels/BUILD:2713:1: not all outputs were created or valid.

这个还是cuda的原因，因为NVIDIA对TX1的CUDA支持版本最高是8.0.34，而github上的一些issues提到对于TensorFlow需要用更高版本的：

https://github.com/tensorflow/tensorflow/issues/851
https://github.com/tensorflow/tensorflow/issues/10434
https://devtalk.nvidia.com/default/topic/987306/jetson-tx1/internal-error-in-nvidia-driver-code-while-building-r0-12-or-master-tensorflow-on-tegra-tx1/1

所以这个TX1上GPU版本的TensorFlow到最后因为cuda版本的原因还是失败了，暂时不知怎么解决。
另，CPU版本是可以用的。

解决方案：若要用GPU版本的，换TX2…………（配置方法如上同样）
最后将得到的whl包pip install的时候可能会有：locale.Error: unsupported locale setting，不要担心，只是个小问题，export LC_ALL=C即可解决。