技术积累

1.CDH中spark连接hbase

   解决：（1）需要将/opt/cloudera/parcels/CDH/lib/hbase/lib 目录中以hbase开头的jar包拷至/opt/cloudera/parcels/CDH/lib/spark/lib/hbase/目录中

    （2）需要将/opt/cloudera/parcels/CDH/jars/目录中的guava-12.0.1.jar,htrace-core-3.1.0-incubatin.jar,protobuf-java-2.5.0.jar也拷至/opt/cloudera/parcels/CDH/lib/spark/lib中

    （3）将/etc/spark/conf下的spark-env.sh中添加jar包的地址。export SPARK_CLASSPATH=/opt/cloudera/parcels/CDH/lib/spark/lib/hbase/*

      (4) 将spark-env.sh和/opt/cloudera/parcels/CDH/lib/spark/lib/hbase/下的jar包都拷至其它节点中。

 2.配置文件和jar包地址

     配置文件：/etc/spark/conf

      jar包地址：/opt/cloudera/parcels/CDH/lib/spark/