spark中出现org.apache.htrace找不到的错误

在CDH中，开发的spark连接Hbase的时候，往往会出现找不到org.apache.htrace包。


具体错误如下：
? ? ? ERROR TableInputFormat: java.io.IOException:?
? ? ? java.lang.reflect.InvocationTargetException
? ? ?Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace
? ? ?Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace


问题原因：
CDH的Spark的classpath中引入的是/opt/cloudera/parcels/CDH/jars/htrace-core-3.0.4.jar，而这个版本的时候，htrace还是org.htrace，而3.1.0的时候已经贡献给Apache了，改叫org.apache.htrace了。


解决方法：
方法一：手动修改作业提交机器的/etc/alternatives/spark-conf/classpath.txt，在最后添上/opt/cloudera/parcels/CDH-5.4.8-1.cdh5.4.8.p0.4/jars/htrace-core-3.1.0-incubating.jar即可。(个人推荐使用)
方法二：通过提交命令的方式提交参数--conf spark.executor.extraClassPath；
#!/bin/bash
HBASE_HOME=/opt/cloudera/parcels/CDH/lib/hbase
SPARK_CLASSPATH="$HBASE_HOME/conf/:$HBASE_HOME/hbase-client.jar:$HBASE_HOME/hbase-protocol.jar:$HBASE_HOME/lib/htrace-core.jar:$HBASE_HOME/lib/htrace-core-3.1.0-incubating.jar"
spark-submit --conf spark.executor.extraClassPath=$SPARK_CLASSPATH --class com.test.Main ?test-jar-with-dependencies.jar


另外一个CDH上：
/opt/cloudera/parcels/CDH/jars/htrace-core-2.00.jar
/opt/cloudera/parcels/CDH/jars/htrace-core-3.2.0-incubating.jar
/opt/cloudera/parcels/CDH/jars/htrace-core4-4.0.1-incubating.jar


/etc/alternatives/spark-conf/classpath.txt文件中：
183 /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/jars/htrace-core-3.2.0-incubating.jar
184 /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/jars/htrace-core4-4.0.1-incubating.jar


[root@webserver2 jars]# jar -tf htrace-core-3.2.0-incubating.jar | grep Trace.
org/apache/htrace/HTraceConfiguration$MapConf.class
org/apache/htrace/HTraceConfiguration.class
org/apache/htrace/Trace.class
org/apache/htrace/TraceInfo.class
org/apache/htrace/Tracer$1.class
org/apache/htrace/Tracer$TracerHolder.class
org/apache/htrace/Tracer.class
org/apache/htrace/TraceScope.class
org/apache/htrace/TraceTree$SpansByParent$1.class
org/apache/htrace/TraceTree$SpansByParent.class
org/apache/htrace/TraceTree$SpansByProcessId$1.class
org/apache/htrace/TraceTree$SpansByProcessId.class
org/apache/htrace/TraceTree.class
org/apache/htrace/wrappers/TraceCallable.class
org/apache/htrace/wrappers/TraceExecutorService.class
org/apache/htrace/wrappers/TraceProxy$1.class
org/apache/htrace/wrappers/TraceProxy.class
org/apache/htrace/wrappers/TraceRunnable.class
org/apache/htrace/fasterxml/jackson/databind/deser/std/StackTraceElementDeserializer.class
[root@webserver2 jars]# jar -tf htrace-core4-4.0.1-incubating.jar | grep Trace.
org/apache/htrace/core/Tracer$ThreadLocalContext.class
org/apache/htrace/core/TraceRunnable.class
org/apache/htrace/core/TracerPool.class
org/apache/htrace/core/Tracer.class
org/apache/htrace/core/Tracer$1.class
org/apache/htrace/core/HTraceConfiguration.class
org/apache/htrace/core/Tracer$ThreadContext.class
org/apache/htrace/core/TraceScope.class
org/apache/htrace/core/TraceCallable.class
org/apache/htrace/core/HTraceConfiguration$MapConf.class
org/apache/htrace/core/TraceExecutorService.class
org/apache/htrace/core/TracerId.class
org/apache/htrace/core/Tracer$Builder.class
org/apache/htrace/core/TracerPool$SpanReceiverShutdownHook.class
org/apache/htrace/fasterxml/jackson/databind/deser/std/StackTraceElementDeserializer.class
[root@webserver2 jars]# jar -tf htrace-core-2.00.jar | grep Trace.
org/cloudera/htrace/HTraceConfiguration$MapConf.class
org/cloudera/htrace/HTraceConfiguration.class
org/cloudera/htrace/Trace.class
org/cloudera/htrace/TraceInfo.class
org/cloudera/htrace/Tracer$1.class
org/cloudera/htrace/Tracer.class
org/cloudera/htrace/TraceScope.class
org/cloudera/htrace/TraceTree.class
org/cloudera/htrace/wrappers/TraceCallable.class
org/cloudera/htrace/wrappers/TraceExecutorService.class
org/cloudera/htrace/wrappers/TraceProxy$1.class
org/cloudera/htrace/wrappers/TraceProxy.class
org/cloudera/htrace/wrappers/TraceRunnable.class


本测试环境：
/opt/cloudera/parcels/CDH/jars/htrace-core-2.00.jar
/opt/cloudera/parcels/CDH/jars/htrace-core-3.0.4.jar
/opt/cloudera/parcels/CDH/jars/htrace-core-3.1.0-incubating.jar


/etc/alternatives/spark-conf/classpath.txt文件中：
41 /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/jars/htrace-core-3.0.4.jar




[hdper@dsjptSIT01 jars]$ jar -tf htrace-core-2.00.jar | grep Trace.
org/cloudera/htrace/HTraceConfiguration$MapConf.class
org/cloudera/htrace/HTraceConfiguration.class
org/cloudera/htrace/Trace.class
org/cloudera/htrace/TraceInfo.class
org/cloudera/htrace/Tracer$1.class
org/cloudera/htrace/Tracer.class
org/cloudera/htrace/TraceScope.class
org/cloudera/htrace/TraceTree.class
org/cloudera/htrace/wrappers/TraceCallable.class
org/cloudera/htrace/wrappers/TraceExecutorService.class
org/cloudera/htrace/wrappers/TraceProxy$1.class
org/cloudera/htrace/wrappers/TraceProxy.class
org/cloudera/htrace/wrappers/TraceRunnable.class
[hdper@dsjptSIT01 jars]$ jar -tf htrace-core-3.0.4.jar | grep Trace
org/htrace/HTraceConfiguration$MapConf.class
org/htrace/HTraceConfiguration.class
org/htrace/Trace.class
org/htrace/TraceInfo.class
org/htrace/TraceScope.class
org/htrace/TraceTree.class
org/htrace/Tracer$1.class
org/htrace/Tracer$TracerHolder.class
org/htrace/Tracer.class
org/htrace/wrappers/TraceCallable.class
org/htrace/wrappers/TraceExecutorService.class
org/htrace/wrappers/TraceProxy$1.class
org/htrace/wrappers/TraceProxy.class
org/htrace/wrappers/TraceRunnable.class
[hdper@dsjptSIT01 jars]$ jar -tf htrace-core-3.1.0-incubating.jar | grep Trace
org/apache/htrace/HTraceConfiguration$MapConf.class
org/apache/htrace/HTraceConfiguration.class
org/apache/htrace/Trace.class
org/apache/htrace/TraceInfo.class
org/apache/htrace/Tracer$1.class
org/apache/htrace/Tracer$TracerHolder.class
org/apache/htrace/Tracer.class
org/apache/htrace/TraceScope.class
org/apache/htrace/TraceTree$SpansByParent$1.class
org/apache/htrace/TraceTree$SpansByParent.class
org/apache/htrace/TraceTree$SpansByProcessId$1.class
org/apache/htrace/TraceTree$SpansByProcessId.class
org/apache/htrace/TraceTree.class
org/apache/htrace/wrappers/TraceCallable.class
org/apache/htrace/wrappers/TraceExecutorService.class
org/apache/htrace/wrappers/TraceProxy$1.class
org/apache/htrace/wrappers/TraceProxy.class
org/apache/htrace/wrappers/TraceRunnable.class
org/apache/htrace/fasterxml/jackson/databind/deser/std/StackTraceElementDeserializer.class