Issues of Zeppelin

issue:

keep pending for a long time when run the code 

method:

check the .env at the /conf file

restart the zeppelin by ./bin/zeppelin-daemon.sh start

issue:

 <console>:135: error: value registerTempTable is not a member of object Roc Roc.registerTempTable("roc")

                            data.registerTempTable("data") , but it failed.

method:

CHECK THE FORMAT OF YOUR DATA

CHECK IF THE import sqlContext.implicits._ AND import sys.process._ IMPORT OR NOT

CHECK THE VERSION OF YOUR SPARK AND HADOOP 

issue:

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 4 times, most recent failure: Lost task 0.3 in stage 4.0 (TID 207, 15.50.55.162): java.io.FileNotFoundException: File file:/opt/mount2/zeppelin/data/car-eval-data.csv does not exist

method:

Spark 1.5是有问题，那个报错是Hive meta data 的问题，可能会影响Spark SQL.  

Zeppelin With Spark1.5 参数的编译失败了。
local model can be instead of these :

 

Hadoop fs –put /opt/mount2/zeppelin/data/car-eval-data.csv     /user/spark/data

val bankText1 = sc.textFile(s"data/car-eval-data.csv")

bankText1.take(10).foreach(println)

 

notes: sc.textFile("file:///data/car-eval-data.csv") should be forbade

issue:

org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://master:9000/user/spark/opt/mount2/zeppelin/data/car-eval-data.csv

method:

we can change the way to load data.

Hadoop fs –put /opt/mount2/zeppelin/data/car-eval-data.csv     /user/spark/data

val bankText1 = sc.textFile(s"data/car-eval-data.csv")

bankText1.take(10).foreach(println)

issue:

java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

method:

check the error about Hive meta data and it may influence the spark sql

vim :nohut.out  If there are ERROR at end, then it failed. Or if the compilation failed

 

issue:

java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:182) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:266) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at org.apache.zeppelin.scheduler.Job.run(Job.java:170) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

method:

add java path to conf/.env will be ok

zeppelin/conf/zeppelin-env.sh   :  export JAVA_HOME=/opt/mount2/java

issue:

java.lang.jackson error

method:

we can refer to the web pages below：

https://issues.apache.org/jira/browse/SPARK-8332

https://github.com/FasterXML/jackson-core/releases?after=jackson-core-2.4.6

http://tutorials.jenkov.com/java-json/jackson-installation.html

check the code at pom.xml below :

<fasterxml.jackson.version>2.5.3</fasterxml.jackson.version>

Zeppelin Tutorial ok, and load aa.csv is ok:

AND JUST  recompiling Spark will be ok . 

issue:

 import extenal jar package

method:

add jars into env: ZEPPELIN_JAVA_OPTS="-Dspark.jars=/mylib1.jar,/mylib2.jar -Dspark.files=/myfile1.dat,/myfile2.dat" and restart the zeppelin

issue:

  java.lang.Double can't be casted to .string 

method:

check the original data type and restart the zeppelin .

issue:

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 174.0 failed 4 times, most recent failure: Lost task 0.3 in stage 174.0 (TID 268, 15.50.55.162): java.lang.NumberFormatException: For input string: "|"

method:

it cannot recognize the input string :"|" and we can change it to any other separator.