livy提交任务报错com.cloudera.livy.shaded.kryo.kryo.KryoException: Unable to find class: GATest.ConJob
来源:互联网 发布:微商城源码 编辑:程序博客网 时间:2024/05/01 14:22
今天在测试的时候,出现这个错误,找了好久网上找不到,最近尝试多次终于成功了。
问题就在于:Unable to find class
你要把你已经uploadJar的jar包中包含的类,引入工程。
我用spring-boot+livy+spark的一个例子:
public class Contains extends Kryo{ @RequestMapping("/contains") @ResponseBody public String contains( @RequestParam(value="path1", defaultValue="") String path1, @RequestParam(value="path2", defaultValue="") String path2 ) { double sum=0; if (path1.length() == 0 || path2.length() == 0) { return "path is null"; }else { try{ LivyClient client = new LivyClientBuilder().setURI(new URI("http://master.esri.local:8998/")).build(); System.out.println("this is the client"+ "*******************************************"); client.uploadJar(new File("/usr/local/jarsubmit/EsriGA.jar")).get(); System.out.println("this is the uploadJar"+ "*******************************************"); //client.uploadJar(new File("C:\\testGA\\GATest2.jar")).get(); sum = client.submit(new GATest.ConJob(10)).get(); //sum = client.submit(com.esrichina.gaextension.app.IntersectApp).get(); System.out.println("the client is submitted "+ "*******************************************"); client.stop(true); System.out.println("the client is stopped "+ "*******************************************"); }catch (Exception e){ e.printStackTrace(); } } return Double.toString(sum); } public static void main(String[] args) throws Exception { SpringApplication.run(Contains.class, args); }}
ConJob的源码:package GATest;import com.cloudera.livy.Job;import com.cloudera.livy.JobContext;/** * Created by xiao on 2017/8/4. */public class ConJob implements Job<Double> { private final int samples; public ConJob(int samples) { this.samples = samples; } int num = 0; @Override public Double call(JobContext ctx) throws Exception { for (int i = 0; i < samples; i++) { System.out.println("this is the number"+ i); num+=i; } double dnum=num; System.out.println("this is the number"+ dnum); return dnum; }}
阅读全文
0 0
- livy提交任务报错com.cloudera.livy.shaded.kryo.kryo.KryoException: Unable to find class: GATest.ConJob
- 关于spark运行FP-growth算法报错com.esotericsoftware.kryo.KryoException
- com.esotericsoftware.kryo.kryoexception java.util.ConcurentModificationException
- org.apache.hive.com.esotericsoftware.kryo.kryoexception: encountered unregistered class id 错误解决办法
- com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 0, required: 1
- Kryo
- 启动cloudera-scm-server报错:unable to load class [com.mysql.jdbc.Driver]
- dubbox 中kryo序列化 KryoException: Encountered unregistered class ID: 114
- 利用 livy 远程提交 spark作业
- 利用 livy 远程提交 spark作业
- Spark & Livy
- spark livy
- Kryo 怎麼用
- 使用Kryo
- Livy 安装使用说明
- Livy原理详解
- 如何源码编译Livy
- spark livy hue
- 菲律宾书法家若飞先生一行来访汪国新北京诗书画院
- linux shell 使用双小括号进行算术运算
- C语言初步学习(适合与学习过C++和java的童鞋)
- MySQL密码忘了怎么办?MySQL重置root密码方法
- Centos7 -samba服务配置
- livy提交任务报错com.cloudera.livy.shaded.kryo.kryo.KryoException: Unable to find class: GATest.ConJob
- iOS程序类型
- Extjs中GridPanel实现单元格自动换行的补充
- MySQL优化
- FAAC函数介绍很好的两篇博文
- PyQt编程之如何在屏幕中央显示窗体
- Codeforces Round #FF (Div. 2)(A-D 未完)
- 最小上升子序列1
- 有关析构的顺序