livy提交任务报错com.cloudera.livy.shaded.kryo.kryo.KryoException: Unable to find class: GATest.ConJob

来源:互联网 发布:微商城源码 编辑:程序博客网 时间:2024/05/01 14:22

今天在测试的时候,出现这个错误,找了好久网上找不到,最近尝试多次终于成功了。

问题就在于:Unable to find class

你要把你已经uploadJar的jar包中包含的类,引入工程。

我用spring-boot+livy+spark的一个例子:

public class Contains extends Kryo{    @RequestMapping("/contains")    @ResponseBody    public String contains(            @RequestParam(value="path1", defaultValue="") String path1,            @RequestParam(value="path2", defaultValue="") String path2            ) {        double sum=0;        if (path1.length() == 0 || path2.length() == 0) {            return "path is null";        }else {            try{                LivyClient client = new LivyClientBuilder().setURI(new URI("http://master.esri.local:8998/")).build();                System.out.println("this is the client"+ "*******************************************");                client.uploadJar(new File("/usr/local/jarsubmit/EsriGA.jar")).get();                System.out.println("this is the uploadJar"+ "*******************************************");               //client.uploadJar(new File("C:\\testGA\\GATest2.jar")).get();                sum = client.submit(new GATest.ConJob(10)).get();                //sum = client.submit(com.esrichina.gaextension.app.IntersectApp).get();                System.out.println("the client is submitted "+ "*******************************************");                client.stop(true);                System.out.println("the client is stopped "+ "*******************************************");            }catch (Exception e){                e.printStackTrace();            }        }        return Double.toString(sum);    }    public static void main(String[] args) throws Exception {        SpringApplication.run(Contains.class, args);    }}



ConJob的源码:package GATest;import com.cloudera.livy.Job;import com.cloudera.livy.JobContext;/** * Created by xiao on 2017/8/4. */public class ConJob  implements Job<Double> {    private final int samples;    public ConJob(int samples) {        this.samples = samples;    }    int num = 0;    @Override    public Double call(JobContext ctx) throws Exception {        for (int i = 0; i < samples; i++) {            System.out.println("this is the number"+ i);            num+=i;        }        double dnum=num;        System.out.println("this is the number"+ dnum);        return dnum;    }}


原创粉丝点击