pig并发执行的错误
来源:互联网 发布:酷派cool1优化 编辑:程序博客网 时间:2024/05/22 08:06
错误信息为:
ERROR 2017: Internal error creating job configuration.
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration.
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:859)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:296)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
at org.apache.pig.PigServer.execute(PigServer.java:1297)
at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at com.bonc.dataplatform.bdi.dataflow.monitor.Mains.run(Mains.java:573)
at com.bonc.dataplatform.bdi.dataflow.monitor.Mains.run(Mains.java:123)
at com.bonc.dataplatform.bdi.dataflow.monitor.Execute.execute(Execute.java:79)
at com.bonc.dataplatform.bdi.dataflow.DataflowTask.runScript(DataflowTask.java:233)
at com.bonc.dataplatform.bdi.dataflow.DataflowTask.pigExecute(DataflowTask.java:203)
at com.bonc.dataplatform.bdi.dataflow.DataflowTask.run(DataflowTask.java:127)
at com.bonc.dataplatform.bdi.taskmanager.task.manager.TaskExecuteCommand.run(TaskExecuteCommand.java:87)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:703)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1755)
at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1124)
at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1398)
at org.apache.pig.backend.hadoop.datastorage.HPath.exists(HPath.java:106)
at org.apache.pig.impl.io.FileLocalizer.getTemporaryPath(FileLocalizer.java:480)
at org.apache.pig.impl.io.FileLocalizer.getTemporaryPath(FileLocalizer.java:474)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.shipToHDFS(JobControlCompiler.java:1521)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.putJarOnClassPathThroughDistributedCache(JobControlCompiler.java:1498)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:513)
... 21 more
解决办法
修改配置文件中的 hdfs-site.xml的以下属性 (没有就加上)
<property>
<name>fs.hdfs.impl.disable.cache</name>
<value>true</value>
</property>
0 0
- pig并发执行的错误
- PIG执行MR时报Connection refused错误
- pig的疑问:如何执行简单查询?
- Pig执行模式
- Apache Pig 执行
- python调用其它脚本:pig,python,shell等等 (如何获取pig脚本执行的返回值)
- 程序的并发执行
- Pig源码分析: 简析执行计划的生成
- pig常见错误汇总
- Pig常见错误分析
- 【pig】pig的注释格式
- 多线程并发执行的注意事项
- 简单的并发执行计时
- GCD并发的执行顺序
- 什么是程序的并发执行
- 【pig】pig的vim高亮设置
- 【pig】pig的vim高亮设置
- Linux服务器并发的错误
- jQuery操作select
- 获取操作系统信息(本机IP地址,Java版本,安装路径等)
- MySQL详解(26)----------数据库安全技术
- solr 异常:missing content stream
- LeetCode之Anagrams
- pig并发执行的错误
- Matlab绘图技巧
- Android Tool Code --- HttpDownloader from URL
- 考试系统总结
- c++学习篇之模版template
- POJ 1321 棋
- Caused by: libcore.io.ErrnoException: open failed: ENOENT (No such file or d.......
- 关于公司技术部门的管理工作的注意事项
- 向上转型与向下转型