"main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file
来源:互联网 发布:php array to json 编辑:程序博客网 时间:2024/05/15 18:12
XP环境下使用cygwin运行nutch
thinkpad@thinkpad-117f6f ~/nutch-1.6
$ bin/nutch crawl urls -dir data -threads 5 depth 2
出现如下异常:
Injector: starting at 2013-05-11 09:28:39
Injector: crawlDb: data/crawldb
Injector: urlDir: 2
Injector: Converting injected urls to crawl db entries.
Exception in thread "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/C:/cygwin/home/thinkpad/nutch-1.6/2
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:989)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:981)
at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1261)
at org.apache.nutch.crawl.Injector.inject(Injector.java:281)
at org.apache.nutch.crawl.Crawl.run(Crawl.java:127)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:55)
分析:1,可能是权限问题:data的权限 或是没有分组,如下为none,通过实验chmod 777,发现不是这个问题。
thinkpad@thinkpad-117f6f ~/nutch-1.6
$ ls -al
总用量 397
drwxr-xr-x+ 1 thinkpad None 0 五月 11 09:43 .
drwxr-xr-x+ 1 thinkpad None 0 五月 11 08:53 ..
drwxr-xr-x+ 1 thinkpad None 0 十一 23 21:55 bin
-rwxr-xr-x 1 thinkpad None 68708 十一 23 21:55 CHANGES.txt
drwxr-xr-x+ 1 thinkpad None 0 十一 23 21:55 conf
drwxrwxrwx+ 1 thinkpad None 0 五月 11 09:44 data
drwxr-xr-x+ 1 thinkpad None 0 十一 23 21:55 docs
drwxr-xr-x+ 1 thinkpad None 0 五月 11 09:27 lib
-rwxr-xr-x 1 thinkpad None 329066 十一 23 21:55 LICENSE.txt
drwxr-xr-x+ 1 thinkpad None 0 五月 11 08:32 logs
-rwxr-xr-x 1 thinkpad None 427 十一 23 21:55 NOTICE.txt
drwxr-xr-x+ 1 thinkpad None 0 十一 23 21:55 plugins
-rwxr-xr-x 1 thinkpad None 1593 十一 23 21:55 README
分析:查看日志,这个是最直接的办法 logs/hadoop.log文件
thinkpad@thinkpad-117f6f ~/nutch-1.6
2013-05-11 10:07:24,484 ERROR security.UserGroupInformation - PriviledgedActionException as:thinkpad cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/C:/cygwin/home/thinkpad/nutch-1.6/2
怎么会多了一个 2呢?难道是自己编译的hadoop-core .jar的问题?
在nutch目录下创建一个文件夹2,再运行,OK。
结论遇到问题,查看日志,不要单纯看异常输出信息。
- "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file
- org.apache.hadoop.mapred.InvalidInputException: Input path does not exist:
- org.apache.hadoop.mapred.InvalidInputException: Input path does not exist问题
- org.apache.hadoop.mapred.InvalidInputException: Input path does not exist问题
- hadoop执行jar包时报 org.apache.hadoop.mapred.InvalidInputException: Input path does not exis
- 错误:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost/
- org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:
- org.apache.hadoop.mapreduce.lib.input.InvalidInputException:input path does not exist:hdfs://10.0.2.
- org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:
- org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:
- hadoop异常: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exi
- Exception in thread "main" org.apache.hadoop.mapred.InvalidJobConfException: Output directory not se
- eclipse编译hadoop错误:Input path does not exist: hdfs://192.168.184.21:9001/user/xxxx/input
- hadoop dfs put File does not exist
- mapreduce error:input path does not exist 的解决方法
- 运行wordcount程序报 Input path does not exist 错
- org.apache.hadoop.mapred.FileAlreadyExistsException
- org.apache.hadoop.mapred.FileAlreadyExistsException
- 实施开发组队
- 关于消费者.生产者,缓冲区的简单java实现
- 强制的类型转换
- 修复数据库脚本
- linux内核返回错误参数
- "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file
- C++中的临时对象(拷贝构造函数)(上)
- 关于dijit.form.Select中addOption()的一点注意事项
- 植物大战僵尸辅助之重叠植物
- Mozilla 免费送火狐手机 刺激开发者兴趣
- 47 个树莓派项目或许能激发你的灵感
- 小组实验任务答案公布:oracle实施前准备清单
- Twitter 工程师眼中的新浪微博
- 自己面试中遇到的for循环问题