跑第一个hadoop自带示例遇到的错误
来源:互联网 发布:淘宝买家敲诈食品卖家 编辑:程序博客网 时间:2024/05/16 09:00
装完hadoop跑一个算圆周率pi值的示例的时候,先在hadoop的安装位置找到那个传说中的
hadoop-examples-xxxx.jar,xxxx因版本而异
由于安装的hadoop是2.7.0,位置在xxx/share/hadoop/mapreduce/里,xxx为hadoop安装根目录,这个东西也因为和书上的版本不同位置不太一样要自己找。
开四个job算圆周率
hadoop jar hadoop-mapreduce-examples-2.7.0.jar pi 4 1000
得到下面的错误:
java.io.IOException: Cannot create input directory QuasiMonteCarlo_1442130301493_820011182/inat org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:283)at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:606)at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:606)at org.apache.hadoop.util.RunJar.run(RunJar.java:221)at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
他说无法创建一个啥临时文件,猜测就是权限问题,修改了mapreduce目录的权限使其可写,在跑就可以了。
Number of Maps = 4Samples per Map = 1000Wrote input for Map #0Wrote input for Map #1Wrote input for Map #2Wrote input for Map #3Starting Job15/09/13 15:46:12 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id15/09/13 15:46:12 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=15/09/13 15:46:12 INFO input.FileInputFormat: Total input paths to process : 415/09/13 15:46:12 INFO mapreduce.JobSubmitter: number of splits:415/09/13 15:46:13 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1667226124_000115/09/13 15:46:13 INFO mapreduce.Job: The url to track the job: http://localhost:8080/15/09/13 15:46:13 INFO mapreduce.Job: Running job: job_local1667226124_000115/09/13 15:46:13 INFO mapred.LocalJobRunner: OutputCommitter set in config null15/09/13 15:46:13 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 115/09/13 15:46:13 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter15/09/13 15:46:13 INFO mapred.LocalJobRunner: Waiting for map tasks15/09/13 15:46:13 INFO mapred.LocalJobRunner: Starting task: attempt_local1667226124_0001_m_000000_0……………………15/09/13 15:46:14 INFO mapred.LocalJobRunner: Finishing task: attempt_local1667226124_0001_r_000000_015/09/13 15:46:14 INFO mapred.LocalJobRunner: reduce task executor complete.15/09/13 15:46:14 INFO mapreduce.Job: Job job_local1667226124_0001 running in uber mode : false15/09/13 15:46:14 INFO mapreduce.Job: map 100% reduce 100%15/09/13 15:46:14 INFO mapreduce.Job: Job job_local1667226124_0001 completed successfully15/09/13 15:46:14 INFO mapreduce.Job: Counters: 30File System CountersFILE: Number of bytes read=1379291FILE: Number of bytes written=2739253FILE: Number of read operations=0FILE: Number of large read operations=0FILE: Number of write operations=0Map-Reduce FrameworkMap input records=4Map output records=8Map output bytes=72Map output materialized bytes=112Input split bytes=660Combine input records=0Combine output records=0Reduce input groups=2Reduce shuffle bytes=112Reduce input records=8Reduce output records=0Spilled Records=16Shuffled Maps =4Failed Shuffles=0Merged Map outputs=4GC time elapsed (ms)=154Total committed heap usage (bytes)=1356857344Shuffle ErrorsBAD_ID=0CONNECTION=0IO_ERROR=0WRONG_LENGTH=0WRONG_MAP=0WRONG_REDUCE=0File Input Format Counters Bytes Read=520File Output Format Counters Bytes Written=109Job Finished in 2.592 secondsEstimated value of Pi is 3.14000000000000000000
0 0
- 跑第一个hadoop自带示例遇到的错误
- Hadoop自带的一些程序示例
- hadoop自带示例wordcount
- CloudSim 自带示例错误:
- Symbian开发遇到的第一个错误.
- ssm遇到的第一个错误
- 创建第一个Android开发工具自带的模拟器
- 使用Carbide C/C++ 遇到的第一个错误
- 第一个android程序,遇到找不到avd设备的错误
- 在Unity世界里遇到的第一个错误!
- Hadoop第一个demo运行错误
- hadoop 自带示例wordcount 详细运行步骤
- 遇到的第一个问题
- centos 6.2 自带的eclipse运行时遇到的一个错误
- 在linux下eclipse中运行hadoop自带的WordCount例子出现的两个错误
- 使用Appium测试你的iOS APP ,然后执行第一个自带的测试实例
- 使用php自带函数删除数组的第一个元素和最后一个元素
- Gtk/Glade自带的示例代码
- Favor the Simple Over the Complex
- JavaEE学习曲线
- HightChart的HTML标签(labels)
- nao机器人远程及本地模块编写
- hdu 5442 长春区域赛网络赛 1006 Favorite Donut(后缀数组)
- 跑第一个hadoop自带示例遇到的错误
- 利用JDBC完成数据库的CRUD
- (4.3.1.16)java.lang.NoSuchMethodException: setHomeActionContentDescription
- 写给自己——卡方检测
- 5、支持向量机SVM
- http://write.blog.csdn.net/postedit
- android 开发环境下载地址
- 斯坦福Developing iOS 8 Apps学习笔记(六)
- 新一代 Linux 文件系统 btrfs 简介