history service 没有启…

2015-11-21 23:07:04,523 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1448025206651_0026_0000022015-11-21 23:07:05,041 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable2015-11-21 23:07:05,094 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:2015-11-21 23:07:05,094 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 26 cluster_timestamp: 1448025206651 } attemptId: 2 } keyId: -161280683)2015-11-21 23:07:05,929 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Attempt num: 2 is last retry: true because a commit was started.2015-11-21 23:07:05,932 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$NoopEventHandler2015-11-21 23:07:05,939 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler2015-11-21 23:07:05,941 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter2015-11-21 23:07:06,016 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]2015-11-21 23:07:06,042 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]2015-11-21 23:07:06,067 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]2015-11-21 23:07:06,097 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled2015-11-21 23:07:06,099 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Will not try to recover. recoveryEnabled: true recoverySupportedByCommitter: false numReduceTasks: 1 shuffleKeyValidForRecovery: true ApplicationAttemptID: 22015-11-21 23:07:06,123 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]2015-11-21 23:07:06,130 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Previous history file is at hdfs://cluster1:8020/tmp/hadoop-yarn/staging/root/.staging/job_1448025206651_0026/job_1448025206651_0026_1.jhist2015-11-21 23:07:06,582 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler2015-11-21 23:07:06,619 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties2015-11-21 23:07:06,675 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).2015-11-21 23:07:06,675 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started2015-11-21 23:07:06,693 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true2015-11-21 23:07:06,694 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 32015-11-21 23:07:06,694 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 332015-11-21 23:07:06,733 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at hadoop0/172.18.32.177:80302015-11-21 23:07:06,810 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: 2015-11-21 23:07:06,811 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default2015-11-21 23:07:06,863 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]2015-11-21 23:07:06,873 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService: History file is at hdfs://cluster1:8020/tmp/hadoop-yarn/staging/root/.staging/job_1448025206651_0026/job_1448025206651_0026_1.jhist2015-11-21 23:07:06,979 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1448025206651_0026, File: hdfs://cluster1:8020/tmp/hadoop-yarn/staging/root/.staging/job_1448025206651_0026/job_1448025206651_0026_2.jhist 

2015-11-21 23:07:07,112 WARN [main] org.apache.hadoop.hdfs.DFSClient: DFSInputStream has been closed already 

2015-11-21 23:07:07,113 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMasterjava.io.IOException: Was asked to shut down. 

     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1521)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:415)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1515)        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1448)2015-11-21 23:07:07,116 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1

问题是 history service 没有启动 解决办法: 配置：选择（xxxxxxxxxx:作为history sever） yarn.log-aggregation-enable truemapreduce.jobhistory.address xxxxxxxxxx::10020
mapreduce.jobhistory.webapp.addressxxxxxxxxxx:19888
sbin/mr-jobhistory-daemon.sh starthistoryserver 

我的mapred-site.xml改为下面的配置则可以在http://hadoop0:8088/cluster/apps/FINISHED看到执行成功的mapreduce任务：