Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
来源:互联网 发布:matlab从excel导入数据 编辑:程序博客网 时间:2024/05/21 08:42
查看 nodemanager 日志发下 如下信息
2015-07-16 15:28:58,643 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed: 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,645 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
2015-07-16 15:28:58,658 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2015-07-16 15:28:58,711 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
解决方法
删除部分没用的数据 让磁盘已用容量下降到90%一下
2015-07-16 15:29:06,808 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests
2015-07-16 15:38:58,817 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) turned good: 1/1 local-dirs are good: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are good: /opt/beh/logs/yarn/nmlogs
0 0
- Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
- Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
- Hadoop YARN: 1/1 local-dirs are bad: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir; 1/1 log-dirs are
- Why does Hadoop report “Unhealthy Node local-dirs and log-dirs are bad”?
- hadoop2.2.0 unhelthy nodes:log-dirs turned bad
- yarn local dirs的管理实现
- dirs
- dirs
- Are the Singleton Really Bad
- kewastUnPackStats(): bad magic 1
- Nested weights are bad for performance警告
- “Nested weights are bad for performance”警告
- Nested weights are bad for performance
- Hadoop运行mapreduce实例时,抛出错误 All datanodes are bad. Aborting…
- error C2601: local function definitions are illegal?
- App Engine datastore tip: monotonically increasing values are bad
- Are You a Good Leader or a Bad Leader
- Mark一下 Nested weights are bad for performance
- Android-studio第一次开始跳过SDK下载
- Android通过反射打造可以存储任何对象的万能SharedPreferences
- java Struts + JQuery +d3.js 动态生成一个简单的拓扑图
- 【LintCode】 Best Time to Buy and Sell Stock II 买卖股票的最佳时机 II
- 总结
- Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
- c++对象模型-the sematics of function
- 2012年5月SAT香港真题解析
- Android触控基础:MotionEvent
- asp.net 生成验证码
- ubuntu备份与恢复
- LCA 最近公共祖先 小结
- 四轴飞行器初步——器件选择
- Executor