hadoop学习(五)
来源:互联网 发布:淘宝联盟账号怎么申请 编辑:程序博客网 时间:2024/06/04 19:51
HDFS Details for Multimachine Clusters(2nd)
Checking the NameNodes
${JAVA_HOME}/bin/jps 结果第一行为java进程的pid
Checking the DataNodes
bin/slaves.sh jps | grep Datanode | sort
在查看过程中,如果有slave失败,则需要去那台机器上查看他们的日志文件。这样会不会造成管理员压力太大的问题?
In fact, I had half of a new cluster fail to start, and it took some time to realize that the newly installed machines had a default firewall that blocked the HDFS port.
bin/hadoop dfsadmin -report 可以查看当前在线的datanode的部分信息
Tuning Factors
most important factors are network bandwidth and disk throughput. Memory use and CPU overhead for thread handling may also be issues.
The large input-split size reduces the ratio of task setup time to task run time.
Set the maximum number of requests in progress. the more requests in progress, the more contention there is for storage operations and network bandwidth, with a corresponding increase in memory requirements and CPU overhead for handling all of the outstanding requests.
Different factors per cluster.
File Descriptors(http://en.wikipedia.org/wiki/File_descriptor)
Any user that runs processes that access HDFS should have a large limit on file descriptor access, and all applications that open files need careful checking to make sure that the files are explicitly closed.
- hadoop学习(五)
- 一步一步学习hadoop(五)
- Hadoop学习之MapReduce(五)
- Hadoop学习之MapReduce(五)
- hadoop学习(五)------源码编译
- Hadoop学习五:hdfs federation
- Hadoop HDFS源码学习笔记(五)
- hadoop API 学习小结(五)
- 博客学习五——hadoop
- 学习总结五:Hadoop中CombineFileInputFormat详解
- Hadoop学习笔记(五)日志系统
- hadoop学习笔记五 -- hdfs原理
- Hadoop 学习笔记五 Hive JDBC(未完)
- Hadoop学习笔记(五)MapReduce
- hadoop学习(五)----HDFS的java操作
- Hadoop学习总结之五:Hadoop的运行痕迹
- Hadoop学习总结之五:Hadoop的运行痕迹
- Hadoop学习总结之五:Hadoop的运行痕迹
- 获取中文字符串的首字母
- 解析网卡配置静态Ubuntu IP地址
- ASP.NET网站权限设计实现(一)——使用PowerDesigner进行数据库设计
- 自删除技术 (网上收集加理解)
- OSPF协议介绍
- hadoop学习(五)
- C核心技术手册(二十四)
- C语言的谜题
- 不开心
- 深度探索Activity(1): 系统服务初始化
- JSConf 2010 (二):JS的模式编程、微格式 和 MooTools
- wo
- 用WORD2007写CSDN的BLOG
- Perl Learning (1) —— Perl简介