hadoop 2.6配置记录
来源:互联网 发布:属于结构化数据的是 编辑:程序博客网 时间:2024/05/21 06:28
本地hadoop配置
1)core-site.xml
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.--><!-- Put site-specific property overrides in this file. --><configuration><property><name>hadoop.tmp.dir</name><value>/home/hadoop/tmp</value><description>A base for other temporary directories</description></property><property><name>fs.defaultFS</name><value>hdfs://nameNode:10000</value></property><property><name>io.file.buffer.size</name><value>4096</value></property></configuration>2)hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.--><!-- Put site-specific property overrides in this file. --><configuration><property> <name>dfs.nameservices</name> <value>hadoop-xxxxx</value> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>nameNode:50090</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:///home/hadoop/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:///home/hadoop/dfs/data</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> </configuration>
3) mapred-site.xml
<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.--><!-- Put site-specific property overrides in this file. --><configuration><property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobtracker.http.address</name> <value>nameNode:50030</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>nameNode:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>nameNode:19888</value> </property> </configuration>
4) yarn-site.xml
<?xml version="1.0"?><!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.--><configuration><!-- Site specific YARN configuration properties --> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>nameNode:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>nameNode:8030</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>nameNode:8031</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>nameNode:8033</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>nameNode:8088</value> </property> </configuration>
5)slaves
dataNode
6) hadoop-env.sh和yarn-env.sh
...export JAVA_HOME=/path/to/jdk...
7) 配置完执行
# hadoop namenode -format
记得配置PATH环境变量
8)查看状态
http://ejiajie-thinkpad-t450s:50070
http://ejiajie-thinkpad-t450s:8088
环境:ubuntu
0 0
- hadoop 2.6配置记录
- hadoop集群配置记录
- hadoop 配置中常见问题记录
- Hadoop Capacity Scheduler配置使用记录
- Hadoop Capacity Scheduler配置使用记录
- hadoop 开发调试环境-eclipse配置记录
- hadoop环境配置操作点滴记录
- Hadoop HA集群配置问题记录
- Hadoop的HA机制(Zookeeper集群+Hadoop集群)配置记录
- hadoop学习2 记录配置hadoop环境的那些坑
- spark+hadoop-2.6配置
- Hadoop linux(ubuntu)分布式部署配置问题记录
- Hadoop linux(ubuntu)分布式部署配置问题记录
- Hadoop linux(ubuntu)分布式部署配置问题记录
- 工作记录:win7下配置阅读hadoop源码环境
- 配置Hadoop到Ubuntu Server上的过程记录
- 配置Hadoop-2.4.0的HA+Federation+Yarn详细记录
- hadoop主机名和hbase连接名配置问题记录
- 14.8.2 Specifying the Row Format for a Table 指定表的Row Format
- android 资源读取
- TOC 系统思考培训日记2015-11-30
- 华为USG完整配置ADSL上网
- Linux TCP服务器
- hadoop 2.6配置记录
- IntelliJ IDEA 快捷键篇
- MySQL中find_in_set的用法(某个字段包含某个字符)
- 15款最好用的思维导图(心智图 )工具
- 欢迎使用CSDN-markdown编辑器
- linux下定时执行sql语句
- 第四天 魔小灯(蓝牙智能灯)1
- 数据结构C++语言描述专题系列 (一) 绪论
- 【LeetCode】191Number of 1 Bits