第一篇 Hadoop安装单节点集群
来源:互联网 发布:java方法引用 编辑:程序博客网 时间:2024/06/06 02:52
实验一:hadoop伪分布式环境的搭建
1.实验环境
2.查看实验环境
[root@localhost ~]# cat /etc/issue
CentOS release 6.8 (Final)
Kernel \r on an \m
[root@localhost ~]# ifconfig |grep inet
inet addr:192.168.1.63 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe0a:ce73/64 Scope:Link
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
[root@localhost ~]#
3.用rz命令上次本地下载好的hadoop、jdk
[root@localhost ~]# yum install openssh-client* -y
[root@localhost ~]# yum install lrzsz* -y
Installed:
lrzsz.x86_64 0:0.12.20-27.1.el6
Complete!
[root@localhost ~]# rz
[root@localhost ~]# ls
anaconda-ks.cfg hadoop-2.7.3.tar.gz install.log install.log.syslog jdk-8u121-linux-x64.rpm
4.安装JDK1.8并配置环境变量
[root@localhost ~]# rpm -ivh jdk-8u121-linux-x64.rpm
[root@localhost java]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_121/
export PATH=$PATH:$JAVA_HOME/bin
5.解压hadoop到/usr/local/
[root@localhost ~]# tar -xf hadoop-2.7.3.tar.gz -C /usr/local/bin/hadoop/
[root@localhost ~]# cd !$
cd /usr/local/bin/hadoop/
6.修改etc/hadoop/core-site.xml
[root@localhost hadoop]# cd hadoop-2.7.3/
[root@localhost hadoop-2.7.3]# vi etc/hadoop/core-site.xml
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property></configuration>
7.修改etc/hadoop/hdfs-site.xml
[root@localhost hadoop-2.7.3]# vim etc/hadoop/hdfs-site.xml
<configuration> <property> <name>dfs.replication</name> <value>1</value> </property></configuration>
8.格式化文件系统
[root@localhost hadoop-2.7.3]# bin/hdfs namenode -format
9.修改etc/hadoop/hadoop-env.sh
[root@localhost hadoop-2.7.3]# vim etc/hadoop/hadoop-env.sh
把里面的JAVA_HOME写死
10.启动NameNode守护进程和DataNode守护进程
[root@localhost hadoop-2.7.3]# sbin/start-dfs.sh
Starting namenodes on [localhost]
root@localhost’s password:
localhost: starting namenode, logging to /usr/local/bin/hadoop/hadoop-2.7.3/logs/hadoop-root-namenode-localhost.localdomain.out
root@localhost’s password:
localhost: starting datanode, logging to /usr/local/bin/hadoop/hadoop-2.7.3/logs/hadoop-root-datanode-localhost.localdomain.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host ‘0.0.0.0 (0.0.0.0)’ can’t be established.
RSA key fingerprint is d2:fb:cd:1e:10:0d:ec:6c:ad:20:7c:cf:63:d8:01:18.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added ‘0.0.0.0’ (RSA) to the list of known hosts.
root@0.0.0.0’s password:
0.0.0.0: starting secondarynamenode, logging to /usr/local/bin/hadoop/hadoop-2.7.3/logs/hadoop-root-secondarynamenode-localhost.localdomain.out
[root@localhost hadoop-2.7.3]# jps
1952 DataNode
2242 Jps
2133 SecondaryNameNode
1863 NameNode
注:jps命令为JDK自带的查看所有Java进程的pid(将在JVM性能调优中提到)
11.配置ssh免密钥登陆(因为我们发现不设置,会频繁请求输入密码)
[root@localhost hadoop-2.7.3]# ssh-keygen -t rsa -P ” -f ~/.ssh/id_rsa #生成密钥
Generating public/private rsa key pair.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
c2:9e:3d:2f:89:b4:aa:70:2a:ac:8f:29:d8:d9:f7:b7 root@localhost.localdomain
The key’s randomart image is:
+–[ RSA 2048]—-+
| |
| |
| |
| . |
| o S |
| ..+ |
|+..o .ooo. |
|+Bo . + oo. |
|O.o..o ..oE. |
[root@localhost hadoop-2.7.3]# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[root@localhost hadoop-2.7.3]# chmod 0600 ~/.ssh/authorized_keys
12.停止并验证是否配置成功
[root@localhost hadoop-2.7.3]# sbin/stop-dfs.sh
Stopping namenodes on [localhost]
localhost: stopping namenode
localhost: stopping datanode
Stopping secondary namenodes [0.0.0.0]
0.0.0.0: stopping secondarynamenode
[root@localhost hadoop-2.7.3]# sbin/start-dfs.sh
Starting namenodes on [localhost]
localhost: starting namenode, logging to /usr/local/bin/hadoop/hadoop-2.7.3/logs/hadoop-root-namenode-localhost.localdomain.out
localhost: starting datanode, logging to /usr/local/bin/hadoop/hadoop-2.7.3/logs/hadoop-root-datanode-localhost.localdomain.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/bin/hadoop/hadoop-2.7.3/logs/hadoop-root-secondarynamenode-localhost.localdomain.out
[root@localhost hadoop-2.7.3]# jps
2897 SecondaryNameNode
2678 NameNode
3068 Jps
2767 DataNode
13.浏览器访问NameNode
http://192.168.1.63:50070/
- 第一篇 Hadoop安装单节点集群
- Hadoop单机单节点集群安装配置
- Hadoop笔记之一:安装单节点集群
- Hadoop安装-单节点伪集群
- hadoop安装,并配置单节点hadoop集群
- Hadoop单节点集群安装(伪分布式安装)
- Hadoop:设置单节点集群
- Hadoop 单节点集群配置
- Hadoop 单节点集群配置
- 创建单节点Hadoop集群
- 安装单节点伪分布式 CDH hadoop 集群
- Hadoop集群安装部署---单节点伪分布式
- HADOOP单节点安装
- hadoop单节点安装
- hadoop单节点安装
- hadoop单节点安装
- 单节点hadoop安装
- Hadoop学习---第一篇搭建Hadoop集群
- 安卓 多媒体音乐播放器
- 使用包管理java中的类
- AT&T 帝国的余晖
- Object类有哪些公用方法?
- java运算符
- 第一篇 Hadoop安装单节点集群
- zlib的minizip和lua 5.1.4安卓编译需要改动的地方
- 项目与运营
- Java读取网页信息
- python字符串编码问题
- C#并发编程(一)
- 一篇文章带你搞懂DEX文件的结构
- Git中经常使用的命令小结
- Gym 100796C Minimax Tree