Hadoop install
来源:互联网 发布:linux ant打包 编辑:程序博客网 时间:2024/05/17 10:42
Version:hadoop-1.0.3
1. download the newest version
http://www.apache.org/dyn/closer.cgi/hadoop/common/
2. unzip the downloaded zip file.
3. configuration
3.1) configure core-site.xml
<configuration> <property> <name>fs.default.name</name> <value>hdfs://[hostname]:9000</value> </property><property><name>hadoop.tmp.dir</name><value>/home/[username]/Hadoop/hadoop-1.0.3/tmp</value></property></configuration>If we don't specify the parameter:hadoop.tmp.dir , each time we restart the hadoop cluster, we need to reformat the hadoop system.
3.2) hdfs-site.xml
<configuration> <property> <name>dfs.replication</name> <value>1</value> </property></configuration>
3.3) mapred-site.xml
<configuration> <property> <name>mapred.job.tracker</name> <value>[<span style="font-family:Arial, Helvetica, sans-serif;">hostname]</span>:9001</value> </property></configuration>
3.4) hadoop-env.sh
export JAVA_HOME=/opt/java/jdk1.7.0_51
3.5) masters and slaves
Since we set up a Pseudo-Distributed Mode, we change the file content from localhost to [hostname] for these 2 files.
4. Format the new distributed-system
In the terminal, input the following command:
$ bin/hadoop namenode -format
5. Setup passphraseless ssh
Check if you can ssh to the your hostname without a passphrase:
$ ssh [hostname]
If you cannot ssh to [hostname] without a passphrase, execute the following commands:
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Possible problem:
ssh: connect to host localhost port 22: Connection refused
It should be from ssh, sshd not installed or firewall blocked.
install ssh:
$ sudo apt-get install ssh
install sshd:
$ sudo apt-get install openssh-server<span><span>$ sudo net start sshd</span></span>
disable firewall
$ sudo ufw disable
6. start the hadoop
$ bin/start-all.sh
7. Succeed
1) Check from the terminal:
in the terminal, input:
$ jps
As we can see that, jobtracker, namenode, datanode, secondaryNamenode, taskTracker have already been started.
2) Check from the webpage:
namenode :
http://[hostname]:50070/jobtracker:
http://[hostname]:50030/
0 0
- Hadoop install
- Hadoop Install
- VM ubuntu install Hadoop
- hadoop native lib install
- Apache hadoop install
- CentOS7 Hadoop Install
- install hadoop in fedora
- install hadoop in docker
- hadoop standalone install
- install cluster hadoop 安装集群版hadoop
- Quick Install Hadoop on Windows
- Hadoop install and set up
- mac hadoop install & run application
- Install Hadoop on local machine
- install apache hadoop on ubuntu
- hadoop 学习笔记(1)-for linux install
- Install Hadoop on Ubuntu with two computers
- Hadoop 源码下载,编译,install,导入eclipse
- 设计模式之单例模式 (一)
- oracle知识点
- JVM详解之Java垃圾回收机制详解和调优
- Java IO 文件进行复制
- c printf 输出格式
- Hadoop install
- 现在淘客很难做吗?一个做淘客三年来的经历与心得
- C++文件操作
- Java编程 练习一
- jQuery - Select相关
- Introduction to Informatica PowerCenter
- 判断一个链表有否有环。有的话求出链表长度、环长度和连接点。
- VS2013运行错误:error LNK2001: 无法解析的外部符号
- C++ 命名空间namespace