单机Hadoop的安装与使用

第一步：安装操作系统并创建Hadoop用户OS：RHEL6.5[root@hadoop ~]# useradd hadoop[root@hadoop ~]# passwd hadoop第二步：Java安装自带Java[root@hadoop ~]# java -versionjava version "1.7.0_45"OpenJDK Runtime Environment (rhel-2.4.3.3.el6-x86_64 u45-b15)OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)JAVA_HOME为/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64第三步：SSH登陆权限设置对于Hadoop的伪分布和全分布而言，Hadoop的NameNode需要启动集群中所有机器的Hadoop守护进程。通过SSH实现。配置SSHsu - hadoopmkdir ~/.sshchmod 700 ~/.ssh/usr/bin/ssh-keygen -t rsa/usr/bin/ssh-keygen -t dsa检查是否有~/.ssh/authorized_keys 如果没有执行下面，如果有，跳过$ touch ~/.ssh/authorized_keys$ cd ~/.ssh$ ls----------------------------------ssh rac1 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keysssh rac1 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keysssh rac2 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keysssh rac2 cat /home/oracle/.ssh/id_dsa.pub >>authorized_keysscp authorized_keys rac2:/home/oracle/.ssh/第四步：单机Hadoop安装下载安装包：hadoop-2.8.1.tar.gz上传安装包创建合适的目录，解压安装包。 cd /usr/local mkdir hadoopcp /usr/hadoop-2.8.1.tar.gz /usr/local/hadoop/tar -xzvf hadoop-2.8.1.tar.gz [hadoop@hadoop hadoop-2.8.1]$ export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre[hadoop@hadoop hadoop-2.8.1]$ ./bin/hadoop versionHadoop 2.8.1Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 20fe5304904fc2f5a18053c389e43cd26f7a70feCompiled by vinodkv on 2017-06-02T06:14ZCompiled with protoc 2.5.0From source with checksum 60125541c2b3e266cbf3becc5bda666This command was run using /usr/local/hadoop/hadoop-2.8.1/share/hadoop/common/hadoop-common-2.8.1.jar测试：mkdir inputcp /usr/local/hadoop/hadoop-2.8.1/etc/hadoop /usr/local/hadoop/hadoop-2.8.1/input./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar grep input output 'dfs[a-z.]+'结果：。。。 File System Counters                FILE: Number of bytes read=1500730                FILE: Number of bytes written=2509126                FILE: Number of read operations=0                FILE: Number of large read operations=0                FILE: Number of write operations=0        Map-Reduce Framework                Map input records=12                Map output records=12                Map output bytes=274                Map output materialized bytes=304                Input split bytes=133                Combine input records=0                Combine output records=0                Reduce input groups=5                Reduce shuffle bytes=304                Reduce input records=12                Reduce output records=12                Spilled Records=24                Shuffled Maps =1                Failed Shuffles=0                Merged Map outputs=1                GC time elapsed (ms)=34                Total committed heap usage (bytes)=274628608        Shuffle Errors                BAD_ID=0                CONNECTION=0                IO_ERROR=0                WRONG_LENGTH=0                WRONG_MAP=0                WRONG_REDUCE=0        File Input Format Counters                Bytes Read=468        File Output Format Counters                Bytes Written=214output下的信息：[root@hadoop output]# lltotal 4-rw-r--r--. 1 hadoop hadoop 202 Jul 23 14:57 part-r-00000-rw-r--r--. 1 hadoop hadoop   0 Jul 23 14:57 _SUCCESS[root@hadoop output]# vi part-r-000006       dfs.audit.logger4       dfs.class3       dfs.server.namenode.3       dfs.logger2       dfs.period2       dfs.audit.log.maxfilesize2       dfs.audit.log.maxbackupindex1       dfsmetrics.log1       dfsadmin1       dfs.servers1       dfs.log1       dfs.file