Hadoop实验：用java把hdfs系统里的文件内容读取出来，显示到屏幕

来源：互联网发布：刀哥swift3.0微博源码编辑：程序博客网时间：2024/05/07 16:55

《Haoop权威指南》实验

1.设置Hadoop的类目录，hadoop-env.sh

[hadoop@Master conf]$ cd /usr/hadoop/conf
[hadoop@Master conf]$ vim hadoop-env.sh

增加 export HADOOP_CLASSPATH=/usr/hadoop/myclass

2.设置搜索目录：使到运行javac，jps等程序时，省去打入一长串路径

[hadoop@Master hadoop]$ cd /home/hadoop/
[hadoop@Master ~]$ ls -a |grep .bash_profile
.bash_profile

[hadoop@Master ~]$ vim .bash_profile

3.编写java
[hadoop@Master ~]$ cd /usr/hadoop/myclass
[hadoop@Master myclass]$ vim URLCat.java

import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;import org.apache.hadoop.io.IOUtils;import java.io.InputStream;import java.net.URL;public class URLCat{static {URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());}public static void main(String[] args) throws Exception {InputStream in = null;try {in = new URL(args[0]).openStream();IOUtils.copyBytes(in, System.out, 4096, false);} finally {IOUtils.closeStream(in);}}}

4.编译java

[hadoop@Master myclass]$ javac -classpath /usr/hadoop/hadoop-core-1.2.1.jar URLCat.java

5.用java把hdfs系统里的文件内容读取出来，显示到屏幕

[hadoop@Master myclass]$ hadoop URLCat hdfs://192.168.150.2:9000/user/hadoop/in/test1.txt

hello world

0 0