hadoop第二周作业
来源:互联网 发布:ubuntu进入单用户模式 编辑:程序博客网 时间:2024/06/03 23:38
一、在Hadoop集群中编译并运行《权威指南》中的例3.2
- 在hdfs系统中创建测试文件test1.txt,并随便编写一些内容
- 在hadoop目录下创建myclass文件夹
- 编写class文件FileSystemCat.java
import java.io.IOException;import java.io.InputStream;import java.net.URI;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FileSystem;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IOUtils;public class FileSystemCat { public static void main(String[] args) throws IOException { String uri =args[0]; Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(URI.create(uri),conf); InputStream in = null; try{ in = fs.open(new Path(uri)); IOUtils.copyBytes(in, System.out, 4096,false); }finally{ IOUtils.closeStream(in); } }}
- 编译java文件
// 由于环境是hadoop2.x所以,没有hadoop1.x中的hadoop-core.jar文件,需要用到hadoop-common和hadoop-annotationsjavac -classpath $HADOOP_HOME/share/hadoop/common/hadoop-common-2.4.1.jar:$HADOOP_HOME/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar FileSystemCat.java
执行命令
生成的.class文件
- 利用编译得到的class文件执行查看功能
可以看到,test1.txt中的内容是“hello hadoop”
二、上传
自行在本地文件系统生成一个大约一百多字节的文本文件,写一段程序(可以利用Java API或C API),读入这个文件,并将其第101-120字节的内容写入HDFS成为一个新文件,提供代码和执行结果演示抓图
创建本地测试文件test2.txt
There are moments in life when you miss someone so much that you just want to pick them from your dreams and hug them for real! Dream what you want to dream;go where you want to go;be what you want to be,because you have only one life and one chance to do all the things you want to doThere are moments in life when you miss someone so much that you just want to pick them from your dreams and hug them for real! Dream what you want to dream;go where you want to go;
编写上传到hdfs的代码
import java.io.File;import java.io.FileInputStream;import java.io.IOException;import java.io.OutputStream;import java.net.URI;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FileSystem;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IOUtils;public class UploadFileBetwen101To102Byte { public static void main(String[] args) throws IOException { String src = args[0]; String tar = args[1]; FileInputStream in = null; OutputStream out = null; Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(URI.create(tar),conf); try{ in = new FileInputStream(new File(src)); out = fs.create(new Path(tar)); in.skip(100); byte[] buffer = new byte[20]; int read = in.read(buffer); out.write(buffer,0,read); }finally{ IOUtils.closeStream(in); IOUtils.closeStream(out); } }}
- 编译运行,并查看文件
三、下载
2的反向操作,在HDFS中生成一个大约一百多字节的文本文件,写一段程序(可以利用Java API或C API),读入这个文件,并将其第101-120字节的内容写入本地文件系统成为一个新文件,提供代码和执行结果演示抓图
在hdfs上创建测试文件test3.txt
There are moments in life when you miss someone so much that you just want to pick them from your dreams and hug them for real! Dream what you want to dream;go where you want to go;be what you want to be,because you have only one life and one chance to do all the things you want to doThere are moments in life when you miss someone so much that you just want to pick them from your dreams and hug them for real! Dream what you want to dream;go where you want to go;
编写下载到本地的代码
import java.io.File;import java.io.FileOutputStream;import java.io.IOException;import java.io.InputStream;import java.io.OutputStream;import java.net.URI;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FileSystem;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IOUtils;public class DownloadFileBetwen101To102Byte { public static void main(String[] args) throws IOException { String src = args[0]; String tar = args[1]; InputStream in = null; OutputStream out = null; Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(URI.create(tar),conf); try{ in = fs.open(new Path(src)); out = new FileOutputStream(new File(tar)); in.skip(100); byte[] buffer = new byte[20]; int read = in.read(buffer); out.write(buffer,0,read); }finally{ IOUtils.closeStream(in); IOUtils.closeStream(out); } }}
- 编译,运行,查看文件
0 0
- hadoop第二周作业
- 第二周作业第二题
- 第二周作业
- 第二周作业
- 第二周作业
- 第二周作业
- 第二周作业
- 第二周作业 1
- 第二周作业 2
- 第二周作业 3
- 第二周作业 4
- 第二周作业 3
- Java第二周作业
- 第二周课后作业
- 网络第二周作业
- 第二周作业c++
- 第二周c++作业
- 第二周作业
- 快速排序的Java实现
- 读取文件时用String的split()函数出现数组越界,但是实际应该有值
- JVM性能调优监控工具jps、jstack、jmap、jhat、jstat等使用详解
- 在一个页面上,一块区域上显示另一个页面(或者文件)的内容
- 阻塞队列实现的生产者/消费者模式
- hadoop第二周作业
- ubuntu上安装mysql
- java面试题转载地址
- 【剑指offer】实现包含min函数的栈
- LDA人脸识别详细解读
- css 选择器使用之注意事项
- Leetcode 189 Rotate Array
- 非常简单的重装win10 64位系统教程
- Eclipse UML插件推荐 Amateras Modeler