Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://node1:9000/input/pia

来源:互联网 发布:单簧管 笛头 知乎 编辑:程序博客网 时间:2024/06/10 13:25

在Windows本地开发时遇到这样的问题:

Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://node1:9000/input/piao.txt, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:519)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:737)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:514)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:140)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:341)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at com.ibeifeng.hadoop.senior.hdfs.HdfsApp.main(HdfsApp.java:36)


部分代码:

String fileName = "hdfs://node1:9000/input/piao.txt";
Path readPath = new Path(fileName);
fileSystem.open(readPath);
FSDataInputStream inputStream = fileSystem.open(readPath);


问题的解决:

1,首先检查访问服务器的路径是否正确:hdfs://node1:9000/input/piao.txt。

2,因为本地连接服务器开发,本地的Eclipse中需要服务器中的两个配置文件:core-site.xml 和hdfs.site.xml


但是修改完了,再次运行,结果还是报错:

Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://node1:9000/input/piao.txt, expected: hdfs://node1:8020
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:184)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:101)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:293)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:289)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:289)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at com.ibeifeng.hadoop.senior.hdfs.HdfsApp.main(HdfsApp.java:35)


这里的端口需要将9000 改成8020 ,即:hdfs://node1:8020/input/piao.txt 

于是就可以愉快的跑起来了。


阅读全文
0 0