Flume学习12_Flume Spooling directory source读取文件格式的要求
来源:互联网 发布:用友u8软件 编辑:程序博客网 时间:2024/06/13 10:58
把Flume的Source设置为 Spooling directory source,在设定的目录下放置需要读取的文件,一些文件在读取过程中会报错。
文件格式和报错如下:
实验一
读取汉子和“;"符号的TXT文档
会报一下错误
2015-11-06 22:16:02,386 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-2 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.java.nio.charset.MalformedInputException: Input length = 1at java.nio.charset.CoderResult.throwException(CoderResult.java:281)at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
实验二
读取数字和”;“符号的TXT文档
报错
2015-11-06 22:22:49,012 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-3 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.java.nio.charset.MalformedInputException: Input length = 1at java.nio.charset.CoderResult.throwException(CoderResult.java:281)at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)此报错与前面的报错相同。
实验三
读取数字和”回车符“的TXT文档
报错:
2015-11-06 22:28:06,889 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-7 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.java.nio.charset.MalformedInputException: Input length = 1at java.nio.charset.CoderResult.throwException(CoderResult.java:281)at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
实验四
读取数字,字母,汉字组成的csv文件
报错:
2015-11-06 22:40:34,781 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-6 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.java.nio.charset.MalformedInputException: Input length = 1at java.nio.charset.CoderResult.throwException(CoderResult.java:281)at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)
实验五
读取数字和汉字组成的TXT文件
报错:
2015-11-06 23:34:29,455 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /home/hadoop/flumeSpool-10 }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.java.nio.charset.MalformedInputException: Input length = 1<span style="white-space:pre"></span>at java.nio.charset.CoderResult.throwException(CoderResult.java:281)<span style="white-space:pre"></span>at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)<span style="white-space:pre"></span>at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)<span style="white-space:pre"></span>at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)<span style="white-space:pre"></span>at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)<span style="white-space:pre"></span>at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)<span style="white-space:pre"></span>at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)<span style="white-space:pre"></span>at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<span style="white-space:pre"></span>at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)<span style="white-space:pre"></span>at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
成功的实验一
一排数字,什么符号都没有
成功的实验二
如上所示,不缺定是否使用了”回车符“。
0 0
- Flume学习12_Flume Spooling directory source读取文件格式的要求
- flume学习05---Spooling Directory Source
- flume学习(十一):如何使用Spooling Directory Source
- flume学习(十一):如何使用Spooling Directory Source
- flume学习(十一):如何使用Spooling Directory Source
- flume学习(十一):如何使用Spooling Directory Source
- flume学习(十):如何使用Spooling Directory Source
- Flume的Spooling Directory Source支持Sub-directories
- Flume学习6_flume配置文件
- [FLUME-2052]Spooling directory source should be able to replace or ignore malformed characters
- Flume学习3_Flume NG:Flume 发展史上的第一次革命
- Flume学习7_Flume配置文件内容详解
- Flume学习10_Flume->Kafka数据流操作
- Flume学习2_Flume NG简介、配置实战、技术架构应用和可能遇到的问题
- Flume-ng-1.3.0 spooling source的方式增加了对目录的递归检测的支持
- Flume学习1_Flume NG架构设计要点及配置实践
- Flume学习5_Flume环境部署和配置详解及案例大全
- Flume学习02 — Source
- hdu2196 Computer(树形dp)
- Log4j没有正确配置导致项目没有发布成功
- HDU 3534 Tree(dfs统计树的直径的数量)
- 对于重载new函数所遇到的问题记录
- 手势识别器的用法
- Flume学习12_Flume Spooling directory source读取文件格式的要求
- 个人博客配置百度Ueditor--jsp版本
- 多线程
- WIN7 完全卸载oracle11g步骤
- Android中的onActivityResult和setResult方法的使用
- S3C2440的内存管理
- 数据结构实验之栈二:一般算术表达式转换成后缀式
- STM32按键输入、所需要的C语言复习、时钟系统框图
- 寻找下标(C程序设计进阶第2周)