MapReduce控制台打印释义

来源:互联网 发布:书法边框软件 编辑:程序博客网 时间:2024/05/21 05:40

MapReduce控制台最后的打印释义如下

File System Counters 组名
FILE: Number of bytes read=204404 读入的字节数,等于输入的文件
FILE: Number of bytes written=1154836 写了多少字节,包括Mapper与Reducer
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=7868967 多读了,shellf读了很多
HDFS: Number of bytes written=97537 写操作,有可能存在了内存中
HDFS: Number of read operations=24
HDFS: Number of large read operations=0
HDFS: Number of write operations=9
Map-Reduce Framework
Map input records=3151 文本一共多少段落
Map output records=4759
Map output bytes=42831
Map output materialized bytes=52361
Input split bytes=100 切片的字节数
Combine input records=0
Combine output records=0
Reduce input groups=4752 输入了多少组
Reduce shuffle bytes=52361 shuffle字节数
Reduce input records=4759 接收的数据
Reduce output records=4759 写出来的记录数
Spilled Records=9518
Shuffled Maps =2 分区数,洗牌数
Failed Shuffles=0
Merged Map outputs=2 合并输出
GC time elapsed (ms)=14
Total committed heap usage (bytes)=898105344 在堆上使用了这么多内存
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=2622989
File Output Format Counters
Bytes Written=52298