Nutch的日志系统
来源:互联网 发布:js replaceall 编辑:程序博客网 时间:2024/05/16 05:20
一、Nutch日志实现方式
1、Nutch使用slf4j作为日志接口,使用log4j作为具体实现。关于二者的基础,请参考
http://blog.csdn.net/jediael_lu/article/details/43854571
http://blog.csdn.net/jediael_lu/article/details/43865571
2、在java类文件中,通过以下方式输出日志消息:
(1)获取Logger对象
public static final Logger LOG = LoggerFactory.getLogger(InjectorJob.class);
(2)使用Logger进行输出
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); long start = System.currentTimeMillis(); LOG.info("InjectorJob: starting at " + sdf.format(start));
3、在log4j.properties中定义各个属性
# Define some default values that can be overridden by system propertieshadoop.log.dir=.hadoop.log.file=hadoop.log# RootLogger - DailyRollingFileAppenderlog4j.rootLogger=INFO,DRFA# Logging Thresholdlog4j.threshold=ALL#special logging requirements for some commandline toolslog4j.logger.org.apache.nutch.crawl.Crawl=INFO,cmdstdoutlog4j.logger.org.apache.nutch.crawl.InjectorJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.host.HostInjectorJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.crawl.GeneratorJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.crawl.DbUpdaterJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.host.HostDbUpdateJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.fetcher.FetcherJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.parse.ParserJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.indexer.IndexingJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.indexer.DeleteDuplicates=INFO,cmdstdoutlog4j.logger.org.apache.nutch.indexer.CleaningJob=INFO,cmdstdoutlog4j.logger.org.apache.nutch.crawl.WebTableReader=INFO,cmdstdoutlog4j.logger.org.apache.nutch.host.HostDbReader=INFO,cmdstdoutlog4j.logger.org.apache.nutch.parse.ParserChecker=INFO,cmdstdoutlog4j.logger.org.apache.nutch.indexer.IndexingFiltersChecker=INFO,cmdstdoutlog4j.logger.org.apache.nutch.plugin.PluginRepository=WARNlog4j.logger.org.apache.nutch.api.NutchServer=INFO,cmdstdoutlog4j.logger.org.apache.nutch=INFOlog4j.logger.org.apache.hadoop=WARNlog4j.logger.org.apache.zookeeper=WARNlog4j.logger.org.apache.gora=WARN## Daily Rolling File Appender#log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppenderlog4j.appender.DRFA.File=${hadoop.log.dir}/${hadoop.log.file}# Rollver at midnightlog4j.appender.DRFA.DatePattern=.yyyy-MM-dd# 30-day backup#log4j.appender.DRFA.MaxBackupIndex=30log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout# Pattern format: Date LogLevel LoggerName LogMessagelog4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} - %m%n# Debugging Pattern format: Date LogLevel LoggerName (FileName:MethodName:LineNo) LogMessage#log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n## stdout# Add *stdout* to rootlogger above if you want to use this #log4j.appender.stdout=org.apache.log4j.ConsoleAppenderlog4j.appender.stdout.layout=org.apache.log4j.PatternLayoutlog4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n## plain layout used for commandline tools to output to console#log4j.appender.cmdstdout=org.apache.log4j.ConsoleAppenderlog4j.appender.cmdstdout.layout=org.apache.log4j.PatternLayoutlog4j.appender.cmdstdout.layout.ConversionPattern=%m%n## Rolling File Appender##log4j.appender.RFA=org.apache.log4j.RollingFileAppender#log4j.appender.RFA.File=${hadoop.log.dir}/${hadoop.log.file}# Logfile size and and 30-day backups#log4j.appender.RFA.MaxFileSize=1MB#log4j.appender.RFA.MaxBackupIndex=30#log4j.appender.RFA.layout=org.apache.log4j.PatternLayout#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} - %m%n#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n
二、Nutch日志分析
1、nutch日志输出有2个appender: cmdstdout 与 DRFA。
前者将日志输出至标准输出中,后者将文件输出到每日一个的日志文件中。
2、整个工程的默认日志设置为INFO, DRFA
而nutch自身的日志被重定义为INFO,cmdstdout
hadoop, gora, zookeeper等则重定义为WARN,DRFA, 默认日志为./hadoop.log
0 0
- Nutch的日志系统
- Nutch的日志系统
- nutch优雅的插件系统,nutch中文推广的缺陷
- 从Nutch的输出日志分析其流程
- 从Nutch的输出日志分析其流程
- 从Nutch的输出日志分析其流程
- 从Nutch的输出日志分析其流程
- 【Nutch】Nutch的抓取流程
- Nutch 插件系统浅析
- 详解Nutch插件系统
- Nutch搜索引擎系统架构
- Nutch 插件系统浅析
- nutch插件系统解析
- Nutch 插件系统浅析
- Nutch 插件系统浅析
- (搜索引擎之Nutch) Nutch 插件系统浅析
- nutch系统是如何加载和管理URLFilter插件的
- Java的日志系统
- 一个硬件高手的设计经验分享
- Hdu 3336 Count the String(DP+KMP)(好题)
- 泛型编程~~好思想好思想
- 位运算
- 《精通开关电源设计》笔记
- Nutch的日志系统
- UVALive 3387 || HOJ 2196 Job Scheduling by Open Bidding
- POJ-1016 Numbers That Count
- hdu3374---String Problem
- 【SICP练习】29 练习1.35
- HTML页面间传递参数
- MOS管驱动电阻怎么选择?
- 【SICP练习】30 练习1.36
- 【SICP练习】31 练习1.37