Dissecting The Nutch Crawler -Factory classes: Overview
来源:互联网 发布:软件测试工作总结 编辑:程序博客网 时间:2024/04/29 15:03
英文原文出处:DissectingTheNutchCrawler
转载本文请注明出处:http://blog.csdn.net/pwlazy
Factory classes: Overview
> Class net.nutch.parser.ParserFactory
> used by:
> - net.nutch.db.WebDBInjector
> - net.nutch.fetcher.Fetcher
> - net.nutch.parser.ParserChecker
>
> Class net.nutch.protocol.ProtocolFactory
> used by:
> - net.nutch.fetcher.Fetcher
> - net.nutch.parser.ParserChecker
>
> Class net.nutch.net.URLFilterFactory
> used by:
> - net.nutch.db.WebDBInjector
> - net.nutch.tools.UpdateDatabaseTool
>
> Class net.nutch.plugin.PluginRepository: used by (Parser/Protocol)Factory
Nutch's ParserFactory and ProtocolFactory classes are the key extension points for the crawler. URLFilterFactory additionally provides an extension point for other components, including WebDBInjector and UpdateDatabaseTool. These "Factory" classes can all be reconfigured by editingXML config files. So before we describe the mechanics of any of the Factory classes, we need take a quick look at Nutch's configuration system.
工厂类概览
net.nutch.parser.ParserFactory 被以下几个类使用
- net.nutch.db.WebDBInjector
- net.nutch.fetcher.Fetcher
- net.nutch.parser.ParserChecker
net.nutch.protocol.ProtocolFactory 被以下几个类使用
- net.nutch.fetcher.Fetcher
- net.nutch.parser.ParserChecker
net.nutch.net.URLFilterFactory 被以下几个类使用
- net.nutch.db.WebDBInjector
- net.nutch.tools.UpdateDatabaseTool
net.nutch.plugin.PluginRepository: 被 (Parser/Protocol)Factory 使用
对于crawler来说 ParserFactory 和 ProtocolFactory 是关键的扩展点
URLFilterFactory 又另外为其他组件(比如WebDBInjector 和 UpdateDatabaseTool)提供了一个扩展点.这些工厂类可以通过编辑xml配置文件重新配置。所以在我们阐述任何一个工厂类的机制之前,我们需要迅速浏览一下nutch的配置系统
- Dissecting The Nutch Crawler -Factory classes: Overview
- Dissecting The Nutch Crawler -Factory classes: '''URLFilterFactory'''
- Dissecting The Nutch Crawler -Factory classes: '''ParserFactory''', '''ProtocolFactory'''
- Dissecting The Nutch Crawler -introduction
- Dissecting The Nutch Crawler -Summary: Nutch crawler extension points
- Dissecting The Nutch Crawler - The "nutch" shell script
- Dissecting The Nutch Crawler - Command "crawl": net.nutch.tools.CrawlTool
- Dissecting The Nutch Crawler - Command "inject": net.nutch.db.WebDBInjector
- Dissecting The Nutch Crawler -Command "generate": net.nutch.tools.FetchListTool
- Dissecting The Nutch Crawler -Command "fetch": net.nutch.fetcher.Fetcher
- Dissecting The Nutch Crawler -Aside: net.nutch.util.NutchConfig
- Dissecting The Nutch Crawler - Command "admin -create": net.nutch.tools.WebDBAdminTool
- Heritrix Crawler vs. Nutch Crawler
- 深入学习heritrix---体系结构(Overview of the crawler)
- 深入学习heritrix---体系结构(Overview of the crawler)
- 深入学习heritrix---体系结构(Overview of the crawler)
- Nutch Crawler工作流程
- Nutch Crawler工作流程
- 关于Windows XP sp2下金山词霸2005取词BUG,桌面重启修正方法
- sql
- 从头开始
- 微软ERP Ax(Axapta) 3.0 & sp4 安装过程演示
- 报表数据库
- Dissecting The Nutch Crawler -Factory classes: Overview
- 常见datagrid错误
- 微软ERP Ax(Axapta) 3.0 & sp4 安装过程演示
- C#操作数据仓库
- Cisco交换机命令清单
- 如何避免面试失败(2)
- 怎样应用osworkflow
- 《寓言中的经济学》简明纪要 - Part 2
- 中间件技术的概念和分类