nutch-1.4中IndexingFilter的变化
来源:互联网 发布:硬笔书法字帖 知乎 编辑:程序博客网 时间:2024/05/24 05:56
nutch-1.4后IndexingFilter扩展点中少了一个用于添加索引Field属性的函数addIndexBackendOptions(Configuration conf),这样如果再用nutch-1.2中的LuceneWriter做索引,需要自己设置Field的属性,下面代码为nutch-1.4的IndexingFilter类源代码:
/** * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package org.apache.nutch.indexer;// Hadoop importsimport org.apache.hadoop.conf.Configurable;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.io.Text;// Nutch importsimport org.apache.nutch.parse.Parse;import org.apache.nutch.crawl.CrawlDatum;import org.apache.nutch.crawl.Inlinks;import org.apache.nutch.plugin.Pluggable;/** Extension point for indexing. Permits one to add metadata to the indexed * fields. All plugins found which implement this extension point are run * sequentially on the parse. */public interface IndexingFilter extends Pluggable, Configurable { /** The name of the extension point. */ final static String X_POINT_ID = IndexingFilter.class.getName(); /** * Adds fields or otherwise modifies the document that will be indexed for a * parse. Unwanted documents can be removed from indexing by returning a null value. * * @param doc document instance for collecting fields * @param parse parse data instance * @param url page url * @param datum crawl datum for the page * @param inlinks page inlinks * @return modified (or a new) document instance, or null (meaning the document * should be discarded) * @throws IndexingException */ NutchDocument filter(NutchDocument doc, Parse parse, Text url, CrawlDatum datum, Inlinks inlinks) throws IndexingException;}
- nutch-1.4中IndexingFilter的变化
- org.apache.nutch.indexer之IndexingFilter
- Nutch 1.3 学习笔记 12 Nutch 2.0 的主要变化
- Nutch 1.3 学习笔记 12 Nutch 2.0 的主要变化
- Nutch中metadata的分析
- Nutch中MapReduce的分析
- Nutch中MapReduce的分析
- Nutch中MapReduce的分析
- Nutch中MapReduce的分析
- Nutch中MapReduce的分析
- nutch中bin/crawl和bin/nutch crawl的用法
- [Nutch]Nutch抓取过程中生成的目录内容分析
- nutch中插件是如何调用的?
- Nutch中需要重写的部分
- 如何设置nutch中摘要的长度
- nutch学习中遇到的问题
- Eclipse中加载Nutch遇到的问题
- nutch索引中添加自己的Field
- UVa 11150 - Cola
- PDEVICE_OBJECT 定义在 ntddk.h中
- python特殊属性和方法的运
- UVa Problem 100 The 3n+1 problem (3n+1 问题)
- android 第一天课程
- nutch-1.4中IndexingFilter的变化
- UVa 11461 - Square Numbers
- “指针的指针”的作用
- Java 类修饰符
- 编程规范,借鉴他人,摘抄
- hdu 2191 悼念512汶川大地震遇难同胞——珍惜现在,感恩生活(完全背包)
- UVa Problem 10270 Bigger Square Please... (拼接正方形)
- aodv-0.9.6移植
- android应用