Package org.apache.nutch.indexer.more
Class MoreIndexingFilter
- java.lang.Object
-
- org.apache.nutch.indexer.more.MoreIndexingFilter
-
- All Implemented Interfaces:
Configurable
,IndexingFilter
,Pluggable
public class MoreIndexingFilter extends Object implements IndexingFilter
Add (or reset) a few metaData properties as respective fields (if they are available), so that they can be accurately used within the search index. 'lastModifed' is indexed to support query by date, 'contentLength' obtains content length from the HTTP header, 'type' field is indexed to support query by type and finally the 'title' field is an attempt to reset the title if a content-disposition hint exists. The logic is that such a presence is indicative that the content provider wants the filename therein to be used as the title. Still need to make content-length searchable!- Author:
- John Xing
-
-
Field Summary
-
Fields inherited from interface org.apache.nutch.indexer.IndexingFilter
X_POINT_ID
-
-
Constructor Summary
Constructors Constructor Description MoreIndexingFilter()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description NutchDocument
filter(NutchDocument doc, Parse parse, Text url, CrawlDatum datum, Inlinks inlinks)
Adds fields or otherwise modifies the document that will be indexed for a parse.Configuration
getConf()
void
setConf(Configuration conf)
-
-
-
Method Detail
-
filter
public NutchDocument filter(NutchDocument doc, Parse parse, Text url, CrawlDatum datum, Inlinks inlinks) throws IndexingException
Description copied from interface:IndexingFilter
Adds fields or otherwise modifies the document that will be indexed for a parse. Unwanted documents can be removed from indexing by returning a null value.- Specified by:
filter
in interfaceIndexingFilter
- Parameters:
doc
- document instance for collecting fieldsparse
- parse data instanceurl
- page urldatum
- crawl datum for the page (fetch datum from segment containing fetch status and fetch time)inlinks
- page inlinks- Returns:
- modified (or a new) document instance, or null (meaning the document should be discarded)
- Throws:
IndexingException
- if an error occurs during during filtering
-
setConf
public void setConf(Configuration conf)
- Specified by:
setConf
in interfaceConfigurable
-
getConf
public Configuration getConf()
- Specified by:
getConf
in interfaceConfigurable
-
-