in conf/nutch-site.xml
#add the parse-pdf plugin to plugin.includes
<property>
<name>plugin.includes</name>
<value>protocol-http|parse-(text|html|pdf)|index-basic|query-
(stemmer|site|url)</v
alue>
<description>Regular expression naming plugin directory names to
include. Any plugin not matching this expression is excluded. By
default Nutch includes crawling just HTML and plain text via HTTP,
and basic indexing and search plugins.
</description>
</property>
#set http.content.limit large enough for the pdf's you want to parse
<property>
<name>http.content.limit</name>
<value>65536</value>
<description>The length limit for downloaded content, in bytes.
If this value is nonnegative (>=0), content longer than it will be
truncated;
otherwise, no truncation at all.
</description>
</property>
# in regex-urlfilter.txt remove pdf from the list of link suffixes
that you skip
# skip image and other suffixes we can't yet parse
-\.(gif|GIF|jpg|JPG|ico|ICO|css|sit|eps|wmf|rtf|zip|ppt|mpg|xls|gz|
rpm|tgz|mov|MOV|exe|bin)$
;rob
--
Robert C. Pettengill, Ph.D.
[EMAIL PROTECTED]
Questions about petroleum?
Goto: http://AskAboutOil.com/
Need help implementing search?
Goto: http://MesaVida.com/
On 2005, Jul 13, at 8:41 PM, Clint Cagle wrote:
Nutch question - how do you enable the pdf plugin to parse pdf
