[ https://issues.apache.org/jira/browse/NUTCH-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797398#action_12797398 ]
Godmar Back commented on NUTCH-407: ----------------------------------- Hardwiring a directory such as "file:///c:/top/directory" in urlfilter-regexp is a bizarre idea that does not solve this problem. It would tie the 'urls' file specified in 'nutch crawl' to the urlfilter-regex in conf/, which is obviously bad. (Alternatively, if each 'urls' file required a separate conf/ directory, then 'urls' should be made part of conf/.) Please accept the original patch or find a better solution. > Make Nutch crawling parent directories for file protocol configurable > --------------------------------------------------------------------- > > Key: NUTCH-407 > URL: https://issues.apache.org/jira/browse/NUTCH-407 > Project: Nutch > Issue Type: Improvement > Affects Versions: 0.8 > Reporter: Thorsten Scherler > Assignee: Andrzej Bialecki > Attachments: 407.fix.diff > > > http://www.mail-archive.com/nutch-u...@lucene.apache.org/msg06698.html > I am looking into fixing some very weird behavior of the file protocol. > I am using 0.8. > Researching this topic I found > http://www.mail-archive.com/nutch-u...@lucene.apache.org/msg06536.html > and > http://www.folge2.de/tp/search/1/crawling-the-local-filesystem-with-nutch > I am on Ubuntu but I have the same problem that nutch is going down the > tree (including parents) and not up (including children from the root > url). > Further I would vote to make the fetch-parents optional and defined per > a property whether I would like this not very intuitive "feature". -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.