[
https://issues.apache.org/jira/browse/NUTCH-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461303#comment-17461303
]
Hudson commented on NUTCH-2807:
-------------------------------
FAILURE: Integrated in Jenkins build Nutch ยป Nutch-trunk #62 (See
[https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/62/])
NUTCH-2807 SitemapProcessor to warn that ignoring robots.txt affects detection
of sitemaps (#710) (github:
[https://github.com/apache/nutch/commit/3fe291bd00fe4152c6e496cbac151c0f094856c8])
* (edit) src/java/org/apache/nutch/util/SitemapProcessor.java
> SitemapProcessor to warn that ignoring robots.txt affects detection of
> sitemaps
> -------------------------------------------------------------------------------
>
> Key: NUTCH-2807
> URL: https://issues.apache.org/jira/browse/NUTCH-2807
> Project: Nutch
> Issue Type: Improvement
> Components: robots, sitemap
> Reporter: Sebastian Nagel
> Assignee: Sebastian Nagel
> Priority: Minor
> Labels: easytask
> Fix For: 1.19
>
>
> Ignoring the robots.txt causes as a site effect that no sitemaps can be
> detected via robots.txt.
> SitemapProcessor should log a warning if robots.txt is ignored by
> configuration (NUTCH-1927/NUTCH-2803).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
