[jira] [Updated] (NUTCH-2573) Suspend crawling if robots.txt fails to fetch with 5xx status

2022-01-14 Thread Sebastian Nagel (Jira)
[ https://issues.apache.org/jira/browse/NUTCH-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Nagel updated NUTCH-2573: --- Description: Fetcher should optionally (by default) suspend crawling by a configurable inter

[jira] [Updated] (NUTCH-2573) Suspend crawling if robots.txt fails to fetch with 5xx status

2019-09-26 Thread Sebastian Nagel (Jira)
[ https://issues.apache.org/jira/browse/NUTCH-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Nagel updated NUTCH-2573: --- Fix Version/s: (was: 1.16) 1.17 > Suspend crawling if robots.txt fails

[jira] [Updated] (NUTCH-2573) Suspend crawling if robots.txt fails to fetch with 5xx status

2018-06-28 Thread Sebastian Nagel (JIRA)
[ https://issues.apache.org/jira/browse/NUTCH-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Nagel updated NUTCH-2573: --- Fix Version/s: (was: 1.15) 1.16 > Suspend crawling if robots.txt fails

[jira] [Updated] (NUTCH-2573) Suspend crawling if robots.txt fails to fetch with 5xx status

2018-04-26 Thread Sebastian Nagel (JIRA)
[ https://issues.apache.org/jira/browse/NUTCH-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Nagel updated NUTCH-2573: --- Description: Fetcher should optionally (by default) suspend crawling by a configurable interv