[ https://issues.apache.org/jira/browse/NUTCH-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398431#comment-16398431 ]
Kenneth McFarland commented on NUTCH-2518: ------------------------------------------ Thank you for the very quick response I have begun taking a look at the PR. It looks like the basic thing is to filter on an Exception, log and if necessary re throw. I promise to take an extended look at this ASAP, possibly tonight at 9:00pm PST (its 4:45 am here now). Thank you again sir I really appreciate the response. On Wed, Mar 14, 2018 at 2:22 AM, Sebastian Nagel (JIRA) <j...@apache.org> > Must check return value of job.waitForCompletion() > -------------------------------------------------- > > Key: NUTCH-2518 > URL: https://issues.apache.org/jira/browse/NUTCH-2518 > Project: Nutch > Issue Type: Bug > Components: crawldb, fetcher, generator, hostdb, linkdb > Affects Versions: 1.15 > Reporter: Sebastian Nagel > Assignee: Kenneth McFarland > Priority: Critical > Fix For: 1.15 > > > The return value of job.waitForCompletion() of the new MapReduce API > (NUTCH-2375) must always be checked. If it's not true, the job has been > failed or killed. Accordingly, the program > - should not proceed with further jobs/steps > - must clean-up temporary data, unlock CrawlDB, etc. > - exit with non-zero exit value, so that scripts running the crawl workflow > can handle the failure > Cf. NUTCH-2076, NUTCH-2442, [NUTCH-2375 PR > #221|https://github.com/apache/nutch/pull/221#issuecomment-332941883]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)