[jira] Commented: (NUTCH-799) SOLRIndexer to commit once all reducers have finished
[ https://issues.apache.org/jira/browse/NUTCH-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12841790#action_12841790 ] Andrzej Bialecki commented on NUTCH-799: - I think it's ok to do it this way - the commit per reducer may be actually harmful if commit succeeds but the task is killed for any reason and re-ran. Note: the patch has some formatting errors. SOLRIndexer to commit once all reducers have finished - Key: NUTCH-799 URL: https://issues.apache.org/jira/browse/NUTCH-799 Project: Nutch Issue Type: Improvement Components: indexer Reporter: Julien Nioche Fix For: 1.1 Attachments: NUTCH-799.patch What about doing only one SOLR commit after the MR job has finished in SOLRIndexer instead of doing that at the end of every Reducer? I ran into timeout exceptions in some of my reducers and I suspect that this was due to the fact that other reducers had already finished and called commit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
unsubscribe
unsubscribe plz
[jira] Commented: (NUTCH-799) SOLRIndexer to commit once all reducers have finished
[ https://issues.apache.org/jira/browse/NUTCH-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12842177#action_12842177 ] Hudson commented on NUTCH-799: -- Integrated in Nutch-trunk #1087 (See [http://hudson.zones.apache.org/hudson/job/Nutch-trunk/1087/]) SOLRIndexer to commit once all reducers have finished SOLRIndexer to commit once all reducers have finished - Key: NUTCH-799 URL: https://issues.apache.org/jira/browse/NUTCH-799 Project: Nutch Issue Type: Improvement Components: indexer Reporter: Julien Nioche Assignee: Julien Nioche Fix For: 1.1 Attachments: NUTCH-799.patch What about doing only one SOLR commit after the MR job has finished in SOLRIndexer instead of doing that at the end of every Reducer? I ran into timeout exceptions in some of my reducers and I suspect that this was due to the fact that other reducers had already finished and called commit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
