[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062041#comment-14062041 ] Gokhan Capan commented on MAHOUT-1565: -- Sorry guys, I committed this 2 weeks ago, but I forgot to close the issue. Thank you, [~nravi] > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Fix For: 1.0 > > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060254#comment-14060254 ] Suneel Marthi commented on MAHOUT-1565: --- [~gokhancapan] can this be marked as resolved? > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Fix For: 1.0 > > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048262#comment-14048262 ] Hudson commented on MAHOUT-1565: SUCCESS: Integrated in Mahout-Quality #2678 (See [https://builds.apache.org/job/Mahout-Quality/2678/]) MAHOUT-1565 Remove mapreduce-job-specific MAHOUT_OPTS (nishkamravi via gcapan): this closes apache/mahout#4 (gcapan: rev 37b85253981770b455bd9936cef23de460662483) * bin/mahout * bin/mahout.cmd > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Fix For: 1.0 > > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019252#comment-14019252 ] Nishkam Ravi commented on MAHOUT-1565: -- Done some testing at our end. Performance seems to improve after those options are removed. Does Mahout have an automated testing infrastructure for PR's? If not, how will further testing be done and by who. > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Fix For: 1.0 > > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016378#comment-14016378 ] Gokhan Capan commented on MAHOUT-1565: -- We agree, conceptually, but this needs some further testing. > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Fix For: 1.0 > > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016135#comment-14016135 ] Nishkam Ravi commented on MAHOUT-1565: -- Has this PR been merged yet? > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Fix For: 1.0 > > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012152#comment-14012152 ] Nishkam Ravi commented on MAHOUT-1565: -- Modified the PR to remove those options from both bin/mahout and bin/mahout.cmd. > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012140#comment-14012140 ] Gokhan Capan commented on MAHOUT-1565: -- Sorry, now I can read the patch properly. The MR1 versions of those configurations are already set in bin/mahout, and you're suggesting to add MR2 versions of them, too, right? I am personally not a fan of setting such configurations in Mahout, and I would remove them as well. > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012136#comment-14012136 ] Sebastian Schelter commented on MAHOUT-1565: I'd favor removing that > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012132#comment-14012132 ] Nishkam Ravi commented on MAHOUT-1565: -- In that case, should the following be deleted from bin/mahout (lines 248-257)? MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.min.split.size=512MB" MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.map.child.java.opts=-Xmx4096m" MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.reduce.child.java.opts=-Xmx4096m" MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.output.compress=true" MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.compress.map.output=true" MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.map.tasks=1" MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.reduce.tasks=1" MAHOUT_OPTS="$MAHOUT_OPTS -Dio.sort.factor=30" MAHOUT_OPTS="$MAHOUT_OPTS -Dio.sort.mb=1024" > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012126#comment-14012126 ] Gokhan Capan commented on MAHOUT-1565: -- I think there is no point of configuring output compression, number of reducers, etc. for Mahout. > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012119#comment-14012119 ] Nishkam Ravi commented on MAHOUT-1565: -- Slight clarification: these options existed in bin/mahout for MR1 and not MR2. This is an attempt at making the two consistent. Difference in configuration may lead an observer to conclude that MR1 and MR2 are performing differently for an application built against Mahout. > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012095#comment-14012095 ] Suneel Marthi commented on MAHOUT-1565: --- It is benign indeed. We have been running the present trunk snapshot on Hadoop 2x without having to specify any of these options. What do u think [~gokhancapan] ? Resolve this as 'Not a problem' ? > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012089#comment-14012089 ] ASF GitHub Bot commented on MAHOUT-1565: Github user dlyubimov commented on the pull request: https://github.com/apache/mahout/pull/4#issuecomment-44496355 looks benign to me. Need review from folks that were working on Hadoop 2 integration, to make sure this is in line with the rest of the effort. MAHOUT-1565 > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010629#comment-14010629 ] ASF GitHub Bot commented on MAHOUT-1565: Github user nishkamravi2 commented on the pull request: https://github.com/apache/mahout/pull/4#issuecomment-44354961 JIRA: MAHOUT-1565 (https://issues.apache.org/jira/browse/MAHOUT-1565) > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAHOUT-1565) add MR2 options to MAHOUT_OPTS in bin/mahout
[ https://issues.apache.org/jira/browse/MAHOUT-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010493#comment-14010493 ] Nishkam Ravi commented on MAHOUT-1565: -- Pull request: https://github.com/apache/mahout/pull/4 > add MR2 options to MAHOUT_OPTS in bin/mahout > > > Key: MAHOUT-1565 > URL: https://issues.apache.org/jira/browse/MAHOUT-1565 > Project: Mahout > Issue Type: Improvement >Affects Versions: 1.0, 0.9 >Reporter: Nishkam Ravi > Attachments: MAHOUT-1565.patch > > > MR2 options are missing in MAHOUT_OPTS in bin/mahout and bin/mahout.cmd. Add > those options. -- This message was sent by Atlassian JIRA (v6.2#6252)
