[jira] Commented: (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13000388#comment-13000388
]
Hadoop QA commented on MAPREDUCE-1207:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12451069/mapred-1207.patch
against trunk revision 1075216.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results:
https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/90//testReport/
Findbugs warnings:
https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/90//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/90//console
This message is automatically generated.
Allow admins to set java options for map/reduce tasks
-
Key: MAPREDUCE-1207
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: Arun C Murthy
Assignee: Krishna Ramachandran
Attachments: mapred-1207.patch
It will be useful for allow cluster-admins to set some java options for child
map/reduce tasks.
E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
jobs, it would be nice to do it for all users in such scenarios even when
people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895079#action_12895079
]
Dick King commented on MAPREDUCE-1207:
--
{{TaskRunner.java}} :
API nomenclature: {{MAPRED_MAP_ADMIN_JAVA_OPTS}} etc. is a property that
contains options, not a set of options like {{DEFAULT_MAPRED_ADMIN_JAVA_OPTS}}
, and should probably be named {{MAP_ADMIN_JAVA_OPTS_PROPNAME}} . There should
probably be such a property for mapred as well as the two separate properties
for map and reduce.
funny return value: {{getVMEnvironment}} [and after the patch,
{{setEnvFromInputString}} ] only ever returns its first parameter,
{{errorInfo}} , if it returns at all. The return value is certainly not
pulling its weight, and the method should be {{void}} .
functionality lack: Environment value substitution is extremely specialized.
It can basically only handle expansions to path-like environment variables.
The original code had this property, but with the new code it looks like we
should to be able to handle more cases. In particular,
{{updateUserLoginEnv(...) }} sets several environment variables such as
{{USER}} . The administrator is likely to want to be able to expand {{$USER}}
in a new environment variable value [ie., perhaps a path name].
{{TestTaskEnvironment.java}}
Environment variable substitution is not tested, even the limited version that
we support.
Allow admins to set java options for map/reduce tasks
-
Key: MAPREDUCE-1207
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: Arun C Murthy
Assignee: Krishna Ramachandran
Attachments: mapred-1207.patch
It will be useful for allow cluster-admins to set some java options for child
map/reduce tasks.
E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
jobs, it would be nice to do it for all users in such scenarios even when
people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12889787#action_12889787
]
Arun C Murthy commented on MAPREDUCE-1207:
--
Forcing users to set 'mapred.child.java.opts.user' rather than
mapred.child.java.opts is an incompabtible change.
Allow admins to set java options for map/reduce tasks
-
Key: MAPREDUCE-1207
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: Arun C Murthy
It will be useful for allow cluster-admins to set some java options for child
map/reduce tasks.
E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
jobs, it would be nice to do it for all users in such scenarios even when
people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12889120#action_12889120
]
Alejandro Abdelnur commented on MAPREDUCE-1207:
---
A simple solution by configuration changes would be setting the following
defaults:
mapred.child.java.opts.admin.pre=-Xmx512m
mapred.child.java.opts.user=
mapred.child.java.opts.admin.post=
mapred.child.java.opts=${mapred.child.java.opts.admin.pre}${mapred.child.java.opts.user}${mapred.child.java.opts.admin.post}
Cluster admins can set the 'admin.pre' and 'admin.post' properties to required
values and make them final.
Users can still set values using the 'user' property.
The down side of this approach is that we are changing the current contract.
Allow admins to set java options for map/reduce tasks
-
Key: MAPREDUCE-1207
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: Arun C Murthy
It will be useful for allow cluster-admins to set some java options for child
map/reduce tasks.
E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
jobs, it would be nice to do it for all users in such scenarios even when
people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12790228#action_12790228
]
Steve Loughran commented on MAPREDUCE-1207:
---
Given IPv6 is so much trouble to Hadoop, maybe that particular setting should
be everywhere anyway
Allow admins to set java options for map/reduce tasks
-
Key: MAPREDUCE-1207
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: Arun C Murthy
It will be useful for allow cluster-admins to set some java options for child
map/reduce tasks.
E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
jobs, it would be nice to do it for all users in such scenarios even when
people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12776311#action_12776311
]
Todd Lipcon commented on MAPREDUCE-1207:
+1 for the idea, though I wonder which should take precedence - should the
admin options come first or second?
Could this not be done by changing the default configuration to be final
-Djava.net.preferIPv4Stack=true ${mapred.child.java.opts.user} and then
having users use that property to append their options?
Allow admins to set java options for map/reduce tasks
-
Key: MAPREDUCE-1207
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Reporter: Arun C Murthy
It will be useful for allow cluster-admins to set some java options for child
map/reduce tasks.
E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
jobs, it would be nice to do it for all users in such scenarios even when
people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
