[
https://issues.apache.org/jira/browse/TEZ-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044860#comment-14044860
]
Bikas Saha edited comment on TEZ-1127 at 6/26/14 5:03 PM:
----------------------------------------------------------
bq. AM command opts raising the AM memory to 2-2.5G because of huge number of
tasks
To be clear, you may want to survey these opts a bit and see what these extra
opts are.
E.g. in the above stmt do you mean they are setting -xmx values for a high
memory AM JVM (because container size for AM is not set via this config)? In
that case, you are better off not using that post TEZ-699 because Tez will
auto-determine then from the memory resource size. The auto-determined value
might be better in some case because it depends on the real value of the
resource allocated by the RM which may be higher than what was asked for. So
the xmx value may be higher than what the user initially thought it should be.
was (Author: bikassaha):
bq. AM command opts raising the AM memory to 2-2.5G because of huge number of
tasks
To be clear, you may want to survey these opts a bit and see what these extra
opts are.
E.g. in the above stmt do you mean they are setting -xmx values for a high
memory AM JVM? In that case, you are better off not using that post TEZ-699
because Tez will auto-determine then from the memory resource size. The
auto-determined value might be better in some case because it depends on the
real value of the resource allocated by the RM which may be higher than what
was asked for. So the xmx value may be higher than what the user initially
thought it should be.
> Add TEZ_TASK_JAVA_OPTS and TEZ_ENV configs to specify values from config
> ------------------------------------------------------------------------
>
> Key: TEZ-1127
> URL: https://issues.apache.org/jira/browse/TEZ-1127
> Project: Apache Tez
> Issue Type: Sub-task
> Affects Versions: 0.4.0
> Reporter: Pala M Muthaia
> Assignee: Bikas Saha
> Fix For: 0.5.0
>
> Attachments: PIG-TEZ-1127.patch, TEZ-1127.1.patch, TEZ-1127.2.patch
>
>
> There is no global setting available to specify JVM parameters for tez tasks
> (e.g: specifying native library location with java.library.path). Yarn allows
> specifying application.classpath, but not java opts. The only option right
> now is to set mapreduce.map.java.opts in the tez job config (or
> hive.tez.java.opts for Hive on Tez).
> Suggestion from Bikas, while discussing this in tez-user group, is tez should
> have "TEZ_TASK_JAVA_OPTS so that you don’t have to specify this via mapreduce
> configuration parameters"
--
This message was sent by Atlassian JIRA
(v6.2#6252)
