[
https://issues.apache.org/jira/browse/HIVE-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378620#comment-15378620
]
Wei Zheng commented on HIVE-13934:
----------------------------------
Thanks [~hagleitn] for the review.
{quote}
DagUtils has methods to retrieve container size and xmx. The patch as is
doesn't work if container size isn't set for instance.
It's also not clear to me why these tez settings would be available in the conf
object in the map join conversion. I don't think that works.
{quote}
I will use getContainerResource to get the container size. getContainerJavaOpts
only contains hive's java opts. I assume in the context of DagUtils I can get
the Tez settings properly?
{quote}
There is also a branch missing in adjustMemoryReserveFraction. What happens
when the computed value is bigger than the max allowed? Should we set it to max?
{quote}
My previous thought was to return the default fraction if our calculation
exceeds the max. But I think returning max in that case makes more sense.
> Configure Tez to make nocondiional task size memory available for the
> Processor
> -------------------------------------------------------------------------------
>
> Key: HIVE-13934
> URL: https://issues.apache.org/jira/browse/HIVE-13934
> Project: Hive
> Issue Type: Bug
> Reporter: Wei Zheng
> Assignee: Wei Zheng
> Attachments: HIVE-13934.1.patch, HIVE-13934.2.patch,
> HIVE-13934.3.patch, HIVE-13934.4.patch, HIVE-13934.6.patch,
> HIVE-13934.7.patch, HIVE-13934.8.patch
>
>
> Currently, noconditionaltasksize is not validated against the container size,
> the reservations made in the container by Tez for Inputs / Outputs etc.
> Check this at compile time to see if enough memory is available, or set up
> the vertex to reserve additional memory for the Processor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
