[
https://issues.apache.org/jira/browse/TEZ-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15534650#comment-15534650
]
Hitesh Shah commented on TEZ-3362:
----------------------------------
Generally looks fine. I am not sure how this works in the hive-llap mode. The
current impl seems hardwired to use the yarn scheduler service but shouldnt the
impl invoke the appropriate service plugin to do the actual clean up? For
Hive-LLAP, this would mean sending a signal to each llap daemon that the dag
task ran on. This does get a bit more complex for cases where each vertex could
be using a diff service plugin. i.e. vertex1 ran all its tasks in llap but
vertex2 ran all its tasks in yarn containers.
\cc [~sseth]
> Delete intermediate data at DAG level for Shuffle Handler
> ---------------------------------------------------------
>
> Key: TEZ-3362
> URL: https://issues.apache.org/jira/browse/TEZ-3362
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Kuhu Shukla
> Attachments: TEZ-3362.001.patch, TEZ-3362.002.patch,
> TEZ-3362.003.patch, TEZ-3362.004.patch, TEZ-3362.005.patch
>
>
> Applications like hive that use tez in session mode need the ability to
> delete intermediate data after a DAG completes and while the application
> continues to run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
