[
https://issues.apache.org/jira/browse/TEZ-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536590#comment-15536590
]
Kuhu Shukla commented on TEZ-3362:
----------------------------------
bq. 1. dagDeleteService - if I'm not mistaken this is setup each time a
dagCompletes, but shutdown on app completion?
True. I did not initialize it upfront to save the fixed threadpool allocation
in a case where dagDelete is not called. Shutting down the service at the end
of dagDelete might kill runnables that are queued but not executing at the time
of shutdown. May be init-ing in the beginning may not be a bad idea after all.
bq. 2. NodeTracker transitions missing on states other than ACTIVE. This will
lead to InvalidStateTransitions. A node state can change between
ContainerLaunch sending out the message, and the AMNodeTracker state machine
processing it.
Great catch! Needs fixing. Will update patch.
> Delete intermediate data at DAG level for Shuffle Handler
> ---------------------------------------------------------
>
> Key: TEZ-3362
> URL: https://issues.apache.org/jira/browse/TEZ-3362
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Kuhu Shukla
> Attachments: TEZ-3362.001.patch, TEZ-3362.002.patch,
> TEZ-3362.003.patch, TEZ-3362.004.patch, TEZ-3362.005.patch
>
>
> Applications like hive that use tez in session mode need the ability to
> delete intermediate data after a DAG completes and while the application
> continues to run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
