[
https://issues.apache.org/jira/browse/TEZ-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536352#comment-15536352
]
Kuhu Shukla commented on TEZ-3362:
----------------------------------
[~hitesh], we can move the generic service plugin effort, which would provide
APIs for shuffle Port and scheduler service discovery; allowing path based
deletion to a separate JIRA under TEZ-3334. The DagAppMaster will initialize
this "Deletion Service" in a manner similar to how TaskSchedulers and
ContainerLaunchers are initialized today. Does that sound as a good starting
point?
Are we ok to have this patch in for an initial POC in a deployed environment?
Let me know and I can open a JIRA for the proposal. Thanks a lot and looking
forward to more comments and ideas!
> Delete intermediate data at DAG level for Shuffle Handler
> ---------------------------------------------------------
>
> Key: TEZ-3362
> URL: https://issues.apache.org/jira/browse/TEZ-3362
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Kuhu Shukla
> Attachments: TEZ-3362.001.patch, TEZ-3362.002.patch,
> TEZ-3362.003.patch, TEZ-3362.004.patch, TEZ-3362.005.patch
>
>
> Applications like hive that use tez in session mode need the ability to
> delete intermediate data after a DAG completes and while the application
> continues to run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
