[
https://issues.apache.org/jira/browse/TEZ-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17777505#comment-17777505
]
Ayush Saxena commented on TEZ-4518:
-----------------------------------
[~mudit-97] let me find some time & discuss with [~abstractdog] ({_}Which ain't
happening until next week, we are occupied{_}). I think there is a similar PR
with MR:
[https://github.com/apache/hadoop/pull/6155]
Which I see is approved by [~slfan1989] already, let me explore if there are
similar concerns in MR as Rajesh mentioned are there in Tez or not, maybe if MR
supports it we can do from Tez as well.
-> Thoughts from gallery, I haven't gone through this, I just know there are
some concerns mentioned above.
> Limit number of spill files getting created
> -------------------------------------------
>
> Key: TEZ-4518
> URL: https://issues.apache.org/jira/browse/TEZ-4518
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Mudit Sharma
> Priority: Critical
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Hi,
>
> We have been facing some issues where many of our cluster node disks go full
> because of some rogue applications creating a lot of spill data
> We wanted to fail the app if more than a threshold amount of spill files are
> written
> Please let us know if any such capability is supported
>
> If the capability is not there, we are proposing it to support it via a
> config, we have added a PR for the same:
> https://github.com/apache/tez/pull/312, please let us know your thoughts on it
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
