[
https://issues.apache.org/jira/browse/TEZ-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171964#comment-14171964
]
Gopal V commented on TEZ-1277:
------------------------------
Need to add a NativeIO impl to do {{fallocate}}.
> Tez Spill handler should truncate files to reserve space on disk
> ----------------------------------------------------------------
>
> Key: TEZ-1277
> URL: https://issues.apache.org/jira/browse/TEZ-1277
> Project: Apache Tez
> Issue Type: Improvement
> Affects Versions: 0.5.0
> Reporter: Gopal V
> Assignee: Gopal V
>
> Occasionally tasks fail due to full disks because the disks had space when
> the task was allocating via LocalDirAllocator, but the disk space was
> actually promised to many tasks instead of just one.
> This race condition shows up when a 1Gb spill can be done in ~10s or so.
> There is no way to do this via the hadoop-fs abstraction - but an SSD based
> spill wastes most of the IOPS on journal updates about the file length
> changing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
