[
https://issues.apache.org/jira/browse/TEZ-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4479:
------------------------------
Fix Version/s: 0.10.4
(was: 0.10.3)
> Eagerly Init/Load FileSystem In Tez Task Containers
> ---------------------------------------------------
>
> Key: TEZ-4479
> URL: https://issues.apache.org/jira/browse/TEZ-4479
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
> Fix For: 0.10.4
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Initing/Loading FileSystem such as S3 can take ~10s - ~20s when called for
> the first time and the time taken for subsequent calls are negligable. If we
> can load the FileSystem much before it is used can help us to save some time.
> It can be especially useful in case of pre-warm Tez containers where the Tez
> task containers comes up when the Application Master (AM) is launched and not
> on-demand which is the default behavior. It can be also useful in cases where
> the Mapper tasks spends considerable time consuming the upstream shuffle data
> and then heads to process some FileSystem operations, in all such cases we
> have few FileSystem load up time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
