Syed Shameerur Rahman created TEZ-4479:
------------------------------------------
Summary: Eagerly Init/Load FileSystem In Tez Task Containers
Key: TEZ-4479
URL: https://issues.apache.org/jira/browse/TEZ-4479
Project: Apache Tez
Issue Type: Improvement
Reporter: Syed Shameerur Rahman
Assignee: Syed Shameerur Rahman
Fix For: 0.10.3
Initing/Loading FileSystem such as S3 can take ~10s - ~20s when called for the
first time and the time taken for subsequent calls are negligable. If we can
load the FileSystem much before it is used can help us to save some time. It
can be especially useful in case of pre-warm Tez containers where the Tez task
containers comes up when the Application Master (AM) is launched and not
on-demand which is the default behavior. It can be also useful in cases where
the Mapper tasks spends considerable time consuming the upstream shuffle data
and then heads to process some FileSystem operations, in all such cases we have
few FileSystem load up time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
