HDFS-13759
I could use FileSystem
RemoteIterator listed = fs.listStatusIterator(new
Path("hdfs://warehousestore/user/chang.wu/flat_level_1"));
like this to get files asynchronously.
But in fact what I want is a pagination support, where I could pass two
parameters, the
offset
I know, when AM making container request, if the request container is
NODE_LOCAL, then AM will also send out RACK_LOCAL and OFF_SWITCH requests. On
the ResourceManager side, if RM successfully assigned a NODE_LOCAL container, I
saw that RM canceled RACK_LOCAL and OFF_SWITCH requests because it w
I am studying the FairScheduler source cod of yarn 2.7.3.By the code of class
FSAppAttempt:
public boolean hasContainerForNode(Priority prio, FSSchedulerNode node) {
ResourceRequest anyRequest = getResourceRequest(prio, ResourceRequest.ANY);
ResourceRequest rackRequest = getResou
Hadoop Version: 2.7.2
My Yarn cluster have (1100TB,368vCores) totallly with 15** nodemangers** .
My cluster use fair-scheduler and I have 4 queues for different kinds of jobs:
10 mb, 30 vcores
422280 mb, 132 vcores
0.5f
90
90
5
From the official document of Backup Node?? it Says:
The Backup node does not need to download fsimage and edits files from the
active NameNode in order to create a checkpoint, as would be required with a
Checkpoint node or Secondary NameNode, since it already has an up-to-date state
of the name
I want to configure two JobHistoryServers and use nginx as a HA.My gold is that
even when on of my JobHistoryServer crashed , my oozie jobs can still work
without any influence.
I doubt if I can do so.
If not , what can I do to make JobHistoryServer support HighAvailability?
Many thanks.
