+1 (non-bindings)
We have just deployed the latest bits (62f1ce2a3d9) from YARN-2915 in our
test cluster and ran multiple jobs. We confirm that Federation is working
e2e!
Our cluster setup: eight sub-clusters, each with one RM and four NM nodes.
One Router machine. SQL Server in Ubuntu is used as
>
> > >> -
> > >> Srinivas
> > >>
> > >> - Typed on tiny keys. pls ignore typos.{mobile app}
> > >>
> > >> On Thu 22 Nov, 2018, 03:27 Chang Qiang Cao > wrote:
> > >>
> > >>> Congrats Bo
Botong Huang created YARN-5836:
--
Summary: NMToken passwd not checked in ContainerManagerImpl, so
that malicious AM can fake the Token and kill containers of other apps at will
Key: YARN-5836
URL: https
Botong Huang created YARN-6016:
--
Summary: Bugs in AMRMProxy handling AMRMToken and local AMRMToken
Key: YARN-6016
URL: https://issues.apache.org/jira/browse/YARN-6016
Project: Hadoop YARN
Issue
Botong Huang created YARN-6093:
--
Summary: Invalid AMRM token exception when RM renew AMRMtoken and
FederationRMFailoverProxyProvider failover
Key: YARN-6093
URL: https://issues.apache.org/jira/browse/YARN-6093
Botong Huang created YARN-6190:
--
Summary: Bug fixes in federation polices
Key: YARN-6190
URL: https://issues.apache.org/jira/browse/YARN-6190
Project: Hadoop YARN
Issue Type: Bug
Botong Huang created YARN-6203:
--
Summary: Occasional test failure in TestWeightedRandomRouterPolicy
Key: YARN-6203
URL: https://issues.apache.org/jira/browse/YARN-6203
Project: Hadoop YARN
Botong Huang created YARN-6213:
--
Summary: Failure handling and retry for performFailover in
RetryInvocationHandler
Key: YARN-6213
URL: https://issues.apache.org/jira/browse/YARN-6213
Project: Hadoop
Botong Huang created YARN-6247:
--
Summary: Add SubClusterResolver into FederationStateStoreFacade
Key: YARN-6247
URL: https://issues.apache.org/jira/browse/YARN-6247
Project: Hadoop YARN
Issue
Botong Huang created YARN-6281:
--
Summary: Cleanup when AMRMProxy fails to initialize a new
interceptor chain
Key: YARN-6281
URL: https://issues.apache.org/jira/browse/YARN-6281
Project: Hadoop YARN
Botong Huang created YARN-6282:
--
Summary: Recreate interceptor chain when different attempt in the
same node in AMRMProxy
Key: YARN-6282
URL: https://issues.apache.org/jira/browse/YARN-6282
Project
Botong Huang created YARN-6370:
--
Summary: Properly handle rack requests for non-active subclusters
in LocalityMulticastAMRMProxyPolicy
Key: YARN-6370
URL: https://issues.apache.org/jira/browse/YARN-6370
Botong Huang created YARN-6404:
--
Summary: Avoid misleading NoClassDefFoundError caused by
ExceptionInInitializerError in FederationStateStoreFacade
Key: YARN-6404
URL: https://issues.apache.org/jira/browse/YARN-6404
Botong Huang created YARN-6511:
--
Summary: Federation Intercepting and propagating AM-RM
communications (part two: secondary subclusters added)
Key: YARN-6511
URL: https://issues.apache.org/jira/browse/YARN-6511
Botong Huang created YARN-6565:
--
Summary: Fix memory leak and finish app trigger in AMRMProxy
Key: YARN-6565
URL: https://issues.apache.org/jira/browse/YARN-6565
Project: Hadoop YARN
Issue Type
Botong Huang created YARN-6640:
--
Summary: AM heartbeat stuck when responseId overflows MAX_INT
Key: YARN-6640
URL: https://issues.apache.org/jira/browse/YARN-6640
Project: Hadoop YARN
Issue
Botong Huang created YARN-6648:
--
Summary: Add FederationStateStore interfaces for Global Policy
Generator
Key: YARN-6648
URL: https://issues.apache.org/jira/browse/YARN-6648
Project: Hadoop YARN
Botong Huang created YARN-:
--
Summary: Fix unit test in TestRouterClientRMService
Key: YARN-
URL: https://issues.apache.org/jira/browse/YARN-
Project: Hadoop YARN
Issue Type: Bug
Botong Huang created YARN-6667:
--
Summary: Handle containerId duplicate without throwing in
Federation Interceptor
Key: YARN-6667
URL: https://issues.apache.org/jira/browse/YARN-6667
Project: Hadoop YARN
Botong Huang created YARN-6704:
--
Summary: Add Federation Interceptor restart when work preserving
NM is enabled
Key: YARN-6704
URL: https://issues.apache.org/jira/browse/YARN-6704
Project: Hadoop YARN
Botong Huang created YARN-6730:
--
Summary: Make sure NM state store is not null consistently
Key: YARN-6730
URL: https://issues.apache.org/jira/browse/YARN-6730
Project: Hadoop YARN
Issue Type
Botong Huang created YARN-6902:
--
Summary: Update SQL server note in License.txt
Key: YARN-6902
URL: https://issues.apache.org/jira/browse/YARN-6902
Project: Hadoop YARN
Issue Type: Task
Botong Huang created YARN-6955:
--
Summary: Concurrent registerAM thread in Federation Interceptor
Key: YARN-6955
URL: https://issues.apache.org/jira/browse/YARN-6955
Project: Hadoop YARN
Issue
Botong Huang created YARN-6962:
--
Summary: Federation interceptor should support full allocate
request/response api
Key: YARN-6962
URL: https://issues.apache.org/jira/browse/YARN-6962
Project: Hadoop
Botong Huang created YARN-7074:
--
Summary: Fix NM state store update comment
Key: YARN-7074
URL: https://issues.apache.org/jira/browse/YARN-7074
Project: Hadoop YARN
Issue Type: Bug
Botong Huang created YARN-7102:
--
Summary: NM heartbeat stuck when responseId overflows MAX_INT
Key: YARN-7102
URL: https://issues.apache.org/jira/browse/YARN-7102
Project: Hadoop YARN
Issue
Botong Huang created YARN-7199:
--
Summary:
TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is
failing in trunk
Key: YARN-7199
URL: https://issues.apache.org/jira/browse/YARN-7199
Botong Huang created YARN-7203:
--
Summary: Add container ExecutionType into ContainerReport
Key: YARN-7203
URL: https://issues.apache.org/jira/browse/YARN-7203
Project: Hadoop YARN
Issue Type
Botong Huang created YARN-7281:
--
Summary: Auto inject AllocationRequestId in
AMRMClient.ContainerRequest when not supplied
Key: YARN-7281
URL: https://issues.apache.org/jira/browse/YARN-7281
Project
Botong Huang created YARN-7317:
--
Summary: Fix overallocation resulted from ceiling in
LocalityMulticastAMRMProxyPolicy
Key: YARN-7317
URL: https://issues.apache.org/jira/browse/YARN-7317
Project: Hadoop
Botong Huang created YARN-7339:
--
Summary: LocalityMulticastAMRMProxyPolicy should handle cancel
request properly
Key: YARN-7339
URL: https://issues.apache.org/jira/browse/YARN-7339
Project: Hadoop YARN
Botong Huang created YARN-7479:
--
Summary: TestContainerManagerSecurity.testContainerManager[Simple]
flaky in trunk
Key: YARN-7479
URL: https://issues.apache.org/jira/browse/YARN-7479
Project: Hadoop
Botong Huang created YARN-7599:
--
Summary: Application cleaner and subcluster cleaner in Global
Policy Generator
Key: YARN-7599
URL: https://issues.apache.org/jira/browse/YARN-7599
Project: Hadoop YARN
Botong Huang created YARN-7630:
--
Summary: Fix AMRMToken handling in AMRMProxy
Key: YARN-7630
URL: https://issues.apache.org/jira/browse/YARN-7630
Project: Hadoop YARN
Issue Type: Bug
Botong Huang created YARN-7631:
--
Summary: ResourceRequest with different Capacity (Resource)
overrides each other in RM
Key: YARN-7631
URL: https://issues.apache.org/jira/browse/YARN-7631
Project
Botong Huang created YARN-7676:
--
Summary: Fix inconsistent priority ordering in Priority and
SchedulerRequestKey
Key: YARN-7676
URL: https://issues.apache.org/jira/browse/YARN-7676
Project: Hadoop YARN
Botong Huang created YARN-7720:
--
Summary: [Federation] Race condition between second app attempt
and UAM heartbeat when first attempt node is down
Key: YARN-7720
URL: https://issues.apache.org/jira/browse/YARN-7720
Botong Huang created YARN-7899:
--
Summary: [AMRMProxy] Stateful FederationInterceptor for pending
requests
Key: YARN-7899
URL: https://issues.apache.org/jira/browse/YARN-7899
Project: Hadoop YARN
Botong Huang created YARN-7900:
--
Summary: [AMRMProxy] AMRMClientRelayer for stateful
FederationInterceptor
Key: YARN-7900
URL: https://issues.apache.org/jira/browse/YARN-7900
Project: Hadoop YARN
Botong Huang created YARN-7918:
--
Summary:
TestAMRMClientPlacementConstraints.testAMRMClientWithPlacementConstraints
failing in trunk
Key: YARN-7918
URL: https://issues.apache.org/jira/browse/YARN-7918
Botong Huang created YARN-8010:
--
Summary: add config in FederationRMFailoverProxy to not bypass
facade cache when failing over
Key: YARN-8010
URL: https://issues.apache.org/jira/browse/YARN-8010
Project
Botong Huang created YARN-8110:
--
Summary: AMRMProxy recover should catch for all throwable retrying
to recover apps
Key: YARN-8110
URL: https://issues.apache.org/jira/browse/YARN-8110
Project: Hadoop
Botong Huang created YARN-8227:
--
Summary: TestPlacementConstraintTransformations is failing in trunk
Key: YARN-8227
URL: https://issues.apache.org/jira/browse/YARN-8227
Project: Hadoop YARN
[
https://issues.apache.org/jira/browse/YARN-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Botong Huang resolved YARN-8334.
Resolution: Fixed
> [GPG] Fix potential connection leak in GPGUt
Botong Huang created YARN-8412:
--
Summary: Move ResourceRequest.clone logic everywhere into a proper
API
Key: YARN-8412
URL: https://issues.apache.org/jira/browse/YARN-8412
Project: Hadoop YARN
Botong Huang created YARN-8433:
--
Summary: TestAMRestart flaky in trunk
Key: YARN-8433
URL: https://issues.apache.org/jira/browse/YARN-8433
Project: Hadoop YARN
Issue Type: Task
Botong Huang created YARN-8451:
--
Summary: Multiple NM heartbeat thread created when a slow NM
resync with RM
Key: YARN-8451
URL: https://issues.apache.org/jira/browse/YARN-8451
Project: Hadoop YARN
Botong Huang created YARN-8481:
--
Summary: AMRMProxyPolicies should accept heartbeat response from
new/unknown subclusters
Key: YARN-8481
URL: https://issues.apache.org/jira/browse/YARN-8481
Project
Botong Huang created YARN-8534:
--
Summary: Add max heap config option for Federation Router and GPG
Key: YARN-8534
URL: https://issues.apache.org/jira/browse/YARN-8534
Project: Hadoop YARN
Issue
Botong Huang created YARN-8536:
--
Summary: Add max heap config option for Federation Router
Key: YARN-8536
URL: https://issues.apache.org/jira/browse/YARN-8536
Project: Hadoop YARN
Issue Type
Botong Huang created YARN-8581:
--
Summary: [AMRMProxy] Add sub-cluster timeout in
LocalityMulticastAMRMProxyPolicy
Key: YARN-8581
URL: https://issues.apache.org/jira/browse/YARN-8581
Project: Hadoop YARN
Botong Huang created YARN-8637:
--
Summary: Add FederationStateStore getAppInfo API for
GlobalPolicyGenerator
Key: YARN-8637
URL: https://issues.apache.org/jira/browse/YARN-8637
Project: Hadoop YARN
Botong Huang created YARN-8658:
--
Summary: Metrics for AMRMClientRelayer inside FederationInterceptor
Key: YARN-8658
URL: https://issues.apache.org/jira/browse/YARN-8658
Project: Hadoop YARN
Botong Huang created YARN-8673:
--
Summary: [AMRMProxy] More robust responseId resync after an YarnRM
master slave switch
Key: YARN-8673
URL: https://issues.apache.org/jira/browse/YARN-8673
Project
Botong Huang created YARN-8696:
--
Summary: FederationInterceptor upgrade: home sub-cluster heartbeat
async
Key: YARN-8696
URL: https://issues.apache.org/jira/browse/YARN-8696
Project: Hadoop YARN
Botong Huang created YARN-8697:
--
Summary: LocalityMulticastAMRMProxyPolicy should fallback to
random sub-cluster when cannot resolve resource
Key: YARN-8697
URL: https://issues.apache.org/jira/browse/YARN-8697
Botong Huang created YARN-8705:
--
Summary: Refactor in preparation for YARN-8696
Key: YARN-8705
URL: https://issues.apache.org/jira/browse/YARN-8705
Project: Hadoop YARN
Issue Type: Task
Botong Huang created YARN-8760:
--
Summary: Fix concurrent re-register due to YarnRM failover in
AMRMClientRelayer
Key: YARN-8760
URL: https://issues.apache.org/jira/browse/YARN-8760
Project: Hadoop YARN
[
https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Botong Huang resolved YARN-7599.
Resolution: Fixed
> [GPG] ApplicationCleaner in Global Policy Genera
Botong Huang created YARN-8862:
--
Summary: [GPG] add Yarn Registry cleanup in ApplicationCleaner
Key: YARN-8862
URL: https://issues.apache.org/jira/browse/YARN-8862
Project: Hadoop YARN
Issue
Botong Huang created YARN-8893:
--
Summary: [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM
client
Key: YARN-8893
URL: https://issues.apache.org/jira/browse/YARN-8893
Project: Hadoop YARN
Botong Huang created YARN-8933:
--
Summary: [AMRMProxy] Fix potential null AvailableResource and
NumClusterNode in allocation response
Key: YARN-8933
URL: https://issues.apache.org/jira/browse/YARN-8933
Botong Huang created YARN-9013:
--
Summary: [GPG] fix order of steps cleaning Registry entries in
ApplicationCleaner
Key: YARN-9013
URL: https://issues.apache.org/jira/browse/YARN-9013
Project: Hadoop
63 matches
Mail list logo