[
https://issues.apache.org/jira/browse/YARN-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210026#comment-16210026
]
Daniel Templeton commented on YARN-7262:
----------------------------------------
bq. most of the other properties in YarnConfiguration don't have Javadocs
And if most of the other properties jumped off a bridge, would you do it, too?
C'mon, give a downstream developer a break.
bq. Added messages to some assert statements
Add some more. At a bare minimum, please make sure you have messages for all
{{assertTrue()}} and {{assertFalse()}} calls.
I'm looking carefully at the ZK code now. Here are my comments:
# I kinda want to suggest that you also use the HIERARCHIES directory just for
consistency. It would also make that _if_ to test for bad nodes simpler.
# Should you invert the _if_ statements in {{loadRMDelegationTokenState()}}?
Right now you're testing first if it starts with the prefix and second if it is
split. The net result is that you quietly ignore nodes called 1, 2, 3, and 4,
even if they're in places where they shouldn't be.
# Why is {{TestZKRMStateStore.getDelegationTokenNode()}} public?
# In {{TestZKRMStateStore.storeUpdateAndVerifyDelegationToken()}},
{{renewDate}} doesn't need the explicit boxing.
> Add a hierarchy into the ZKRMStateStore for delegation token znodes to
> prevent jute buffer overflow
> ---------------------------------------------------------------------------------------------------
>
> Key: YARN-7262
> URL: https://issues.apache.org/jira/browse/YARN-7262
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 2.6.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: YARN-7262.001.patch, YARN-7262.002.patch
>
>
> We've seen users who are running into a problem where the RM is storing so
> many delegation tokens in the {{ZKRMStateStore}} that the _listing_ of those
> znodes is higher than the jute buffer. This is fine during operations, but
> becomes a problem on a fail over because the RM will try to read in all of
> the token znodes (i.e. call {{getChildren}} on the parent znode). This is
> particularly bad because everything appears to be okay, but then if a
> failover occurs you end up with no active RMs.
> There was a similar problem with the Yarn application data that was fixed in
> YARN-2962 by adding a (configurable) hierarchy of znodes so the RM could pull
> subchildren without overflowing the jute buffer (though it's off by default).
> We should add a hierarchy similar to that of YARN-2962, but for the
> delegation token znodes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
