[jira] [Updated] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-503: - Attachment: YARN-503.patch Overhaul and simplify the RM's token renewer. Apps now track their tokens, and tokens track their apps. The 3 background threads - renewer, canceler, delayed canceler - are consolidated into a single timer thread. Tokens are cancelled after all their active apps have completed, unless an app requested the token not be cancelled (compatibility), in which case the renewal is simply stopped. This solves the multiple timers per token, and persistent renewal issues. This change should allow the deprecation and removal of support for not canceling tokens. > DelegationTokens will be renewed forever if multiple jobs share tokens and > the first one sets JOB_CANCEL_DELEGATION_TOKEN to false > -- > > Key: YARN-503 > URL: https://issues.apache.org/jira/browse/YARN-503 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha >Reporter: Siddharth Seth >Assignee: Daryn Sharp > Attachments: YARN-503.patch > > > The first Job/App to register a token is the one which DelegationTokenRenewer > associates with a a specific Token. An attempt to remove/cancel these shared > tokens by subsequent jobs doesn't work - since the JobId will not match. > As a result, Even if subsequent jobs have > MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be > cancelled when those jobs complete. > Tokens will eventually be removed from the RM / JT when the service that > issued them considers them to have expired or via an explicit > cancelDelegationTokens call (not implemented yet in 23). > A side affect of this is that the same delegation token will end up being > renewed multiple times (a separate TimerTask for each job which uses the > token). > DelegationTokenRenewer could maintain a reference count/list of jobIds for > shared tokens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated YARN-503: - Attachment: YARN-503.patch Fixed findbugs issues. Upon review of those issues, I realized I was trying too hard to prevent tight & unlikely race conditions. I simplified the design further which allowed for the removal of virtually all the synchronization which could lead to tricky deadlocks. Updated tests to ensure apps and/or tokens don't leak. > DelegationTokens will be renewed forever if multiple jobs share tokens and > the first one sets JOB_CANCEL_DELEGATION_TOKEN to false > -- > > Key: YARN-503 > URL: https://issues.apache.org/jira/browse/YARN-503 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha >Reporter: Siddharth Seth >Assignee: Daryn Sharp > Attachments: YARN-503.patch, YARN-503.patch > > > The first Job/App to register a token is the one which DelegationTokenRenewer > associates with a a specific Token. An attempt to remove/cancel these shared > tokens by subsequent jobs doesn't work - since the JobId will not match. > As a result, Even if subsequent jobs have > MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be > cancelled when those jobs complete. > Tokens will eventually be removed from the RM / JT when the service that > issued them considers them to have expired or via an explicit > cancelDelegationTokens call (not implemented yet in 23). > A side affect of this is that the same delegation token will end up being > renewed multiple times (a separate TimerTask for each job which uses the > token). > DelegationTokenRenewer could maintain a reference count/list of jobIds for > shared tokens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-503) DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false
[ https://issues.apache.org/jira/browse/YARN-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-503: - Issue Type: Sub-task (was: Bug) Parent: YARN-47 > DelegationTokens will be renewed forever if multiple jobs share tokens and > the first one sets JOB_CANCEL_DELEGATION_TOKEN to false > -- > > Key: YARN-503 > URL: https://issues.apache.org/jira/browse/YARN-503 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha >Reporter: Siddharth Seth >Assignee: Daryn Sharp > Attachments: YARN-503.patch, YARN-503.patch > > > The first Job/App to register a token is the one which DelegationTokenRenewer > associates with a a specific Token. An attempt to remove/cancel these shared > tokens by subsequent jobs doesn't work - since the JobId will not match. > As a result, Even if subsequent jobs have > MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be > cancelled when those jobs complete. > Tokens will eventually be removed from the RM / JT when the service that > issued them considers them to have expired or via an explicit > cancelDelegationTokens call (not implemented yet in 23). > A side affect of this is that the same delegation token will end up being > renewed multiple times (a separate TimerTask for each job which uses the > token). > DelegationTokenRenewer could maintain a reference count/list of jobIds for > shared tokens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
