[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853967#action_12853967 ] Hudson commented on HDFS-984: - Integrated in Hadoop-Hdfs-trunk #275 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/]) > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Fix For: 0.22.0 > > Attachments: HDFS-984-0_20.4.patch, HDFS-984.10.patch, > HDFS-984.11.patch, HDFS-984.12.patch, HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853875#action_12853875 ] Hudson commented on HDFS-984: - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #302 (See [http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/302/]) > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Fix For: 0.22.0 > > Attachments: HDFS-984-0_20.4.patch, HDFS-984.10.patch, > HDFS-984.11.patch, HDFS-984.12.patch, HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853668#action_12853668 ] Hudson commented on HDFS-984: - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See [http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/]) > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Fix For: 0.22.0 > > Attachments: HDFS-984-0_20.4.patch, HDFS-984.10.patch, > HDFS-984.11.patch, HDFS-984.12.patch, HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12840041#action_12840041 ] Hudson commented on HDFS-984: - Integrated in Hadoop-Mapreduce-trunk-Commit #252 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/252/]) > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Fix For: 0.22.0 > > Attachments: HDFS-984-0_20.4.patch, HDFS-984.10.patch, > HDFS-984.11.patch, HDFS-984.12.patch, HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838692#action_12838692 ] Hudson commented on HDFS-984: - Integrated in Hadoop-Hdfs-trunk-Commit #204 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/204/]) . Persistent delegation tokens. Contributed by Jitendra Pandey. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Fix For: 0.22.0 > > Attachments: HDFS-984.10.patch, HDFS-984.11.patch, HDFS-984.12.patch, > HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838538#action_12838538 ] Jitendra Nath Pandey commented on HDFS-984: --- Other tests also run fine. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.10.patch, HDFS-984.11.patch, HDFS-984.12.patch, > HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838247#action_12838247 ] Konstantin Shvachko commented on HDFS-984: -- Please run ant test-core. run-test-hdfs target does not cover some tests like fi tests. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.10.patch, HDFS-984.11.patch, HDFS-984.12.patch, > HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838144#action_12838144 ] Jitendra Nath Pandey commented on HDFS-984: --- ant run-test-hdfs ran successfully. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.10.patch, HDFS-984.11.patch, HDFS-984.12.patch, > HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838143#action_12838143 ] Jitendra Nath Pandey commented on HDFS-984: --- test patch results- [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 15 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.10.patch, HDFS-984.11.patch, HDFS-984.12.patch, > HDFS-984.14.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838012#action_12838012 ] Konstantin Shvachko commented on HDFS-984: -- - AbstractDelegationTokenSecretManager.renewToken() should be synchronized - Same with cancelToken() - imports in DelegationTokenSecretManager and Tests +1 other than that. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.10.patch, HDFS-984.11.patch, HDFS-984.12.patch, > HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836998#action_12836998 ] Konstantin Shvachko commented on HDFS-984: -- Including comments for both issue this one and HADOOP-6573. # Interchanging names logGetDelegationToken() and logAddDelegationToken() are confusing - should be consistent either all Get or all Add. Should OP_GET_DELEGATION_TOKEN be OP_ADD_DELEGATION_TOKEN? # Reuse the instance of DelegationTokenIdentifier in loadFSEdits() # loadFSEdits() should log new token ops in debug mode. # I think we should throw an exception if the same token is added (not renewed) twice. # AbstractDelegationTokenSecretManager uses 2 locks (this and currentTokens) - probably an attempt to do a read/write lock. You don't use them consistently. currentTokens.get() returns a reference to the value, not a copy. So when you access the value fields outside the lock they may be changing by other threads. It would be better to have just one lock (this). # No need to make isInSafeMode() public. # Import warnings in FSNamesystem, DelegationTokenSecretManager # @Override // # public methods and classes must have JavaDoc. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.10.patch, HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835644#action_12835644 ] Jitendra Nath Pandey commented on HDFS-984: --- > Can you please explain the use-case why these need to be persisted? Delegation tokens are used by tasks to authenticate with namenode. If namenode restarts and looses information for previously issued delegation tokens, the running tasks will not be able to connect to the namenode. The scope of this jira is limited to delegation tokens issued by namenode. The delgation tokens in MR issued by JT are not being persisted. > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > Attachments: HDFS-984.7.patch > > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-984) Delegation Tokens should be persisted in Namenode
[ https://issues.apache.org/jira/browse/HDFS-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835128#action_12835128 ] dhruba borthakur commented on HDFS-984: --- Can you please explain the use-case why these need to be persisted? > Delegation Tokens should be persisted in Namenode > - > > Key: HDFS-984 > URL: https://issues.apache.org/jira/browse/HDFS-984 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Jitendra Nath Pandey >Assignee: Jitendra Nath Pandey > > The Delegation tokens should be persisted in the FsImage and EditLogs so that > they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
