[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721387#comment-14721387 ] Hudson commented on HDFS-8155: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2270 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2270/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721382#comment-14721382 ] Hudson commented on HDFS-8155: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2251 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2251/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721369#comment-14721369 ] Hudson commented on HDFS-8155: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #327 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/327/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721360#comment-14721360 ] Hudson commented on HDFS-8155: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1054 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1054/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721357#comment-14721357 ] Hudson commented on HDFS-8155: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #312 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/312/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721353#comment-14721353 ] Hudson commented on HDFS-8155: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #321 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/321/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721345#comment-14721345 ] Hudson commented on HDFS-8155: -- FAILURE: Integrated in Hadoop-trunk-Commit #8370 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8370/]) HDFS-8155. Support OAuth2 in WebHDFS. (jghoman: rev 837fb75e8e03b2f016bcea2f4605106a5022491c) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestAccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestRefreshTokenTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSOAuth2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/oauth2/TestClientCredentialTimeBasedTokenRefresher.java * hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/Utils.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2Constants.java > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8155) Support OAuth2 in WebHDFS
[ https://issues.apache.org/jira/browse/HDFS-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8155: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8 Status: Resolved (was: Patch Available) The test failures, numerous though they are, are unrelated and happening to other JIRAS as well (HDFS-8983). Having received the rare double-Chris(D|N) +1, I've committed this to trunk and branch-2. Resolving. > Support OAuth2 in WebHDFS > - > > Key: HDFS-8155 > URL: https://issues.apache.org/jira/browse/HDFS-8155 > Project: Hadoop HDFS > Issue Type: New Feature > Components: webhdfs >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 2.8 > > Attachments: HDFS-8155-1.patch, HDFS-8155.002.patch, > HDFS-8155.003.patch, HDFS-8155.004.patch, HDFS-8155.005.patch, > HDFS-8155.006.patch > > > WebHDFS should be able to accept OAuth2 credentials. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721258#comment-14721258 ] Hudson commented on HDFS-8983: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2250 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2250/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721254#comment-14721254 ] Hudson commented on HDFS-8983: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #311 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/311/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721243#comment-14721243 ] Hudson commented on HDFS-8983: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2269 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2269/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721231#comment-14721231 ] Hudson commented on HDFS-8983: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #320 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/320/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721229#comment-14721229 ] Hudson commented on HDFS-8983: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1053 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1053/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721222#comment-14721222 ] Hudson commented on HDFS-8983: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #326 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/326/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721158#comment-14721158 ] Hudson commented on HDFS-8983: -- FAILURE: Integrated in Hadoop-trunk-Commit #8369 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8369/]) HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) (arp: rev bdbe53c676dd4ff135ea2f64d3b9193fe43d7c8e) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProtectedDirectories.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java * hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirDeleteOp.java > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8983) NameNode support for protected directories
[ https://issues.apache.org/jira/browse/HDFS-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8983: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Target Version/s: (was: 2.8.0) Status: Resolved (was: Patch Available) Thanks [~jnp]! Committed to trunk and branch-2. The test failures were unrelated. > NameNode support for protected directories > -- > > Key: HDFS-8983 > URL: https://issues.apache.org/jira/browse/HDFS-8983 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Fix For: 2.8.0 > > Attachments: HDFS-8393.01.patch, HDFS-8393.02.patch, > HDFS-8983.03.patch, HDFS-8983.04.patch > > > To protect important system directories from inadvertent deletion (e.g. > /Users) the NameNode can allow marking directories as _protected_. Such > directories cannot be deleted unless they are empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8990) Move RemoteBlockReader to hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721064#comment-14721064
]
Hadoop QA commented on HDFS-8990:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 49s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 57s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 13s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 31s | The applied patch generated
61 new checkstyle issues (total was 0, now 61). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 40s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 33s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 23s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 190m 48s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 30s | Tests passed in
hadoop-hdfs-client. |
| | | 242m 25s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestClientBlockVerification |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12753096/HDFS-8990.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e2c9b28 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12212/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12212/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12212/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12212/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12212/console |
This message was automatically generated.
> Move RemoteBlockReader to hdfs-client module
>
>
> Key: HDFS-8990
> URL: https://issues.apache.org/jira/browse/HDFS-8990
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: build
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-8990.000.patch, HDFS-8990.001.patch
>
>
> This jira tracks the effort of moving the {{RemoteBlockReader}} class into
> the {{hdfs-client}} module. {{BlockReader}} interface and
> {{BlockReaderLocal}} class were moved to {{hadoop-hdfs-client}} module in
> jira [HDFS-8925|https://issues.apache.org/jira/browse/HDFS-8925].
> The extant checkstyle warnings can be fixed in
> [HDFS-8979|https://issues.apache.org/jira/browse/HDFS-8979]. While we need to
> replace the _log4j_ with _slf4j_ in this patch, we track the effort of
> removing the guards when calling LOG.debug() and LOG.trace() in jira
> [HDFS-8971|https://issues.apache.org/jira/browse/HDFS-8971].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8985) Restarted namenode suffer from block report storm
[ https://issues.apache.org/jira/browse/HDFS-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721058#comment-14721058 ] Zhihua Deng commented on HDFS-8985: --- yes, thank you. > Restarted namenode suffer from block report storm > - > > Key: HDFS-8985 > URL: https://issues.apache.org/jira/browse/HDFS-8985 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 2.4.1 >Reporter: Zhihua Deng >Priority: Trivial > Labels: test > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8964) Provide max TxId when validating in-progress edit log files
[
https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721056#comment-14721056
]
Hadoop QA commented on HDFS-8964:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 24s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 5s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 13s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 20s | The applied patch generated 2
new checkstyle issues (total was 162, now 162). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 2 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 29s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 32s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 19s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 190m 38s | Tests failed in hadoop-hdfs. |
| | | 235m 58s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestDFSRollback |
| | hadoop.hdfs.qjournal.server.TestJournal |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.qjournal.TestSecureNNWithQJM |
| | hadoop.hdfs.server.namenode.TestFileJournalManager |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12753121/HDFS-8964.02.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e2c9b28 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12211/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12211/artifact/patchprocess/whitespace.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12211/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12211/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12211/console |
This message was automatically generated.
> Provide max TxId when validating in-progress edit log files
> ---
>
> Key: HDFS-8964
> URL: https://issues.apache.org/jira/browse/HDFS-8964
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: journal-node, namenode
>Affects Versions: 2.7.1
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
> Attachments: HDFS-8964.00.patch, HDFS-8964.01.patch,
> HDFS-8964.02.patch
>
>
> NN/JN validates in-progress edit log files in multiple scenarios, via
> {{EditLogFile#validateLog}}. The method scans through the edit log file to
> find the last transaction ID.
> However, an in-progress edit log file could be actively written to, which
> creates a race condition and causes incorrect data to be read (and later we
> attempt to interpret the data as ops).
> Currently {{validateLog}} is used in 3 places:
> # NN {{getEditsFromTxid}}
> # JN {{getEditLogManifest}}
> # NN/JN {{recoverUnfinalizedSegments}}
> In the first two scenarios we should provide a maximum TxId to validate in
> the in-progress file. The 3rd scenario won't cause a race condition because
> only non-current in-progress edit log files are validated.
> {{validateLog}} is actually only used with in-progress files, and could use a
> better name and Javadoc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8985) Restarted namenode suffer from block report storm
[ https://issues.apache.org/jira/browse/HDFS-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721057#comment-14721057 ] Zhihua Deng commented on HDFS-8985: --- yes, thank you. > Restarted namenode suffer from block report storm > - > > Key: HDFS-8985 > URL: https://issues.apache.org/jira/browse/HDFS-8985 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 2.4.1 >Reporter: Zhihua Deng >Priority: Trivial > Labels: test > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721030#comment-14721030
]
He Xiaoqiao commented on HDFS-8992:
---
thanks [~brahmareddy] for ur comment. i will close this ticket.
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Xiaoqiao resolved HDFS-8992.
---
Resolution: Duplicate
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721025#comment-14721025
]
Brahma Reddy Battula commented on HDFS-8992:
can you please take look at HDFS-6364.seems to be this is similar to that.
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8993) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721022#comment-14721022
]
Brahma Reddy Battula commented on HDFS-8993:
Just made duplicate of HDFS-8992, I did it for you..
> Balancer throws NPE
> ---
>
> Key: HDFS-8993
> URL: https://issues.apache.org/jira/browse/HDFS-8993
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:title=Balancer.java}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HDFS-8993) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula resolved HDFS-8993.
Resolution: Duplicate
> Balancer throws NPE
> ---
>
> Key: HDFS-8993
> URL: https://issues.apache.org/jira/browse/HDFS-8993
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:title=Balancer.java}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Xiaoqiao updated HDFS-8992:
--
Description:
Balancer may throw NPE because the following {{if Statements}} at line 709 of
Balancer.java
{code:java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
was:
Balancer may throw NPE because the following {{if Statements}}
{code:java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8993) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721021#comment-14721021
]
He Xiaoqiao commented on HDFS-8993:
---
sorry to create to the same ticket since first creation not complete... BTW,
how to delete this ticket?
> Balancer throws NPE
> ---
>
> Key: HDFS-8993
> URL: https://issues.apache.org/jira/browse/HDFS-8993
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:title=Balancer.java}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8993) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721019#comment-14721019
]
Brahma Reddy Battula commented on HDFS-8993:
HI [~hexiaoqiao] , this is dupe of HDFS-8992..?
> Balancer throws NPE
> ---
>
> Key: HDFS-8993
> URL: https://issues.apache.org/jira/browse/HDFS-8993
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:title=Balancer.java}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721018#comment-14721018
]
He Xiaoqiao commented on HDFS-8992:
---
Actually, it can be fixed only add condition at {{if Statements}}
{code}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null && null != d) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}}
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721016#comment-14721016
]
He Xiaoqiao commented on HDFS-8992:
---
Thanks Tsz Wo Nicholas Sze for your comments, there is only one line of stack
trace:
{code}
2015-06-29 14:06:35,280 WARN org.apache.hadoop.hdfs.server.balancer.Balancer:
Dispatcher thread failed
java.lang.NullPointerException
{code}
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}}
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721017#comment-14721017
]
He Xiaoqiao commented on HDFS-8992:
---
Thanks Tsz Wo Nicholas Sze for your comments, there is only one line of stack
trace:
{code}
2015-06-29 14:06:35,280 WARN org.apache.hadoop.hdfs.server.balancer.Balancer:
Dispatcher thread failed
java.lang.NullPointerException
{code}
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}}
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721015#comment-14721015
]
He Xiaoqiao commented on HDFS-8992:
---
Thanks Tsz Wo Nicholas Sze for your comments, there is only one line of stack
trace:
{code}
2015-06-29 14:06:35,280 WARN org.apache.hadoop.hdfs.server.balancer.Balancer:
Dispatcher thread failed
java.lang.NullPointerException
{code}
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}}
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8993) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Xiaoqiao updated HDFS-8993:
--
Description:
Balancer may throw NPE because the following {{if Statements}} at line 709 of
Balancer.java
{code:title=Balancer.java}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
was:
Balancer may throw NPE because the following {{if Statements}}
{code:title=Balancer.java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
> Balancer throws NPE
> ---
>
> Key: HDFS-8993
> URL: https://issues.apache.org/jira/browse/HDFS-8993
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}} at line 709 of
> Balancer.java
> {code:title=Balancer.java}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-8994) Fair HDFS bandwidth throttling on DataNode
Yong Zhang created HDFS-8994: Summary: Fair HDFS bandwidth throttling on DataNode Key: HDFS-8994 URL: https://issues.apache.org/jira/browse/HDFS-8994 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Reporter: Yong Zhang Assignee: Yong Zhang HADOOP-9640 present a fair RPC call, this JIRA will follow this fair mode for network IO on block read and write. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8993) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Xiaoqiao updated HDFS-8993:
--
Description:
Balancer may throw NPE because the following {{if Statements}}
{code:title=Balancer.java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
was:
Balancer may throw NPE because the following {{if Statements}}
{code:java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
> Balancer throws NPE
> ---
>
> Key: HDFS-8993
> URL: https://issues.apache.org/jira/browse/HDFS-8993
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}}
> {code:title=Balancer.java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8992) Balancer throws NPE
[
https://issues.apache.org/jira/browse/HDFS-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721012#comment-14721012
]
Tsz Wo Nicholas Sze commented on HDFS-8992:
---
Could you post the stack trace?
> Balancer throws NPE
> ---
>
> Key: HDFS-8992
> URL: https://issues.apache.org/jira/browse/HDFS-8992
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
>Affects Versions: 2.4.1
>Reporter: He Xiaoqiao
>
> Balancer may throw NPE because the following {{if Statements}}
> {code:java|firstline=705}
> synchronized (block) {
> // update locations
> for (String datanodeUuid : blk.getDatanodeUuids()) {
> final BalancerDatanode d = datanodeMap.get(datanodeUuid);
> if (datanode != null) { // not an unknown datanode
> block.addLocation(d);
> }
> }
> }
> {code}
> Before moving block, Balancer divides into two step to get all DNs info &
> some blocks of DN from NN. Regarding one DN commission after
> {{getDatanodeReport}} then {{getBlockList}} and one block's target is above
> DN just right, It will throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-8993) Balancer throws NPE
He Xiaoqiao created HDFS-8993:
-
Summary: Balancer throws NPE
Key: HDFS-8993
URL: https://issues.apache.org/jira/browse/HDFS-8993
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer & mover
Affects Versions: 2.4.1
Reporter: He Xiaoqiao
Balancer may throw NPE because the following {{if Statements}}
{code:java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails
[
https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721009#comment-14721009
]
Walter Su commented on HDFS-8704:
-
Thanks [~libo-intel] for the hard work. I tried {{runTest(length)}} with
{{BLOCK_GROUP_SIZE*2}}. And no good luck. Could you take a look? I'll look
again once it get addressed.
> Erasure Coding: client fails to write large file when one datanode fails
>
>
> Key: HDFS-8704
> URL: https://issues.apache.org/jira/browse/HDFS-8704
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Li Bo
>Assignee: Li Bo
> Attachments: HDFS-8704-000.patch, HDFS-8704-HDFS-7285-002.patch,
> HDFS-8704-HDFS-7285-003.patch, HDFS-8704-HDFS-7285-004.patch,
> HDFS-8704-HDFS-7285-005.patch, HDFS-8704-HDFS-7285-006.patch
>
>
> I test current code on a 5-node cluster using RS(3,2). When a datanode is
> corrupt, client succeeds to write a file smaller than a block group but fails
> to write a large one. {{TestDFSStripeOutputStreamWithFailure}} only tests
> files smaller than a block group, this jira will add more test situations.
> A streamer may encounter some bad datanodes when writing blocks allocated to
> it. When it fails to connect datanode or send a packet, the streamer needs to
> prepare for the next block. First it removes the packets of current block
> from its data queue. If the first packet of next block has already been in
> the data queue, the streamer will reset its state and start to wait for the
> next block allocated for it; otherwise it will just wait for the first packet
> of next block. The streamer will check periodically if it is asked to
> terminate during its waiting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-8992) Balancer throws NPE
He Xiaoqiao created HDFS-8992:
-
Summary: Balancer throws NPE
Key: HDFS-8992
URL: https://issues.apache.org/jira/browse/HDFS-8992
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer & mover
Affects Versions: 2.4.1
Reporter: He Xiaoqiao
Balancer may throw NPE because the following {{if Statements}}
{code:java|firstline=705}
synchronized (block) {
// update locations
for (String datanodeUuid : blk.getDatanodeUuids()) {
final BalancerDatanode d = datanodeMap.get(datanodeUuid);
if (datanode != null) { // not an unknown datanode
block.addLocation(d);
}
}
}
{code}
Before moving block, Balancer divides into two step to get all DNs info & some
blocks of DN from NN. Regarding one DN commission after {{getDatanodeReport}}
then {{getBlockList}} and one block's target is above DN just right, It will
throw NPE when dispatcher.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8990) Move RemoteBlockReader to hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-8990:
Status: Patch Available (was: Open)
> Move RemoteBlockReader to hdfs-client module
>
>
> Key: HDFS-8990
> URL: https://issues.apache.org/jira/browse/HDFS-8990
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: build
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-8990.000.patch, HDFS-8990.001.patch
>
>
> This jira tracks the effort of moving the {{RemoteBlockReader}} class into
> the {{hdfs-client}} module. {{BlockReader}} interface and
> {{BlockReaderLocal}} class were moved to {{hadoop-hdfs-client}} module in
> jira [HDFS-8925|https://issues.apache.org/jira/browse/HDFS-8925].
> The extant checkstyle warnings can be fixed in
> [HDFS-8979|https://issues.apache.org/jira/browse/HDFS-8979]. While we need to
> replace the _log4j_ with _slf4j_ in this patch, we track the effort of
> removing the guards when calling LOG.debug() and LOG.trace() in jira
> [HDFS-8971|https://issues.apache.org/jira/browse/HDFS-8971].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-8991) Provide information on BPOfferService in DN JMX and UI
Haohui Mai created HDFS-8991: Summary: Provide information on BPOfferService in DN JMX and UI Key: HDFS-8991 URL: https://issues.apache.org/jira/browse/HDFS-8991 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Mingliang Liu In cases like HDFS-7714 where the BPOfferService thread is missing, which require nontrivial effort to debug that which NN that the DN thinks it active / standby. It would make sense to make the information more accessible through JMX or Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HDFS-8985) Restarted namenode suffer from block report storm
[ https://issues.apache.org/jira/browse/HDFS-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8985. - Resolution: Invalid > Restarted namenode suffer from block report storm > - > > Key: HDFS-8985 > URL: https://issues.apache.org/jira/browse/HDFS-8985 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 2.4.1 >Reporter: Zhihua Deng >Priority: Trivial > Labels: test > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
