[jira] [Updated] (HDDS-3487) Ozone start fails with NullPointerException in TLS enabled cluster

2020-04-24 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated HDDS-3487: Status: Patch Available (was: Open) > Ozone start fails with NullPointerException in TLS

[jira] [Updated] (HDDS-3487) Ozone start fails with NullPointerException in TLS enabled cluster

2020-04-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3487: - Labels: pull-request-available (was: ) > Ozone start fails with NullPointerException in TLS

[GitHub] [hadoop-ozone] avijayanhwx opened a new pull request #871: HDDS-3487. Ozone start fails with NullPointerException in TLS enabled…

2020-04-24 Thread GitBox
avijayanhwx opened a new pull request #871: URL: https://github.com/apache/hadoop-ozone/pull/871 … cluster. ## What changes were proposed in this pull request? **SCM start failed with exception java.lang.NullPointerException at

[jira] [Updated] (HDDS-2765) security/SecureOzone.md translation

2020-04-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-2765: - Labels: pull-request-available (was: ) > security/SecureOzone.md translation >

[GitHub] [hadoop-ozone] iamabug opened a new pull request #870: HDDS-2765. security/SecureOzone.md translation

2020-04-24 Thread GitBox
iamabug opened a new pull request #870: URL: https://github.com/apache/hadoop-ozone/pull/870 ## What changes were proposed in this pull request? doc ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-2765 ## How was this patch

[GitHub] [hadoop-ozone] hanishakoneru commented on pull request #859: HDDS-3465. OM Failover retry happens too quickly when new leader suggested and retrying on same OM.

2020-04-24 Thread GitBox
hanishakoneru commented on pull request #859: URL: https://github.com/apache/hadoop-ozone/pull/859#issuecomment-619286092 Thanks Uma for working on this. LGTM. +1. This is an automated message from the Apache Git

[jira] [Assigned] (HDDS-3487) Ozone start fails with NullPointerException in TLS enabled cluster

2020-04-24 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan reassigned HDDS-3487: --- Assignee: Aravindan Vijayan > Ozone start fails with NullPointerException in TLS

[jira] [Created] (HDDS-3487) Ozone start fails with NullPointerException in TLS enabled cluster

2020-04-24 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-3487: --- Summary: Ozone start fails with NullPointerException in TLS enabled cluster Key: HDDS-3487 URL: https://issues.apache.org/jira/browse/HDDS-3487 Project: Hadoop

[jira] [Updated] (HDDS-2994) API Docs for Recon - Swagger integration.

2020-04-24 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-2994: - Parent: (was: HDDS-1996) Issue Type: Task (was: Sub-task) >

[jira] [Updated] (HDDS-3182) Add unit tests to Recon Frontend

2020-04-24 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-3182: - Parent: (was: HDDS-1996) Issue Type: Bug (was: Sub-task) >

[jira] [Updated] (HDDS-3182) Add unit tests to Recon Frontend

2020-04-24 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-3182: - Target Version/s: 0.6.0 (was: 0.5.0) > Add unit tests to Recon Frontend

[jira] [Updated] (HDDS-3182) Add unit tests to Recon Frontend

2020-04-24 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-3182: - Issue Type: Task (was: Bug) > Add unit tests to Recon Frontend >

[jira] [Created] (HDDS-3486) Recon cannot track missing containers that were created and went missing while it is down.

2020-04-24 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-3486: --- Summary: Recon cannot track missing containers that were created and went missing while it is down. Key: HDDS-3486 URL: https://issues.apache.org/jira/browse/HDDS-3486

[jira] [Assigned] (HDDS-3101) Depend on lightweight ConfigurationSource interface instead of Hadoop Configuration

2020-04-24 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan reassigned HDDS-3101: --- Assignee: Marton Elek (was: Aravindan Vijayan) > Depend on lightweight

[jira] [Assigned] (HDDS-3101) Depend on lightweight ConfigurationSource interface instead of Hadoop Configuration

2020-04-24 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan reassigned HDDS-3101: --- Assignee: Aravindan Vijayan (was: Marton Elek) > Depend on lightweight

[GitHub] [hadoop-ozone] cku328 commented on pull request #812: HDDS-3161. Block illegal characters when creating keys.

2020-04-24 Thread GitBox
cku328 commented on pull request #812: URL: https://github.com/apache/hadoop-ozone/pull/812#issuecomment-619254557 Thanks @adoroszlai for the review. In the new patch, this keyname check is optional and it can be enabled by Ozone global config (default is false). In

[GitHub] [hadoop-ozone] cku328 commented on a change in pull request #812: HDDS-3161. Block illegal characters when creating keys.

2020-04-24 Thread GitBox
cku328 commented on a change in pull request #812: URL: https://github.com/apache/hadoop-ozone/pull/812#discussion_r414884429 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java ## @@ -328,5 +328,19 @@ private OzoneConsts() { public

[GitHub] [hadoop-ozone] cku328 commented on a change in pull request #812: HDDS-3161. Block illegal characters when creating keys.

2020-04-24 Thread GitBox
cku328 commented on a change in pull request #812: URL: https://github.com/apache/hadoop-ozone/pull/812#discussion_r414884358 ## File path: hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java ## @@ -196,6 +196,27 @@ public static void

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #839: HDDS-3411. Switch Recon SQL DB to Derby.

2020-04-24 Thread GitBox
avijayanhwx commented on a change in pull request #839: URL: https://github.com/apache/hadoop-ozone/pull/839#discussion_r414869275 ## File path: hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/codegen/ReconSqlDbConfig.java ## @@ -0,0 +1,50 @@ +/* + * Licensed

[GitHub] [hadoop-ozone] hanishakoneru commented on a change in pull request #859: HDDS-3465. OM Failover retry happens too quickly when new leader suggested and retrying on same OM.

2020-04-24 Thread GitBox
hanishakoneru commented on a change in pull request #859: URL: https://github.com/apache/hadoop-ozone/pull/859#discussion_r414831758 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/ha/OMFailoverProxyProvider.java ## @@ -318,9 +318,12 @@ synchronized

[GitHub] [hadoop-ozone] elek commented on pull request #863: HDDS-3461. Add acceptance test to smoketest CSI service startup

2020-04-24 Thread GitBox
elek commented on pull request #863: URL: https://github.com/apache/hadoop-ozone/pull/863#issuecomment-619215210 /pending Failed in https://github.com/apache/hadoop-ozone/actions/runs/86963299 This is an automated message

[GitHub] [hadoop-ozone] elek edited a comment on pull request #863: HDDS-3461. Add acceptance test to smoketest CSI service startup

2020-04-24 Thread GitBox
elek edited a comment on pull request #863: URL: https://github.com/apache/hadoop-ozone/pull/863#issuecomment-619214890 Failed in https://github.com/apache/hadoop-ozone/actions/runs/86963299. Reverted immediately. This is

[jira] [Created] (HDDS-3485) Use typesafe ContainerID instead of long value

2020-04-24 Thread Nanda kumar (Jira)
Nanda kumar created HDDS-3485: - Summary: Use typesafe ContainerID instead of long value Key: HDDS-3485 URL: https://issues.apache.org/jira/browse/HDDS-3485 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-3373) Intermittent failure in TestDnRatisLogParser and TestOMRatisLogParser

2020-04-24 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3373: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Intermittent failure in

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #850: HDDS-3222. Add integration test for Recon FSCK.

2020-04-24 Thread GitBox
avijayanhwx commented on pull request #850: URL: https://github.com/apache/hadoop-ozone/pull/850#issuecomment-619152838 Thanks for the review @swagle & @adoroszlai. This is an automated message from the Apache Git Service.

[jira] [Resolved] (HDDS-3222) Add integration test for Recon FSCK.

2020-04-24 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan resolved HDDS-3222. - Resolution: Fixed Merged the PR. > Add integration test for Recon FSCK. >

[GitHub] [hadoop-ozone] anuengineer commented on a change in pull request #839: HDDS-3411. Switch Recon SQL DB to Derby.

2020-04-24 Thread GitBox
anuengineer commented on a change in pull request #839: URL: https://github.com/apache/hadoop-ozone/pull/839#discussion_r414704839 ## File path: hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/codegen/ReconSqlDbConfig.java ## @@ -0,0 +1,50 @@ +/* + * Licensed

[GitHub] [hadoop-ozone] anuengineer commented on a change in pull request #839: HDDS-3411. Switch Recon SQL DB to Derby.

2020-04-24 Thread GitBox
anuengineer commented on a change in pull request #839: URL: https://github.com/apache/hadoop-ozone/pull/839#discussion_r414704839 ## File path: hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/codegen/ReconSqlDbConfig.java ## @@ -0,0 +1,50 @@ +/* + * Licensed

[GitHub] [hadoop-ozone] elek commented on pull request #848: HDDS-3418. Enable TestNodeFailure test cases.

2020-04-24 Thread GitBox
elek commented on pull request #848: URL: https://github.com/apache/hadoop-ozone/pull/848#issuecomment-619055035 #b7bf626 failure is related to the `TestOMRatisLogParser.testRatisLogParsing` not related to any previous flakiness

[GitHub] [hadoop-ozone] bshashikant commented on pull request #399: HDDS-2424. Add the recover-trash command server side handling.

2020-04-24 Thread GitBox
bshashikant commented on pull request #399: URL: https://github.com/apache/hadoop-ozone/pull/399#issuecomment-619039604 @bharatviswa504 , can you please have a look? This is an automated message from the Apache Git Service.

[jira] [Updated] (HDDS-2371) Print out the ozone version during the startup instead of hadoop version

2020-04-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-2371: - Labels: newbie pull-request-available (was: newbie) > Print out the ozone version during the

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #869: HDDS-2371. Print Ozone version during startup instead of Hadoop version

2020-04-24 Thread GitBox
adoroszlai opened a new pull request #869: URL: https://github.com/apache/hadoop-ozone/pull/869 ## What changes were proposed in this pull request? 1. Adapt startup/shutdown related code from Hadoop to ensure Ozone version is logged instead of Hadoop version. 2. Add

[jira] [Commented] (HDDS-3354) OM HA replay optimization

2020-04-24 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091559#comment-17091559 ] Marton Elek commented on HDDS-3354: --- Thanks to wok on this Bharat, I have a few questions to understand

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #867: HDDS-3373. Intermittent failure in TestOMRatisLogParser

2020-04-24 Thread GitBox
adoroszlai commented on pull request #867: URL: https://github.com/apache/hadoop-ozone/pull/867#issuecomment-618957536 Thanks @elek for the review. I'm merging it. This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] captainzmc opened a new pull request #868: HDDS-3373. Fix ozonefs put and mkdir KEY_NOT_FOUND issue when ACL enable

2020-04-24 Thread GitBox
captainzmc opened a new pull request #868: URL: https://github.com/apache/hadoop-ozone/pull/868 ## What changes were proposed in this pull request? When we put file using O3FS, FileSystem will check if it exists first. If the key not exist, checkAccess will report KEY_NOT_FOUND

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #740: HDDS-3307. Improve write efficiency by avoiding reverse DNS lookup

2020-04-24 Thread GitBox
runzhiwang commented on pull request #740: URL: https://github.com/apache/hadoop-ozone/pull/740#issuecomment-618927551 @elek I close it temporarily, because I‘m not sure whether hostname will be used or not in future. This

[jira] [Comment Edited] (HDDS-3457) When ACL enable, use ozonefs put key will get OMException: Key not found, checkAccess failed.

2020-04-24 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091410#comment-17091410 ] mingchao zhao edited comment on HDDS-3457 at 4/24/20, 9:58 AM: --- *After acl

[jira] [Comment Edited] (HDDS-3457) When ACL enable, use ozonefs put key will get OMException: Key not found, checkAccess failed.

2020-04-24 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091410#comment-17091410 ] mingchao zhao edited comment on HDDS-3457 at 4/24/20, 9:57 AM: --- *After acl

[jira] [Comment Edited] (HDDS-3457) When ACL enable, use ozonefs put key will get OMException: Key not found, checkAccess failed.

2020-04-24 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091410#comment-17091410 ] mingchao zhao edited comment on HDDS-3457 at 4/24/20, 9:56 AM: --- *After acl

[jira] [Comment Edited] (HDDS-3457) When ACL enable, use ozonefs put key will get OMException: Key not found, checkAccess failed.

2020-04-24 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091410#comment-17091410 ] mingchao zhao edited comment on HDDS-3457 at 4/24/20, 9:56 AM: --- After acl is

[jira] [Updated] (HDDS-3373) Intermittent failure in TestDnRatisLogParser and TestOMRatisLogParser

2020-04-24 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3373: --- Status: Patch Available (was: In Progress) > Intermittent failure in TestDnRatisLogParser

[jira] [Updated] (HDDS-3373) Intermittent failure in TestDnRatisLogParser and TestOMRatisLogParser

2020-04-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3373: - Labels: pull-request-available (was: ) > Intermittent failure in TestDnRatisLogParser and

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #867: HDDS-3373. Intermittent failure in TestOMRatisLogParser

2020-04-24 Thread GitBox
adoroszlai opened a new pull request #867: URL: https://github.com/apache/hadoop-ozone/pull/867 ## What changes were proposed in this pull request? Fix intermittent failure in `TestOMRatisLogParser`. The problem was that regardless of which `i` index corresponded to non-`snapshot`

[GitHub] [hadoop-ozone] elek commented on pull request #782: HDDS-3352. Support for native ozone filesystem client using libhdfs.

2020-04-24 Thread GitBox
elek commented on pull request #782: URL: https://github.com/apache/hadoop-ozone/pull/782#issuecomment-618886565 Thanks to start this work @aryangupta1998 This is an automated message from the Apache Git Service. To respond

[GitHub] [hadoop-ozone] elek commented on pull request #863: HDDS-3461. Add acceptance test to smoketest CSI service startup

2020-04-24 Thread GitBox
elek commented on pull request #863: URL: https://github.com/apache/hadoop-ozone/pull/863#issuecomment-618885558 > I'd prefer adding csi container to the base ozone environment instead of duplication, but we can discuss that later. Yeah, I don't know what is the best approach. It

[GitHub] [hadoop-ozone] runzhiwang edited a comment on pull request #740: HDDS-3307. Improve write efficiency by avoiding reverse DNS lookup

2020-04-24 Thread GitBox
runzhiwang edited a comment on pull request #740: URL: https://github.com/apache/hadoop-ozone/pull/740#issuecomment-618882580 @elek This PR try to avoid finding hostname from IP address, it is a block call, and sometimes cost a long time, and also has no cache in JDK. It's true getting

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #740: HDDS-3307. Improve write efficiency by avoiding reverse DNS lookup

2020-04-24 Thread GitBox
runzhiwang commented on pull request #740: URL: https://github.com/apache/hadoop-ozone/pull/740#issuecomment-618882580 @elek This PR try to avoid finding DNS from IP address, it is a block call, and sometimes cost a long time, and also has no cache in JDK. It's true getting hostname from

[GitHub] [hadoop-ozone] elek commented on pull request #578: HDDS-3053. Decrease the number of the chunk writer threads

2020-04-24 Thread GitBox
elek commented on pull request #578: URL: https://github.com/apache/hadoop-ozone/pull/578#issuecomment-618881173 > In multi-RAFT setup will we end up creating 10numDisksnumPipelines threads, or will the total number of threads be the same as single RAFT? I am not sure about the

[GitHub] [hadoop-ozone] elek commented on pull request #740: HDDS-3307. Improve write efficiency by avoiding reverse DNS lookup

2020-04-24 Thread GitBox
elek commented on pull request #740: URL: https://github.com/apache/hadoop-ozone/pull/740#issuecomment-618879702 What is the plan this PR? I tried to understand but didn't get. The hostname is either used for authorization (in this case I am not sure if it's safe to use any

[GitHub] [hadoop-ozone] umamaheswararao commented on a change in pull request #859: HDDS-3465. OM Failover retry happens too quickly when new leader suggested and retrying on same OM.

2020-04-24 Thread GitBox
umamaheswararao commented on a change in pull request #859: URL: https://github.com/apache/hadoop-ozone/pull/859#discussion_r414392856 ## File path: hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/ha/TestOMFailoverProxyProvider.java ## @@ -0,0 +1,155 @@ +/** + *

[GitHub] [hadoop-ozone] elek commented on pull request #855: HDDS-3474. Create transactionInfo Table in OmMetadataManager.

2020-04-24 Thread GitBox
elek commented on pull request #855: URL: https://github.com/apache/hadoop-ozone/pull/855#issuecomment-618870672 Thanks @bharatviswa504 to work on this. I am trying to understand the problem to help with the code review but still not clear the proposal. I added my comments to the parent

[GitHub] [hadoop-ozone] maobaolong commented on pull request #825: HDDS-3395. Move the protobuf convert code to the OMHelper

2020-04-24 Thread GitBox
maobaolong commented on pull request #825: URL: https://github.com/apache/hadoop-ozone/pull/825#issuecomment-618869932 @elek Yeah, i think your approach is better, it is not only clear the logic but also avoid a big util class.

[GitHub] [hadoop-ozone] elek commented on a change in pull request #839: HDDS-3411. Switch Recon SQL DB to Derby.

2020-04-24 Thread GitBox
elek commented on a change in pull request #839: URL: https://github.com/apache/hadoop-ozone/pull/839#discussion_r414364877 ## File path: hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/codegen/ReconSqlDbConfig.java ## @@ -0,0 +1,50 @@ +/* + * Licensed to the

[jira] [Reopened] (HDDS-3373) Intermittent failure in TestDnRatisLogParser and TestOMRatisLogParser

2020-04-24 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai reopened HDDS-3373: TestOMRatisLogParser is still failing sometimes:

[GitHub] [hadoop-ozone] xiaoyuyao edited a comment on pull request #691: HDDS-3226. Tracing Ozone Manager DB write batch operations.

2020-04-24 Thread GitBox
xiaoyuyao edited a comment on pull request #691: URL: https://github.com/apache/hadoop-ozone/pull/691#issuecomment-618785177 > Overall LGTM. One minor comment and a general question. I have one question in general, will this add some overhead and is there a way to disable it? The

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #859: HDDS-3465. OM Failover retry happens too quickly when new leader suggested and retrying on same OM.

2020-04-24 Thread GitBox
adoroszlai commented on a change in pull request #859: URL: https://github.com/apache/hadoop-ozone/pull/859#discussion_r414329670 ## File path: hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/ha/TestOMFailoverProxyProvider.java ## @@ -0,0 +1,155 @@ +/** + *