[GitHub] [hadoop] umamaheswararao commented on pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-25 Thread GitBox
umamaheswararao commented on pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#issuecomment-650009704 For some reason its unable to post test results in Github. No related test failures: https://builds.apache.org/job/hadoop-multibranch/job/PR-2092/3/testReport/ N

[GitHub] [hadoop] hadoop-yetus commented on pull request #2076: Hadoop 16961. ABFS: Adding metrics to AbfsInputStream

2020-06-25 Thread GitBox
hadoop-yetus commented on pull request #2076: URL: https://github.com/apache/hadoop/pull/2076#issuecomment-649994808 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10

[GitHub] [hadoop] ishaniahuja commented on pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
ishaniahuja commented on pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#issuecomment-649990701 javadoc failure is happening in trunk (causing yetus -1). JIRA: https://issues.apache.org/jira/browse/HADOOP-16862 --

[GitHub] [hadoop] mehakmeet commented on pull request #2076: Hadoop 16961. ABFS: Adding metrics to AbfsInputStream

2020-06-25 Thread GitBox
mehakmeet commented on pull request #2076: URL: https://github.com/apache/hadoop/pull/2076#issuecomment-649965672 I have changed the positioning of bytesReadFromBuffer, due to a bug found that this counter was actually incrementing ReadAhead buffer rather than local buffer(ReadAhead counte

[GitHub] [hadoop] ayushtkn commented on a change in pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-25 Thread GitBox
ayushtkn commented on a change in pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#discussion_r445953460 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemLinkFallback.java ## @@ -606,4 +602,166 @@ publ

[GitHub] [hadoop] virajith commented on pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
virajith commented on pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#issuecomment-649934530 Thanks for the review @umamaheswararao . I fixed the checkstyle and test failures in the last commit. Will wait for yetus to come back.

[GitHub] [hadoop] virajith commented on a change in pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
virajith commented on a change in pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#discussion_r445946962 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeWithHdfsScheme.java ## @@ -236,6

[GitHub] [hadoop] umamaheswararao commented on a change in pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
umamaheswararao commented on a change in pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#discussion_r445927183 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFsOverloadScheme.md ## @@ -150,6 +150,39 @@ DFSAdmin commands with View File S

[jira] [Commented] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-25 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17145971#comment-17145971 ] Thomas Marqardt commented on HADOOP-17089: -- branch-2.10: commit 0d4f9c778967c

[jira] [Updated] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-25 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt updated HADOOP-17089: - Fix Version/s: 2.10.1 > WASB: Update azure-storage-java SDK > -

[jira] [Commented] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17145965#comment-17145965 ] Hadoop QA commented on HADOOP-17083: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[GitHub] [hadoop] umamaheswararao commented on a change in pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-25 Thread GitBox
umamaheswararao commented on a change in pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#discussion_r445925144 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java ## @@ -1139,6 +1139,27 @@ public void mkdi

[GitHub] [hadoop] umamaheswararao commented on a change in pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-25 Thread GitBox
umamaheswararao commented on a change in pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#discussion_r445925144 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java ## @@ -1139,6 +1139,27 @@ public void mkdi

[GitHub] [hadoop] mehakmeet commented on pull request #2076: Hadoop 16961. ABFS: Adding metrics to AbfsInputStream

2020-06-25 Thread GitBox
mehakmeet commented on pull request #2076: URL: https://github.com/apache/hadoop/pull/2076#issuecomment-649901048 Thanks, @mukund-thakur. Brilliant debugging :) This is an automated message from the Apache Git Service. To res

[GitHub] [hadoop] hadoop-yetus commented on pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
hadoop-yetus commented on pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#issuecomment-649893051 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33

[GitHub] [hadoop] hadoop-yetus commented on pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
hadoop-yetus commented on pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#issuecomment-649891148 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 53

[jira] [Updated] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-25 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt updated HADOOP-17089: - Description: WASB depends on the Azure Storage Java SDK.  There is a concurrency bug in

[GitHub] [hadoop] NickyYe commented on a change in pull request #2095: HDFS-15312. Apply umask when creating directory by WebHDFS

2020-06-25 Thread GitBox
NickyYe commented on a change in pull request #2095: URL: https://github.com/apache/hadoop/pull/2095#discussion_r445903899 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java ## @@ -361,6 +378,23

[jira] [Updated] (HADOOP-17079) Optimize UGI#getGroups by adding UGI#getGroupsSet

2020-06-25 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-17079: Attachment: HADOOP-17079.003.patch > Optimize UGI#getGroups by adding UGI#getGroupsSet >

[GitHub] [hadoop] ayushtkn commented on a change in pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-25 Thread GitBox
ayushtkn commented on a change in pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#discussion_r445862114 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java ## @@ -1139,6 +1139,27 @@ public void mkdir(final

[GitHub] [hadoop] ayushtkn commented on a change in pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-25 Thread GitBox
ayushtkn commented on a change in pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#discussion_r445862114 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java ## @@ -1139,6 +1139,27 @@ public void mkdir(final

[GitHub] [hadoop] hadoop-yetus commented on pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
hadoop-yetus commented on pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#issuecomment-649806589 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35

[jira] [Commented] (HADOOP-17094) vulnerabilities reported in jackson and jackson-databind in branch-2.10

2020-06-25 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17145808#comment-17145808 ] Ahmed Hussein commented on HADOOP-17094: [~weichiu] I opted to stay on 2.9.10 t

[GitHub] [hadoop] virajith commented on a change in pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
virajith commented on a change in pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#discussion_r445808907 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeWithHdfsScheme.java ## @@ -235,6

[GitHub] [hadoop] virajith commented on a change in pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
virajith commented on a change in pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#discussion_r445808825 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeWithHdfsScheme.java ## @@ -235,6

[GitHub] [hadoop] virajith commented on a change in pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
virajith commented on a change in pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#discussion_r445808525 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/Constants.java ## @@ -41,12 +41,17 @@ * then the hadoop d

[GitHub] [hadoop] goiri commented on a change in pull request #2095: HDFS-15312. Apply umask when creating directory by WebHDFS

2020-06-25 Thread GitBox
goiri commented on a change in pull request #2095: URL: https://github.com/apache/hadoop/pull/2095#discussion_r445804683 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java ## @@ -361,6 +378,23 @@

[jira] [Updated] (HADOOP-17087) Add EC flag to stat commands

2020-06-25 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-17087: -- Resolution: Not A Problem Status: Resolved (was: Patch Available) Thanx [~wanghongbi

[GitHub] [hadoop] ishaniahuja commented on pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
ishaniahuja commented on pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#issuecomment-649778447 namespace, rest version:= 2018-11-09 Tests run: 84, Failures: 0, Errors: 0, Skipped: 0 Tests run: 443, Failures: 0, Errors: 0, Skipped: 42 Tests run: 207, Failures

[jira] [Commented] (HADOOP-17094) vulnerabilities reported in jackson and jackson-databind in branch-2.10

2020-06-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17145787#comment-17145787 ] Hadoop QA commented on HADOOP-17094: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[jira] [Created] (HADOOP-17095) findbugs warnings building branch-2.10

2020-06-25 Thread Ahmed Hussein (Jira)
Ahmed Hussein created HADOOP-17095: -- Summary: findbugs warnings building branch-2.10 Key: HADOOP-17095 URL: https://issues.apache.org/jira/browse/HADOOP-17095 Project: Hadoop Common Issue Ty

[jira] [Updated] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-25 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated HADOOP-17083: --- Attachment: HADOOP-17083-branch-2.10.005.patch > Update guava to 27.0-jre in hadoop branch-

[jira] [Updated] (HADOOP-17094) vulnerabilities reported in jackson and jackson-databind in branch-2.10

2020-06-25 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated HADOOP-17094: --- Attachment: HADOOP-17094-branch-2.10.001.patch Status: Patch Available (was: In Pro

[GitHub] [hadoop] ishaniahuja commented on a change in pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
ishaniahuja commented on a change in pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#discussion_r445745291 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -144,6 +145,10 @@ private f

[jira] [Work started] (HADOOP-17094) vulnerabilities reported in jackson and jackson-databind in branch-2.10

2020-06-25 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-17094 started by Ahmed Hussein. -- > vulnerabilities reported in jackson and jackson-databind in branch-2.10

[jira] [Commented] (HADOOP-17094) vulnerabilities reported in jackson and jackson-databind in branch-2.10

2020-06-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17145688#comment-17145688 ] Wei-Chiu Chuang commented on HADOOP-17094: -- +1 to update to jackson-databind 2

[jira] [Created] (HADOOP-17094) vulnerabilities reported in jackson and jackson-databind in branch-2.10

2020-06-25 Thread Ahmed Hussein (Jira)
Ahmed Hussein created HADOOP-17094: -- Summary: vulnerabilities reported in jackson and jackson-databind in branch-2.10 Key: HADOOP-17094 URL: https://issues.apache.org/jira/browse/HADOOP-17094 Project

[GitHub] [hadoop] umamaheswararao commented on pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
umamaheswararao commented on pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#issuecomment-649710533 Seems like there are some other tests to handle this cases from Yetus? This is an automated message from

[GitHub] [hadoop] umamaheswararao commented on a change in pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
umamaheswararao commented on a change in pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#discussion_r445698036 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeWithHdfsScheme.java ## @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #2069: HADOOP-16830. IOStatistics API.

2020-06-25 Thread GitBox
hadoop-yetus commented on pull request #2069: URL: https://github.com/apache/hadoop/pull/2069#issuecomment-649606358 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30

[jira] [Commented] (HADOOP-16254) Add proxy address in IPC connection

2020-06-25 Thread zhengchenyu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17144978#comment-17144978 ] zhengchenyu commented on HADOOP-16254: -- May I take over this issue, or we fix it t

[GitHub] [hadoop] mukund-thakur commented on pull request #2076: Hadoop 16961. ABFS: Adding metrics to AbfsInputStream

2020-06-25 Thread GitBox
mukund-thakur commented on pull request #2076: URL: https://github.com/apache/hadoop/pull/2076#issuecomment-649564035 The test fails only when parallel tests are enabled. mvn -T 1C clean verify -Dtest=none -Dit.test=ITestAbfsInputStreamStatistics#testWithNullStreamStatistics -- Succe

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2038: HADOOP-17022 Tune S3AFileSystem.listFiles() api.

2020-06-25 Thread GitBox
mukund-thakur commented on a change in pull request #2038: URL: https://github.com/apache/hadoop/pull/2038#discussion_r445414974 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -4181,79 +4181,114 @@ public LocatedFileStatus

[GitHub] [hadoop] hadoop-yetus commented on pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-25 Thread GitBox
hadoop-yetus commented on pull request #2100: URL: https://github.com/apache/hadoop/pull/2100#issuecomment-649370125 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 21m 57

[jira] [Updated] (HADOOP-17086) ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-25 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17086: -- Fix Version/s: 3.4.0 > ABFS: Fix the parsing errors in ABFS Driver with creation Time (being

[jira] [Updated] (HADOOP-17086) ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-25 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17086: -- Component/s: fs/azure > ABFS: Fix the parsing errors in ABFS Driver with creation Time (being

[jira] [Updated] (HADOOP-17086) ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-25 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17086: -- Affects Version/s: (was: 3.4.0) 3.3.0 > ABFS: Fix the parsing erro

[jira] [Updated] (HADOOP-17086) ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-25 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17086: -- Affects Version/s: 3.4.0 > ABFS: Fix the parsing errors in ABFS Driver with creation Time (be

[jira] [Updated] (HADOOP-17086) ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-25 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17086: -- Summary: ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in Li

[GitHub] [hadoop] bilaharith commented on a change in pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
bilaharith commented on a change in pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#discussion_r445367412 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java ## @@ -389,6 +423,12 @@ private sync

[GitHub] [hadoop] bilaharith commented on a change in pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
bilaharith commented on a change in pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#discussion_r445364600 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -144,6 +145,10 @@ private fi

[GitHub] [hadoop] snvijaya commented on a change in pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-25 Thread GitBox
snvijaya commented on a change in pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#discussion_r445355225 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java ## @@ -378,6 +428,7 @@ private synchro