[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=631015&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631015 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-49397 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Commented] (HADOOP-17763) DistCp job fails when AM is killed

2021-07-28 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389286#comment-17389286 ] Hadoop QA commented on HADOOP-17763: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[GitHub] [hadoop] jianghuazhu commented on pull request #3170: HDFS-16107.Split RPC configuration to isolate RPC.

2021-07-28 Thread GitBox
jianghuazhu commented on pull request #3170: URL: https://github.com/apache/hadoop/pull/3170#issuecomment-35585 @jojochuang , do you have any new suggestions? I am willing to make more contributions. thank you very much. -- This is an automated message from the Apache Git Serv

[jira] [Updated] (HADOOP-12432) Add support for include/exclude lists on IPv6 setup

2021-07-28 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HADOOP-12432: Attachment: HADOOP-12432-HADOOP-17800.001.patch Status: Patch Available (was: Reo

[jira] [Reopened] (HADOOP-12432) Add support for include/exclude lists on IPv6 setup

2021-07-28 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina reopened HADOOP-12432: - > Add support for include/exclude lists on IPv6 setup > ---

[jira] [Commented] (HADOOP-12432) Add support for include/exclude lists on IPv6 setup

2021-07-28 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389278#comment-17389278 ] Hemanth Boyina commented on HADOOP-12432: - reopened and uploaded patch against

[GitHub] [hadoop] hadoop-yetus commented on pull request #3170: HDFS-16107.Split RPC configuration to isolate RPC.

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3170: URL: https://github.com/apache/hadoop/pull/3170#issuecomment-24322 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=631004&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631004 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-23045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #1808: MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGrou

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #1808: URL: https://github.com/apache/hadoop/pull/1808#issuecomment-19372 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Updated] (HADOOP-12491) Hadoop-common - Avoid unsafe split and append on fields that might be IPv6 literals

2021-07-28 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HADOOP-12491: Attachment: HADOOP-12491-HADOOP-17800.003.patch > Hadoop-common - Avoid unsafe split and

[GitHub] [hadoop] jojochuang merged pull request #2836: HDFS-15936.Solve BlockSender#sendPacket() does not record SocketTimeout exception.

2021-07-28 Thread GitBox
jojochuang merged pull request #2836: URL: https://github.com/apache/hadoop/pull/2836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issue

[GitHub] [hadoop] jojochuang commented on pull request #2836: HDFS-15936.Solve BlockSender#sendPacket() does not record SocketTimeout exception.

2021-07-28 Thread GitBox
jojochuang commented on pull request #2836: URL: https://github.com/apache/hadoop/pull/2836#issuecomment-00975 Merging it per Viraj and cxorm's review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=630980&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630980 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-888794912 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630973&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630973 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
mehakmeet commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888781408 I ran the tests in S3-CSE ON S3Guard ON, S3-CSE OFF S3-Guard ON, and S3-CSE OFF S3-Guard OFF. More of a mistake that I thought I had run the S3-CSE ON and S3-Guard OFF test sui

[jira] [Work logged] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?focusedWorklogId=630971&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630971 ] ASF GitHub Bot logged work on HADOOP-17812: --- Author: ASF GitHub B

[GitHub] [hadoop] wbo4958 commented on pull request #3222: HADOOP-17812. NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread GitBox
wbo4958 commented on pull request #3222: URL: https://github.com/apache/hadoop/pull/3222#issuecomment-888777338 Thx @steveloughran, I re-tested the integration test in AWS. and the result can be found at https://issues.apache.org/jira/browse/HADOOP-17812?focusedCommentId=17389239&page=com.

[jira] [Commented] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389239#comment-17389239 ] Bobby Wang commented on HADOOP-17812: - Hi Steve, I just found an aws account from

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888775284 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Updated] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bobby Wang updated HADOOP-17812: Attachment: failsafe-report.html.gz > NPE in S3AInputStream read() after failure to reconnect to

[jira] [Commented] (HADOOP-17763) DistCp job fails when AM is killed

2021-07-28 Thread Bilwa S T (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389217#comment-17389217 ] Bilwa S T commented on HADOOP-17763: Hi [~ayushtkn] [~epayne] can you please take a

[GitHub] [hadoop] jianghuazhu commented on pull request #2836: HDFS-15936.Solve BlockSender#sendPacket() does not record SocketTimeout exception.

2021-07-28 Thread GitBox
jianghuazhu commented on pull request #2836: URL: https://github.com/apache/hadoop/pull/2836#issuecomment-888762016 @cxorm , thank you very much for your work. Can this pr be merged into the trunk branch? If there is still a lack of other work, I am willing to contribute. thank you very

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888751847 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888747021 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888701605 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Resolved] (HADOOP-17819) Add extensions to ProtobufRpcEngine RequestHeaderProto

2021-07-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-17819. -- Fix Version/s: 3.2.3 2.10.2 3.4.0 H

[jira] [Work logged] (HADOOP-17819) Add extensions to ProtobufRpcEngine RequestHeaderProto

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17819?focusedWorklogId=630816&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630816 ] ASF GitHub Bot logged work on HADOOP-17819: --- Author: ASF GitHub B

[GitHub] [hadoop] shvachko commented on pull request #3242: HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto

2021-07-28 Thread GitBox
shvachko commented on pull request #3242: URL: https://github.com/apache/hadoop/pull/3242#issuecomment-888661912 +1 Will be committing this shortly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[jira] [Work logged] (HADOOP-17811) ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630799 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub B

[GitHub] [hadoop] brianloss commented on pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
brianloss commented on pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#issuecomment-888635485 > Can you do the cherrypick and test of the branch-3.3 locally, let me know if it's all good there and I'll merge it in there too. @steveloughran my local cherry pick aga

[GitHub] [hadoop] hadoop-yetus commented on pull request #3246: YARN-10848. Vcore allocation problem with DefaultResourceCalculator

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3246: URL: https://github.com/apache/hadoop/pull/3246#issuecomment-888614796 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-888603337 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?focusedWorklogId=630731&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630731 ] ASF GitHub Bot logged work on HADOOP-17812: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3222: HADOOP-17812. NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread GitBox
steveloughran commented on pull request #3222: URL: https://github.com/apache/hadoop/pull/3222#issuecomment-888573781 bq. BTW, I will upload the integration test results in JIRA later. don't need the full results, just add a comment here about where you tested (e.g. https://g

[jira] [Work logged] (HADOOP-17139) Re-enable optimized copyFromLocal implementation in S3AFileSystem

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17139?focusedWorklogId=630730&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630730 ] ASF GitHub Bot logged work on HADOOP-17139: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3101: HADOOP-17139 Re-enable optimized copyFromLocal implementation in S3AFileSystem

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3101: URL: https://github.com/apache/hadoop/pull/3101#discussion_r678596071 ## File path: hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md ## @@ -1427,57 +1427,91 @@ set to TRUE. If destination

[jira] [Updated] (HADOOP-17811) ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17811: Summary: ABFS ExponentialRetryPolicy doesn't pick up configuration values (was: ABFS: Al

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630729&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630729 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
steveloughran commented on pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#issuecomment-888563527 > ITestAzureBlobFileSystemLease talk to @billierinaldi there. w.r.t ITestAbfsStreamStatistics, if there's no JIRA on that test failure, file one, with the stack t

[jira] [Comment Edited] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0

2021-07-28 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388979#comment-17388979 ] Viraj Jasani edited comment on HADOOP-17612 at 7/28/21, 7:24 PM:

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630728&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630728 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub B

[GitHub] [hadoop] brianloss commented on pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
brianloss commented on pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#issuecomment-888561184 > +1 for this, merging to trunk and then soon 3.3.2. > > interesting you are seeing all the test failures. I don't go near Oauth myself (long story). Yeah, strange

[jira] [Updated] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17811: Fix Version/s: 3.4.0 > ABFS: Allow all ExponentialRetryPolicy properties to be configured

[jira] [Resolved] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17811. - Resolution: Fixed > ABFS: Allow all ExponentialRetryPolicy properties to be configured

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630726&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630726 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran merged pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
steveloughran merged pull request #3221: URL: https://github.com/apache/hadoop/pull/3221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-is

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630722&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630722 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3221: HADOOP-17811: Configure ExponentialRetryPolicy

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#discussion_r678585653 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -448,7 +448,7 @@ use requires the presence of secret credentials, wh

[jira] [Commented] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0

2021-07-28 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388979#comment-17388979 ] Viraj Jasani commented on HADOOP-17612: --- Thank you [~eolivelli] for Curator relea

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630718&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630718 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
steveloughran commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888553613 hmm. Yes, that'll be an interesting problem. Either test setup() checks for CSE on and skips the s3guard-enabled tests, or we catch the raised PathIOE and covert tha

[jira] [Commented] (HADOOP-17784) hadoop-aws landsat-pds test bucket will be deleted after Jul 1, 2021

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388963#comment-17388963 ] Steve Loughran commented on HADOOP-17784: - update, we're trying to convince the

[GitHub] [hadoop] hadoop-yetus commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-888535151 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] minni31 opened a new pull request #3246: YARN-10848. Vcore allocation problem with DefaultResourceCalculator

2021-07-28 Thread GitBox
minni31 opened a new pull request #3246: URL: https://github.com/apache/hadoop/pull/3246 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HAD

[jira] [Updated] (HADOOP-17814) Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread Takanobu Asanuma (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated HADOOP-17814: -- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch

[jira] [Work logged] (HADOOP-17814) Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17814?focusedWorklogId=630654&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630654 ] ASF GitHub Bot logged work on HADOOP-17814: --- Author: ASF GitHub B

[GitHub] [hadoop] tasanuma commented on pull request #3230: HADOOP-17814. Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread GitBox
tasanuma commented on pull request #3230: URL: https://github.com/apache/hadoop/pull/3230#issuecomment-888477003 Thanks for your contribution, @virajjasani! Thanks for your review, @jojochuang! -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Work logged] (HADOOP-17814) Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17814?focusedWorklogId=630653&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630653 ] ASF GitHub Bot logged work on HADOOP-17814: --- Author: ASF GitHub B

[GitHub] [hadoop] tasanuma merged pull request #3230: HADOOP-17814. Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread GitBox
tasanuma merged pull request #3230: URL: https://github.com/apache/hadoop/pull/3230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888463500 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=630639&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630639 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888452283 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630632&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630632 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet edited a comment on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
mehakmeet edited a comment on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888421643 Seems like, I've broken tests for S3-CSE ON because alot of S3Guard tests don't require your bucket to be s3guard enabled, and force the metastore to be dynamoDB. My lap

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630607&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630607 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
mehakmeet commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888421643 Seems like, I've broken tests for S3-CSE ON because some S3Guard tests don't require your bucket to be s3guard enabled, and force the metastore to be dynamoDB. My lapse on test

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888419375 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Resolved] (HADOOP-17789) S3 read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17789. - Resolution: Works for Me > S3 read performance with Spark with Hadoop 3.3.1 is slower t

[jira] [Commented] (HADOOP-17789) S3 read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388847#comment-17388847 ] Steve Loughran commented on HADOOP-17789: - bq. Can we use the latest one wildfl

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888374231 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Resolved] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17817. - Resolution: Fixed > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled >

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Component/s: fs/s3 > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled >

[jira] [Updated] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13887: Description: Expose the client-side encryption option documented in Amazon S3 documentati

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Fix Version/s: 3.4.0 > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Priority: Major (was: Minor) > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Priority: Minor (was: Major) > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Affects Version/s: 3.4.0 > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enab

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Summary: HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled (was: Throw a

[jira] [Work logged] (HADOOP-17817) Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630561&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630561 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran merged pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
steveloughran merged pull request #3239: URL: https://github.com/apache/hadoop/pull/3239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-is

[GitHub] [hadoop] hadoop-yetus commented on pull request #3243: HDFS-14529. SetTimes to throw FileNotFoundException if inode is not found

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3243: URL: https://github.com/apache/hadoop/pull/3243#issuecomment-888337965 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Commented] (HADOOP-17789) S3 read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

2021-07-28 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388790#comment-17388790 ] Arghya Saha commented on HADOOP-17789: -- [~ste...@apache.org] Thanks again for your

[GitHub] [hadoop] hadoop-yetus commented on pull request #3220: YARN-10355. Refactor NM ContainerLaunch.java#orderEnvByDependencies

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3220: URL: https://github.com/apache/hadoop/pull/3220#issuecomment-888301811 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3195: YARN-10459. containerLaunchedOnNode method not need to hold scheduler…

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3195: URL: https://github.com/apache/hadoop/pull/3195#issuecomment-888281625 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=630485&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630485 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888247513 Latest release * Address review comments * log IOStats after each test case. Important: as the cached FS retains statistics, the numbers get bigger over t

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888263127 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=630471&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630471 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#discussion_r678227498 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ## @@ -532,13 +549,15 @@ pri

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629905&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629905 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
mehakmeet commented on a change in pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#discussion_r678208587 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ## @@ -532,13 +549,15 @@ private

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629408&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629408 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#discussion_r678208290 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ## @@ -758,7 +828,7 @@ publi

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629248&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629248 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
hadoop-yetus removed a comment on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-887887092 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629011&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629011 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran edited a comment on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran edited a comment on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888217775 optional OptionalTestHDFSContractDistCp runs fine from IDE'; 4 minutes. All the performance issues of the HDFS contract are related to the large file tests at

  1   2   >