[GitHub] [hadoop] slfan1989 commented on pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on PR #5443: URL: https://github.com/apache/hadoop/pull/5443#issuecomment-1451434842 > > Purging Delegation Token / Master key on expiry assumes all tokens are available in memory > > > > We only cache tokens in MemeoryStateStore, but MemeoryStateStore is not a

[jira] [Commented] (HADOOP-18640) ABFS: Enabling Client-side Backoff only for new requests

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695521#comment-17695521 ] ASF GitHub Bot commented on HADOOP-18640: - saxenapranav commented on code in PR #5446: URL:

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #5446: HADOOP-18640: [ABFS] Enabling Client-side Backoff only for new requests

2023-03-01 Thread via GitHub
saxenapranav commented on code in PR #5446: URL: https://github.com/apache/hadoop/pull/5446#discussion_r1122694142 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java: ## @@ -272,6 +273,30 @@ DefaultValue =

[jira] [Commented] (HADOOP-18649) CLA and CRLA appenders to be replaced with RFA

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695519#comment-17695519 ] ASF GitHub Bot commented on HADOOP-18649: - virajjasani opened a new pull request, #5448: URL:

[jira] [Updated] (HADOOP-18649) CLA and CRLA appenders to be replaced with RFA

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18649: Labels: pull-request-available (was: ) > CLA and CRLA appenders to be replaced with RFA

[GitHub] [hadoop] virajjasani opened a new pull request, #5448: HADOOP-18649. CLA and CRLA appenders to be replaced with RFA

2023-03-01 Thread via GitHub
virajjasani opened a new pull request, #5448: URL: https://github.com/apache/hadoop/pull/5448 ContainerLogAppender and ContainerRollingLogAppender both have quite similar functionality as RollingFileAppender. Maintenance of custom appenders for Log4J2 is costly when there is very minor

[jira] [Created] (HADOOP-18649) CLA and CRLA appenders to be replaced with RFA

2023-03-01 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-18649: - Summary: CLA and CRLA appenders to be replaced with RFA Key: HADOOP-18649 URL: https://issues.apache.org/jira/browse/HADOOP-18649 Project: Hadoop Common

[jira] [Commented] (HADOOP-17799) Improve the GitHub pull request template

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695515#comment-17695515 ] ASF GitHub Bot commented on HADOOP-17799: - trakos opened a new pull request, #5447: URL:

[GitHub] [hadoop] trakos opened a new pull request, #5447: HADOOP-17799. WebHdfs: add support for basic auth and custom API path

2023-03-01 Thread via GitHub
trakos opened a new pull request, #5447: URL: https://github.com/apache/hadoop/pull/5447 ### Description of PR `WebHdfsFileSystem` didn't provide any support for HTTP BASIC authentication (username/password). This patch adds that feature. When specifying filesystem URI, the

[jira] [Updated] (HADOOP-18640) ABFS: Enabling Client-side Backoff only for new requests

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18640: Labels: pull-request-available (was: ) > ABFS: Enabling Client-side Backoff only for

[jira] [Commented] (HADOOP-18640) ABFS: Enabling Client-side Backoff only for new requests

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695514#comment-17695514 ] ASF GitHub Bot commented on HADOOP-18640: - sreeb-msft opened a new pull request, #5446: URL:

[GitHub] [hadoop] sreeb-msft opened a new pull request, #5446: HADOOP-18640: [ABFS] Enabling Client-side Backoff only for new requests

2023-03-01 Thread via GitHub
sreeb-msft opened a new pull request, #5446: URL: https://github.com/apache/hadoop/pull/5446 This PR introduces two changes that allows client-side throttling and backoff only for new requests, and increases the level of control through new configs in AbfsClientThrottlingAnalyzer.

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122649160 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -419,31 +427,32 @@

[GitHub] [hadoop] slfan1989 commented on pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on PR #5443: URL: https://github.com/apache/hadoop/pull/5443#issuecomment-1451394954 > Yes. By in memory variables I meant the class instance variables like currentTokens, currentKey, allKeys, etc. Even though SQLFederationStateStore throws an exception, all the current

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122649160 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -419,31 +427,32 @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451337470 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] krishan1390 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
krishan1390 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122614703 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -531,9 +557,9 @@

[GitHub] [hadoop] krishan1390 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
krishan1390 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122613979 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -419,31 +427,32 @@

[GitHub] [hadoop] krishan1390 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
krishan1390 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122613543 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -67,9 +67,9 @@

[GitHub] [hadoop] krishan1390 commented on pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
krishan1390 commented on PR #5443: URL: https://github.com/apache/hadoop/pull/5443#issuecomment-1451319920 Thanks for your feedback @slfan1989 . Please find my response below allKeys needs to be consistently updated across all router instances > Multiple Routers will share and

[GitHub] [hadoop] hadoop-yetus commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451264241 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451224598 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5434: HDFS-16934. org.apache.hadoop.hdfs.tools.TestDFSAdmin#testAllDatanodesReconfig regression

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5434: URL: https://github.com/apache/hadoop/pull/5434#issuecomment-1451187716 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-17177) Java 17 support

2023-03-01 Thread Takanobu Asanuma (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695417#comment-17695417 ] Takanobu Asanuma commented on HADOOP-17177: --- Hi [~groot],  Before we upgraded our HDFS

[GitHub] [hadoop] hadoop-yetus commented on pull request #5322: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5322: URL: https://github.com/apache/hadoop/pull/5322#issuecomment-1451124197 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122491264 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -531,9 +557,9 @@

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122488897 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -419,31 +427,32 @@

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451105473 > If someone removes the processQueueMessages itself from the sendHeartbeat, then also this test should fail or atleast some should +1 -- This is an automated message from

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451104757 > It is inducing a race by nextHeartbeatTime Absolutely, that's what I thought too. But yes you are right, other than adding sleeps, it's bit tricky to reproduce. But yeah our

[GitHub] [hadoop] omalley merged pull request #5444: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
omalley merged PR #5444: URL: https://github.com/apache/hadoop/pull/5444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] ayushtkn commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
ayushtkn commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451100991 Playing a bit more with it: Single sleep doesn't repro itself. My wild guess is, it isn't testing from where we started. It is inducing a race by **nextHeartbeatTime**, that too because

[GitHub] [hadoop] xkrogen merged pull request #5400: HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist

2023-03-01 Thread via GitHub
xkrogen merged PR #5400: URL: https://github.com/apache/hadoop/pull/5400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] xkrogen commented on pull request #5400: HDFS-16923. The getlisting rpc will throw NPE if the path does not exist

2023-03-01 Thread via GitHub
xkrogen commented on PR #5400: URL: https://github.com/apache/hadoop/pull/5400#issuecomment-1451080156 The only test failures is `TestDirectoryScanner.testThrottling`: ``` [ERROR] Tests run: 13, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 575.031 s <<< FAILURE! - in

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451077691 > If my intent is just for processQueueMessages, I will expose and just shoot that directly, rather than doing the whole loop. That would also work but as part of the test, we

[GitHub] [hadoop] ayushtkn commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
ayushtkn commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451064227 First patch isn't something to consider itself. Second one is like very unrealistic in general, there aren't any things which we can blame above which can consume time, they are some

[jira] [Resolved] (HADOOP-18648) Avoid loading kms log4j properties dynamically by KMSWebServer

2023-03-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HADOOP-18648. -- Fix Version/s: 3.4.0 Resolution: Fixed > Avoid loading kms log4j properties

[jira] [Commented] (HADOOP-18648) Avoid loading kms log4j properties dynamically by KMSWebServer

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695367#comment-17695367 ] ASF GitHub Bot commented on HADOOP-18648: - jojochuang merged PR #5441: URL:

[GitHub] [hadoop] jojochuang merged pull request #5441: HADOOP-18648. Avoid loading kms log4j properties dynamically by KMSWebServer

2023-03-01 Thread via GitHub
jojochuang merged PR #5441: URL: https://github.com/apache/hadoop/pull/5441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18370) Fix missing package-info in hadoop-common moudle

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695366#comment-17695366 ] ASF GitHub Bot commented on HADOOP-18370: - hadoop-yetus commented on PR #4640: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4640: HADOOP-18370. Fix missing package-info in hadoop-common moudle.

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #4640: URL: https://github.com/apache/hadoop/pull/4640#issuecomment-1451044388 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451021126 The above patch, when applied with this PR changes, the test is passing consistently. Whereas without PR changes, the test is consistently failing (failed 7 times locally without

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451015537 Another way I am able to repro consistently: ``` diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1451008842 Though it's difficult to reproduce, I thought this utility would help test to ensure 100% that namenode has definitely received report as part of `ReportBadBlockAction#reportTo`. --

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1450990755 I tried multiple cases and for some sleeps, I am able to repro, only sometimes. The only way I am able to consistently repro failure is by applying this patch: ``` diff

[GitHub] [hadoop] ayushtkn commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
ayushtkn commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1450903655 Viraj, can you help me repro the scenario. I added 10K sleep before processEnqueed method last time and the test didn’t fail for me, triggerHeartbeat was working there -- This is an

[GitHub] [hadoop] virajjasani commented on pull request #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani commented on PR #5445: URL: https://github.com/apache/hadoop/pull/5445#issuecomment-1450896182 @ayushtkn @tomscut could you please review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] virajjasani opened a new pull request, #5445: HDFS-16938. Utility to trigger heartbeat and wait until BP thread queue is fully processed

2023-03-01 Thread via GitHub
virajjasani opened a new pull request, #5445: URL: https://github.com/apache/hadoop/pull/5445 As a follow-up to HDFS-16935, we should provide utility to trigger heartbeat and wait until BP thread queue is fully processed. This would ensure 100% consistency w.r.t active namenode being able

[GitHub] [hadoop] hadoop-yetus commented on pull request #5444: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5444: URL: https://github.com/apache/hadoop/pull/5444#issuecomment-1450786773 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] mccormickt12 opened a new pull request, #5444: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
mccormickt12 opened a new pull request, #5444: URL: https://github.com/apache/hadoop/pull/5444 …… (#5322) HDFS-16896 clear ignoredNodes list when we clear deadnode list on refetchLocations. ignoredNodes list is only used on hedged read codepath ### Description of PR

[GitHub] [hadoop] hadoop-yetus commented on pull request #4093: YARN-11093: fix for reading files in timeline server in chronological order in case of fs-support-append to false

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #4093: URL: https://github.com/apache/hadoop/pull/4093#issuecomment-1450760341 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-17177) Java 17 support

2023-03-01 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695295#comment-17695295 ] Ashutosh Gupta commented on HADOOP-17177: - [~tasanuma] - [~aajisaka] mentioned to me that you

[GitHub] [hadoop] hadoop-yetus commented on pull request #4093: YARN-11093: fix for reading files in timeline server in chronological order in case of fs-support-append to false

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #4093: URL: https://github.com/apache/hadoop/pull/4093#issuecomment-1450755518 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] omalley merged pull request #5322: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
omalley merged PR #5322: URL: https://github.com/apache/hadoop/pull/5322 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-17177) Java 17 support

2023-03-01 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695289#comment-17695289 ] Ashutosh Gupta commented on HADOOP-17177: - [~ste...@apache.org] - I have been playing/working

[GitHub] [hadoop] mccormickt12 commented on a diff in pull request #5322: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
mccormickt12 commented on code in PR #5322: URL: https://github.com/apache/hadoop/pull/5322#discussion_r1122202223 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -955,6 +965,10 @@ private DNAddrPair

[GitHub] [hadoop] mccormickt12 commented on a diff in pull request #5322: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-03-01 Thread via GitHub
mccormickt12 commented on code in PR #5322: URL: https://github.com/apache/hadoop/pull/5322#discussion_r1122201971 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -197,6 +197,15 @@ private void clearLocalDeadNodes() {

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5432: HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks

2023-03-01 Thread via GitHub
virajjasani commented on code in PR #5432: URL: https://github.com/apache/hadoop/pull/5432#discussion_r1122197968 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java: ## @@ -1101,15 +1099,12 @@ public

[GitHub] [hadoop] hadoop-yetus commented on pull request #5332: Yarn 11041. Replace all occurences of queuePath with the new QueuePath class - followup

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5332: URL: https://github.com/apache/hadoop/pull/5332#issuecomment-1450717538 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18493) update jackson-databind 2.12.7.1 due to CVE fixes

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695269#comment-17695269 ] Steve Loughran commented on HADOOP-18493: - [~phoebemaomao] here are the jackson libs in the

[jira] [Commented] (HADOOP-17844) Upgrade JSON smart to 2.4.7

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695268#comment-17695268 ] Steve Loughran commented on HADOOP-17844: - these are all the json jars of any kind in

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5432: HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks

2023-03-01 Thread via GitHub
virajjasani commented on code in PR #5432: URL: https://github.com/apache/hadoop/pull/5432#discussion_r1122177909 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java: ## @@ -1101,15 +1099,12 @@ public

[GitHub] [hadoop] steveloughran merged pull request #5432: HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks

2023-03-01 Thread via GitHub
steveloughran merged PR #5432: URL: https://github.com/apache/hadoop/pull/5432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Assigned] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18197: --- Assignee: (was: Steve Loughran) > Update protobuf 3.7.1 to a version without

[jira] [Commented] (HADOOP-18148) json smart 1.3.2 still appears in Trivy scan of build

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695257#comment-17695257 ] Steve Loughran commented on HADOOP-18148: - license-bin of the 3.3.5 RC2 thinks there is one

[jira] [Commented] (HADOOP-17177) Java 17 support

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695250#comment-17695250 ] Steve Loughran commented on HADOOP-17177: - i am stuck on java8, do some stuff with java11. once

[GitHub] [hadoop] hadoop-yetus commented on pull request #5332: Yarn 11041. Replace all occurences of queuePath with the new QueuePath class - followup

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5332: URL: https://github.com/apache/hadoop/pull/5332#issuecomment-1450642177 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5317: URL: https://github.com/apache/hadoop/pull/5317#issuecomment-1450611651 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
goiri commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1122098165 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -67,9 +67,9 @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5317: URL: https://github.com/apache/hadoop/pull/5317#issuecomment-1450515013 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5443: URL: https://github.com/apache/hadoop/pull/5443#issuecomment-1450494870 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
szilard-nemeth commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121923107 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public

[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
szilard-nemeth commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121914673 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public

[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
szilard-nemeth commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121911229 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public

[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
szilard-nemeth commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121910542 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public

[GitHub] [hadoop] szilard-nemeth closed pull request #5426: MAPREDUCE-7434. Fix ShuffleHandler tests.

2023-03-01 Thread via GitHub
szilard-nemeth closed pull request #5426: MAPREDUCE-7434. Fix ShuffleHandler tests. URL: https://github.com/apache/hadoop/pull/5426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] szilard-nemeth commented on pull request #5426: MAPREDUCE-7434. Fix ShuffleHandler tests.

2023-03-01 Thread via GitHub
szilard-nemeth commented on PR #5426: URL: https://github.com/apache/hadoop/pull/5426#issuecomment-1450311701 Thanks @tomicooler for working on this! Latest patch LGTM, committed to trunk. Thanks @susheelgupta7 for the review. -- This is an automated message from the Apache Git

[GitHub] [hadoop] hadoop-yetus commented on pull request #5434: HDFS-16934. org.apache.hadoop.hdfs.tools.TestDFSAdmin#testAllDatanodesReconfig regression

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5434: URL: https://github.com/apache/hadoop/pull/5434#issuecomment-1450289619 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on code in PR #5443: URL: https://github.com/apache/hadoop/pull/5443#discussion_r1121844947 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java: ## @@ -67,9 +67,9 @@

[GitHub] [hadoop] slfan1989 commented on pull request #5382: YARN-8972. [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size.

2023-03-01 Thread via GitHub
slfan1989 commented on PR #5382: URL: https://github.com/apache/hadoop/pull/5382#issuecomment-1450248007 @goiri Can you help review this PR again? Thank you very much! I will continue to follow up YARN-11376, YARN-11445. -- This is an automated message from the Apache Git Service.

[jira] [Commented] (HADOOP-17177) Java 17 support

2023-03-01 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695107#comment-17695107 ] Ashutosh Gupta commented on HADOOP-17177: - [~ste...@apache.org] By any chance did you try

[jira] (HADOOP-17177) Java 17 support

2023-03-01 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177 ] Ashutosh Gupta deleted comment on HADOOP-17177: - was (Author: groot): [~ste...@apache.org] By any chance did you try running with Java17 and still building with Java8 ? > Java 17

[jira] [Commented] (HADOOP-17177) Java 17 support

2023-03-01 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695106#comment-17695106 ] Ashutosh Gupta commented on HADOOP-17177: - [~ste...@apache.org] By any chance did you try

[jira] [Assigned] (HADOOP-17177) Java 17 support

2023-03-01 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Gupta reassigned HADOOP-17177: --- Assignee: Ashutosh Gupta > Java 17 support > --- > >

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121794141 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121787996 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121782444 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[GitHub] [hadoop] slfan1989 commented on pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
slfan1989 commented on PR #5443: URL: https://github.com/apache/hadoop/pull/5443#issuecomment-1450205098 @krishan1390 Thank you very much for your contribution, I will take time to look at this pr. I took a quick look at your description. > allKeys needs to be consistently

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121782444 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121779766 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121775542 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5317: YARN-11420 Stabilize TestNMClient

2023-03-01 Thread via GitHub
K0K0V0K commented on code in PR #5317: URL: https://github.com/apache/hadoop/pull/5317#discussion_r1121770810 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestNMClient.java: ## @@ -125,576 +117,362 @@ public void

[jira] [Resolved] (HADOOP-18636) LocalDirAllocator cannot recover from directory tree deletion during the life of a filesystem client

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18636. - Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed >

[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695068#comment-17695068 ] ASF GitHub Bot commented on HADOOP-18565: - ahmarsuhail commented on PR #5421: URL:

[GitHub] [hadoop] ahmarsuhail commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.

2023-03-01 Thread via GitHub
ahmarsuhail commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1450136517 @steveloughran this PR addresses (most) outstanding issues with the SDK upgrade PR, could you please review? Not sure why the yetus unit tests are failing, they run fine locally. And

[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695064#comment-17695064 ] ASF GitHub Bot commented on HADOOP-18565: - hadoop-yetus commented on PR #5421: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1450126641 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items

2023-03-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695062#comment-17695062 ] ASF GitHub Bot commented on HADOOP-18565: - hadoop-yetus commented on PR #5421: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.

2023-03-01 Thread via GitHub
hadoop-yetus commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1450123782 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] krishan1390 commented on pull request #5443: YARN-11448 [Federation] Stateless Router Secret Manager

2023-03-01 Thread via GitHub
krishan1390 commented on PR #5443: URL: https://github.com/apache/hadoop/pull/5443#issuecomment-1450078895 @goiri @slfan1989 can you please help review this PR ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] krishan1390 opened a new pull request, #5443: Stateless Router Secret Manager

2023-03-01 Thread via GitHub
krishan1390 opened a new pull request, #5443: URL: https://github.com/apache/hadoop/pull/5443 ### Description of PR JIRA - https://issues.apache.org/jira/browse/YARN-11448 Currently router secret manager requires routers to be stateful & with clients using sticky sessions.

  1   2   >