[jira] [Commented] (HADOOP-18622) Upgrade ant to 1.10.13

2023-02-14 Thread Aleksandr Nikolaev (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1761#comment-1761 ] Aleksandr Nikolaev commented on HADOOP-18622: - [~groot]  If you do not mind, I would like

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5398: HDFS-16922. The logic of IncrementalBlockReportManager#addRDBI method may cause missing blocks when cluster is busy.

2023-02-14 Thread via GitHub
hfutatzhanghb commented on PR #5398: URL: https://github.com/apache/hadoop/pull/5398#issuecomment-1430888618 > Requires a UT which can reproduce the said issue Hi, @ayushtkn , I have updated the description of this issue. please take a look~ thanks a lot. -- This is an automated

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5398: HDFS-16922. The logic of IncrementalBlockReportManager#addRDBI method may cause missing blocks when cluster is busy.

2023-02-14 Thread via GitHub
hfutatzhanghb commented on PR #5398: URL: https://github.com/apache/hadoop/pull/5398#issuecomment-1430844228 > hello, @ayushtkn . OK, i will try to construct a UT to reproduce this issue, and I will try to describe the issue in this page. -- This is an automated message from the

[GitHub] [hadoop] hfutatzhanghb opened a new pull request, #5398: HDFS-16922. The logic of IncrementalBlockReportManager#addRDBI method may cause missing blocks when cluster is busy.

2023-02-14 Thread via GitHub
hfutatzhanghb opened a new pull request, #5398: URL: https://github.com/apache/hadoop/pull/5398 The current logic of IncrementalBlockReportManager# addRDBI method could lead to the missing blocks when datanodes in pipeline are I/O busy. -- This is an automated message from the Apache Git

[jira] [Created] (HADOOP-18631) Migrate Async appenders to log4j properties

2023-02-14 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-18631: - Summary: Migrate Async appenders to log4j properties Key: HADOOP-18631 URL: https://issues.apache.org/jira/browse/HADOOP-18631 Project: Hadoop Common

[GitHub] [hadoop] tomscut commented on a diff in pull request #5381: HDFS-16914. Add some logs for updateBlockForPipeline RPC.

2023-02-14 Thread via GitHub
tomscut commented on code in PR #5381: URL: https://github.com/apache/hadoop/pull/5381#discussion_r1106710999 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -5943,6 +5943,8 @@ LocatedBlock

[GitHub] [hadoop] virajjasani commented on pull request #5396: HDFS-16918. Optionally shut down datanode if it does not stay connected to active namenode

2023-02-14 Thread via GitHub
virajjasani commented on PR #5396: URL: https://github.com/apache/hadoop/pull/5396#issuecomment-1430828452 > If the datanode is connected to observer namenode, it can serve requests, why we need to shutdown The observer namenode takes a different condition. I was actually thinking

[GitHub] [hadoop] rdingankar opened a new pull request, #5397: HDFS-16917 Add transfer rate quantile metrics for DataNode reads

2023-02-14 Thread via GitHub
rdingankar opened a new pull request, #5397: URL: https://github.com/apache/hadoop/pull/5397 ### Description of PR Transfer rate metric for datanode reads will be calculated as the rate at which bytes are read ( bytes per ms ) With quantiles we will get a distribution of this

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688851#comment-17688851 ] ASF GitHub Bot commented on HADOOP-18215: - ayushtkn commented on PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
ayushtkn commented on PR #4215: URL: https://github.com/apache/hadoop/pull/4215#issuecomment-1430812485 To me it sounds ok, regarding the one liner, if that is a strickt ask and if other people are convinced I am ok with that as well. But it isn't something where I am the only one

[GitHub] [hadoop] ayushtkn commented on pull request #5396: HDFS-16918. Optionally shut down datanode if it does not stay connected to active namenode

2023-02-14 Thread via GitHub
ayushtkn commented on PR #5396: URL: https://github.com/apache/hadoop/pull/5396#issuecomment-1430800093 From Admin I mean cluster Administrator services, they can keep a track of datanodes and decide on what needs to be done to the datanode. If those services can shoot a restart if the

[GitHub] [hadoop] hadoop-yetus commented on pull request #5322: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5322: URL: https://github.com/apache/hadoop/pull/5322#issuecomment-1430775324 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] virajjasani commented on pull request #5396: HDFS-16918. Optionally shut down datanode if it does not stay connected to active namenode

2023-02-14 Thread via GitHub
virajjasani commented on PR #5396: URL: https://github.com/apache/hadoop/pull/5396#issuecomment-1430775234 For this change, the entire behavior is optional: ``` dfs.datanode.health.activennconnect.timeout 0 If the value is greater than 0, each

[GitHub] [hadoop] virajjasani commented on pull request #5396: HDFS-16918. Optionally shut down datanode if it does not stay connected to active namenode

2023-02-14 Thread via GitHub
virajjasani commented on PR #5396: URL: https://github.com/apache/hadoop/pull/5396#issuecomment-1430773119 In the large fleet of datanodes, any datanode that does not stay connected to active namenode due to any connectivity issue, can choose to get itself shutdown rather than impacting

[GitHub] [hadoop] virajjasani opened a new pull request, #5396: HDFS-16918. Optionally shut down datanode if it does not stay connected to active namenode

2023-02-14 Thread via GitHub
virajjasani opened a new pull request, #5396: URL: https://github.com/apache/hadoop/pull/5396 While deploying Hdfs on Envoy proxy setup, depending on the socket timeout configured at envoy, the network connection issues or packet loss could be observed. All of envoys basically form a

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688801#comment-17688801 ] ASF GitHub Bot commented on HADOOP-18215: - bbeaudreault commented on PR #4215: URL:

[GitHub] [hadoop] bbeaudreault commented on pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
bbeaudreault commented on PR #4215: URL: https://github.com/apache/hadoop/pull/4215#issuecomment-1430704163 > In general sounds fine to me. Do we need an extra validation that the class `implements Serialization`? So these serializable classes don't need to implement Serializable

[GitHub] [hadoop] bbeaudreault commented on pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
bbeaudreault commented on PR #4215: URL: https://github.com/apache/hadoop/pull/4215#issuecomment-1430703431 Just want to clarify the use-case here (as a reminder and for the new reviewers): - When you write a SequenceFile, the key class and value class are encoded in the format.

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688800#comment-17688800 ] ASF GitHub Bot commented on HADOOP-18215: - bbeaudreault commented on PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on pull request #5390: HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down

2023-02-14 Thread via GitHub
ayushtkn commented on PR #5390: URL: https://github.com/apache/hadoop/pull/5390#issuecomment-1430673981 Was just gonna hit the merge button, so thought of just trying and reproducing this issue locally as well. But it didn't repro

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688781#comment-17688781 ] ASF GitHub Bot commented on HADOOP-18215: - ayushtkn commented on code in PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
ayushtkn commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106558387 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[jira] [Updated] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18215: -- Status: In Progress (was: Patch Available) > Enhance WritableName to be able to return

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688775#comment-17688775 ] ASF GitHub Bot commented on HADOOP-18215: - ayushtkn commented on code in PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
ayushtkn commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106558387 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688772#comment-17688772 ] ASF GitHub Bot commented on HADOOP-18215: - hadoop-yetus commented on PR #4215: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #4215: URL: https://github.com/apache/hadoop/pull/4215#issuecomment-1430603452 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5381: HDFS-16914. Add some logs for updateBlockForPipeline RPC.

2023-02-14 Thread via GitHub
hfutatzhanghb commented on code in PR #5381: URL: https://github.com/apache/hadoop/pull/5381#discussion_r1106495693 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -5943,6 +5943,8 @@ LocatedBlock

[jira] [Commented] (HADOOP-18628) Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688750#comment-17688750 ] ASF GitHub Bot commented on HADOOP-18628: - hadoop-yetus commented on PR #5395: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5395: HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error (#5385)

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5395: URL: https://github.com/apache/hadoop/pull/5395#issuecomment-1430538769 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5381: HDFS-16914. Add some logs for updateBlockForPipeline RPC.

2023-02-14 Thread via GitHub
slfan1989 commented on code in PR #5381: URL: https://github.com/apache/hadoop/pull/5381#discussion_r1106493967 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -5943,6 +5943,8 @@ LocatedBlock

[GitHub] [hadoop] mccormickt12 commented on pull request #5322: HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref…

2023-02-14 Thread via GitHub
mccormickt12 commented on PR #5322: URL: https://github.com/apache/hadoop/pull/5322#issuecomment-1430524690 > The change generally looks good to me. > > One concern is that when `refetchLocation` is called inside `hedgedFetchBlockByteRange` we could remove a node from the ignoredList

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688712#comment-17688712 ] ASF GitHub Bot commented on HADOOP-18215: - bbeaudreault commented on code in PR #4215: URL:

[GitHub] [hadoop] bbeaudreault commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
bbeaudreault commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106403146 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[jira] [Commented] (HADOOP-18629) Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688711#comment-17688711 ] ASF GitHub Bot commented on HADOOP-18629: - hadoop-yetus commented on PR #5391: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5391: HADOOP-18629. Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5391: URL: https://github.com/apache/hadoop/pull/5391#issuecomment-1430405499 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5363: YARN-11424. [Federation] Router Supports DeregisterSubCluster.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5363: URL: https://github.com/apache/hadoop/pull/5363#issuecomment-1430392232 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18628) Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688700#comment-17688700 ] ASF GitHub Bot commented on HADOOP-18628: - virajjasani commented on PR #5385: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5385: HADOOP-18628. Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread via GitHub
virajjasani commented on PR #5385: URL: https://github.com/apache/hadoop/pull/5385#issuecomment-1430362627 Thanks for the reviews! branch-3.3 backport PR: https://github.com/apache/hadoop/pull/5395 -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (HADOOP-18628) Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688698#comment-17688698 ] ASF GitHub Bot commented on HADOOP-18628: - virajjasani opened a new pull request, #5395: URL:

[jira] [Commented] (HADOOP-18628) Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688699#comment-17688699 ] ASF GitHub Bot commented on HADOOP-18628: - virajjasani commented on PR #5395: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5395: HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error (#5385)

2023-02-14 Thread via GitHub
virajjasani commented on PR #5395: URL: https://github.com/apache/hadoop/pull/5395#issuecomment-1430361924 FYI @steveloughran for branch-3.3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] virajjasani opened a new pull request, #5395: HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error (#5385)

2023-02-14 Thread via GitHub
virajjasani opened a new pull request, #5395: URL: https://github.com/apache/hadoop/pull/5395 branch-3.3 backport PR for #5385 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hadoop-yetus commented on pull request #5312: YARN-11375. [Federation] Support refreshAdminAcls、refreshServiceAcls API's for Federation.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5312: URL: https://github.com/apache/hadoop/pull/5312#issuecomment-1430321253 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5394: YARN-5604. [Federation] Add versioning for FederationStateStore.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5394: URL: https://github.com/apache/hadoop/pull/5394#issuecomment-1430299760 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5335: YARN-11426. Improve YARN NodeLabel Memory Display.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5335: URL: https://github.com/apache/hadoop/pull/5335#issuecomment-1430268097 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5302: YARN-11221. [Federation] Add replaceLabelsOnNodes, replaceLabelsOnNode REST APIs for Router.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5302: URL: https://github.com/apache/hadoop/pull/5302#issuecomment-1430247750 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5326: YARN-11425. [Federation] Router Supports SubClusterCleaner.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5326: URL: https://github.com/apache/hadoop/pull/5326#issuecomment-1430172422 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5382: YARN-8972. [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5382: URL: https://github.com/apache/hadoop/pull/5382#issuecomment-1430160111 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688634#comment-17688634 ] ASF GitHub Bot commented on HADOOP-18215: - ayushtkn commented on code in PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
ayushtkn commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106140456 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[jira] [Commented] (HADOOP-18629) Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688631#comment-17688631 ] ASF GitHub Bot commented on HADOOP-18629: - zhuyaogai commented on PR #5391: URL:

[GitHub] [hadoop] zhuyaogai commented on pull request #5391: HADOOP-18629. Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread via GitHub
zhuyaogai commented on PR #5391: URL: https://github.com/apache/hadoop/pull/5391#issuecomment-1430118718 > @steveloughran hi, I have fixed some problems, could you please review it again? and please correct me if I'm wrong. Thank you :) -- This is an automated message from the

[jira] [Updated] (HADOOP-18627) site intro docs to make clear Kerberos is mandatory for secure clusters

2023-02-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18627: Parent: HADOOP-18470 Issue Type: Sub-task (was: Improvement) > site intro docs

[jira] [Resolved] (HADOOP-18627) site intro docs to make clear Kerberos is mandatory for secure clusters

2023-02-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18627. - Fix Version/s: 3.3.5 Resolution: Fixed added this within the HADOOP-18470 text

[jira] [Commented] (HADOOP-18470) Release hadoop 3.3.5

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688625#comment-17688625 ] ASF GitHub Bot commented on HADOOP-18470: - steveloughran merged PR #5383: URL:

[GitHub] [hadoop] steveloughran merged pull request #5383: HADOOP-18470. More in the index.html about security

2023-02-14 Thread via GitHub
steveloughran merged PR #5383: URL: https://github.com/apache/hadoop/pull/5383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5328: YARN-11222. [Federation] Add addToClusterNodeLabels, removeFromClusterNodeLabels REST APIs for Router.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5328: URL: https://github.com/apache/hadoop/pull/5328#issuecomment-1430074590 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688605#comment-17688605 ] ASF GitHub Bot commented on HADOOP-18215: - bbeaudreault commented on PR #4215: URL:

[GitHub] [hadoop] bbeaudreault commented on pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
bbeaudreault commented on PR #4215: URL: https://github.com/apache/hadoop/pull/4215#issuecomment-1430056678 Thanks for looking and for the suggestions! Just wanted to ask one question about your review comment before I make the change. -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688604#comment-17688604 ] ASF GitHub Bot commented on HADOOP-18215: - bbeaudreault commented on code in PR #4215: URL:

[GitHub] [hadoop] bbeaudreault commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
bbeaudreault commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106084218 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688595#comment-17688595 ] ASF GitHub Bot commented on HADOOP-18215: - ayushtkn commented on code in PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
ayushtkn commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106063559 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[jira] [Commented] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688593#comment-17688593 ] ASF GitHub Bot commented on HADOOP-18215: - ayushtkn commented on code in PR #4215: URL:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2023-02-14 Thread via GitHub
ayushtkn commented on code in PR #4215: URL: https://github.com/apache/hadoop/pull/4215#discussion_r1106063559 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableName.java: ## @@ -92,7 +92,7 @@ public static synchronized Class getClass(String

[GitHub] [hadoop] slfan1989 commented on pull request #5394: YARN-5604. [Federation] Add versioning for FederationStateStore.

2023-02-14 Thread via GitHub
slfan1989 commented on PR #5394: URL: https://github.com/apache/hadoop/pull/5394#issuecomment-1429996394 @goiri Can you help review this pr? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] slfan1989 commented on pull request #5382: YARN-8972. [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size.

2023-02-14 Thread via GitHub
slfan1989 commented on PR #5382: URL: https://github.com/apache/hadoop/pull/5382#issuecomment-1429996105 @goiri Can you help review this pr? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] slfan1989 commented on pull request #5244: YARN-11349. [Federation] Router Support DelegationToken With SQL.

2023-02-14 Thread via GitHub
slfan1989 commented on PR #5244: URL: https://github.com/apache/hadoop/pull/5244#issuecomment-1429993229 @goiri Can you help to merge this pr into the trunk branch? Thank you very much! The SQL scripts involved in this pr have been verified in the environment of `Mysql` and

[GitHub] [hadoop] slfan1989 opened a new pull request, #5394: YARN-5604. [Federation] Add versioning for FederationStateStore.

2023-02-14 Thread via GitHub
slfan1989 opened a new pull request, #5394: URL: https://github.com/apache/hadoop/pull/5394 JIRA: YARN-5604. Add versioning for FederationStateStore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] hadoop-yetus commented on pull request #5244: YARN-11349. [Federation] Router Support DelegationToken With SQL.

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5244: URL: https://github.com/apache/hadoop/pull/5244#issuecomment-1429986002 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18351) S3A prefetching: Error logging during reads

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688546#comment-17688546 ] ASF GitHub Bot commented on HADOOP-18351: - sauraank commented on PR #5274: URL:

[GitHub] [hadoop] sauraank commented on pull request #5274: HADOOP-18351. Reduce excess logging of errors during S3A prefetching reads

2023-02-14 Thread via GitHub
sauraank commented on PR #5274: URL: https://github.com/apache/hadoop/pull/5274#issuecomment-1429906388 Not written a separate test for it, but manually tested logging levels of Error/info locally by throwing errors in case of both cases of async prefetch and sync get calls for read

[jira] [Commented] (HADOOP-18351) S3A prefetching: Error logging during reads

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688533#comment-17688533 ] ASF GitHub Bot commented on HADOOP-18351: - hadoop-yetus commented on PR #5274: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5274: HADOOP-18351. Reduce excess logging of errors during S3A prefetching reads

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5274: URL: https://github.com/apache/hadoop/pull/5274#issuecomment-1429868770 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5392: HDFS-16915. Optimize metrics for operations hold lock times of FsDatasetImpl

2023-02-14 Thread via GitHub
hfutatzhanghb commented on PR #5392: URL: https://github.com/apache/hadoop/pull/5392#issuecomment-1429855095 > @Hexiaoqiao , hi, could you please take a look at this. For the convenience of the existing metrics name, i keep the metric names as original. -- This is an automated

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5278: YARN-11408. Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method

2023-02-14 Thread via GitHub
brumi1024 commented on code in PR #5278: URL: https://github.com/apache/hadoop/pull/5278#discussion_r1105904503 ##

[GitHub] [hadoop] hadoop-yetus commented on pull request #5392: HDFS-16915. Optimize metrics for operations hold lock times of FsDatasetImpl

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5392: URL: https://github.com/apache/hadoop/pull/5392#issuecomment-1429817139 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18630) Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread Simhadri Govindappa (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688513#comment-17688513 ] Simhadri Govindappa commented on HADOOP-18630: -- Thanks, [~ayushtkn] ! > Add gh-pages in

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

2023-02-14 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688506#comment-17688506 ] Ayush Saxena commented on HADOOP-18524: --- Committed to trunk. Thanx [~aajisaka] and

[jira] [Resolved] (HADOOP-18524) Deploy Hadoop trunk version website

2023-02-14 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18524. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Deploy Hadoop

[jira] [Commented] (HADOOP-18630) Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688504#comment-17688504 ] Ayush Saxena commented on HADOOP-18630: --- Committed to trunk. Thanx [~simhadri-g] for the

[jira] [Resolved] (HADOOP-18630) Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18630. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add gh-pages

[jira] [Commented] (HADOOP-18630) Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688490#comment-17688490 ] ASF GitHub Bot commented on HADOOP-18630: - ayushtkn merged PR #5393: URL:

[GitHub] [hadoop] ayushtkn merged pull request #5393: HADOOP-18630: Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread via GitHub
ayushtkn merged PR #5393: URL: https://github.com/apache/hadoop/pull/5393 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18582) No need to clean tmp files in distcp direct mode

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688489#comment-17688489 ] ASF GitHub Bot commented on HADOOP-18582: - ayushtkn commented on code in PR #5251: URL:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5251: HADOOP-18582. skip unnecessary cleanup logic in distcp

2023-02-14 Thread via GitHub
ayushtkn commented on code in PR #5251: URL: https://github.com/apache/hadoop/pull/5251#discussion_r1105764477 ## hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java: ## @@ -152,9 +152,18 @@ public void abortJob(JobContext jobContext, }

[GitHub] [hadoop] hadoop-yetus commented on pull request #5391: HADOOP-18629. Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5391: URL: https://github.com/apache/hadoop/pull/5391#issuecomment-1429680256 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18629) Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688488#comment-17688488 ] ASF GitHub Bot commented on HADOOP-18629: - hadoop-yetus commented on PR #5391: URL:

[jira] [Commented] (HADOOP-18629) Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688483#comment-17688483 ] ASF GitHub Bot commented on HADOOP-18629: - hadoop-yetus commented on PR #5391: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5391: HADOOP-18629. Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5391: URL: https://github.com/apache/hadoop/pull/5391#issuecomment-1429650998 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18630) Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688480#comment-17688480 ] ASF GitHub Bot commented on HADOOP-18630: - hadoop-yetus commented on PR #5393: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5393: HADOOP-18630: Add gh-pages in asf.yaml to deploy the current trunk doc

2023-02-14 Thread via GitHub
hadoop-yetus commented on PR #5393: URL: https://github.com/apache/hadoop/pull/5393#issuecomment-1429629418 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5288: YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors.

2023-02-14 Thread via GitHub
slfan1989 commented on PR #5288: URL: https://github.com/apache/hadoop/pull/5288#issuecomment-1429619265 @steveloughran Can you help review this pr? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (HADOOP-18582) No need to clean tmp files in distcp direct mode

2023-02-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688477#comment-17688477 ] ASF GitHub Bot commented on HADOOP-18582: - steveloughran commented on code in PR #5251: URL:

[jira] [Updated] (HADOOP-18629) Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18629: Component/s: tools/distcp (was: tools) > Hadoop DistCp supports

[jira] [Updated] (HADOOP-18629) Hadoop DistCp supports specifying favoredNodes for data copying

2023-02-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18629: Affects Version/s: 3.3.4 > Hadoop DistCp supports specifying favoredNodes for data

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5251: HADOOP-18582. skip unnecessary cleanup logic in distcp

2023-02-14 Thread via GitHub
steveloughran commented on code in PR #5251: URL: https://github.com/apache/hadoop/pull/5251#discussion_r1105705726 ## hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java: ## @@ -152,9 +152,18 @@ public void abortJob(JobContext jobContext,

[jira] [Updated] (HADOOP-18628) Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18628: Priority: Minor (was: Major) > Server connection should log host name before returning

[jira] [Resolved] (HADOOP-18628) Server connection should log host name before returning VersionMismatch error

2023-02-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18628. - Fix Version/s: 3.4.0 Resolution: Fixed > Server connection should log host name

  1   2   >