[GitHub] [hadoop] jojochuang commented on a diff in pull request #4107: HDFS-16521. DFS API to retrieve slow datanodes

2022-04-26 Thread GitBox
jojochuang commented on code in PR #4107: URL: https://github.com/apache/hadoop/pull/4107#discussion_r859381403 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -4914,6 +4914,33 @@ int

[GitHub] [hadoop] skysiders opened a new pull request, #4236: MAPREDUCE-7375 set permission for JobSubmissionFiles

2022-04-26 Thread GitBox
skysiders opened a new pull request, #4236: URL: https://github.com/apache/hadoop/pull/4236 ### Description of PR JobSubmissionFiles provide getStagingDir to get Staging Directory.If stagingArea missing, method will create new directory with this. fs.mkdirs(stagingArea, new

[GitHub] [hadoop] Hexiaoqiao commented on pull request #4213: HDFS-16554 Remove unused configuration dfs.namenode.block.deletion.increment.

2022-04-26 Thread GitBox
Hexiaoqiao commented on PR #4213: URL: https://github.com/apache/hadoop/pull/4213#issuecomment-1110505405 Committed to trunk. Thanks @smarthanwang for your contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4228: URL: https://github.com/apache/hadoop/pull/4228#issuecomment-1110505106 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] Hexiaoqiao merged pull request #4213: HDFS-16554 Remove unused configuration dfs.namenode.block.deletion.increment.

2022-04-26 Thread GitBox
Hexiaoqiao merged PR #4213: URL: https://github.com/apache/hadoop/pull/4213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] Hexiaoqiao commented on pull request #4230: HDFS-16558 Consider changing the lock of delegation token from write …

2022-04-26 Thread GitBox
Hexiaoqiao commented on PR #4230: URL: https://github.com/apache/hadoop/pull/4230#issuecomment-1110498309 `In a very busy authed cluster, renewing/caneling/getting delegation token get slow and it will slow down the speed of handling rpcs from client. Since

[GitHub] [hadoop] smarthanwang commented on pull request #4213: HDFS-16554 Remove unused configuration dfs.namenode.block.deletion.increment.

2022-04-26 Thread GitBox
smarthanwang commented on PR #4213: URL: https://github.com/apache/hadoop/pull/4213#issuecomment-1110494557 @Hexiaoqiao thanks for review. The failed UTs are not related with this PR which seems failed for timeout. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] tomscut commented on pull request #4209: HDFS-16550. [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-26 Thread GitBox
tomscut commented on PR #4209: URL: https://github.com/apache/hadoop/pull/4209#issuecomment-1110446887 Hi @tasanuma @ayushtkn @sunchao @xkrogen , could you please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] virajjasani commented on pull request #4186: HDFS-16528. Reconfigure slow peer enable for Namenode

2022-04-26 Thread GitBox
virajjasani commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1110437828 Thanks for the review @tomscut ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] tomscut commented on pull request #4201: HDFS-16547. [SBN read] Namenode in safe mode should not be transfered to observer state

2022-04-26 Thread GitBox
tomscut commented on PR #4201: URL: https://github.com/apache/hadoop/pull/4201#issuecomment-1110433872 Hi @sunchao @xkrogen , could you please take a look? Thanks a lot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=762634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762634 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #3964: URL: https://github.com/apache/hadoop/pull/3964#issuecomment-1110371645 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4228: URL: https://github.com/apache/hadoop/pull/4228#issuecomment-1110361697 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-16965) Introduce StreamContext for Abfs Input and Output streams.

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16965?focusedWorklogId=762629=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762629 ] ASF GitHub Bot logged work on HADOOP-16965: --- Author: ASF GitHub Bot

[GitHub] [hadoop] arjun4084346 commented on pull request #4171: HADOOP-16965. Refactor abfs stream configuration. (#1956)

2022-04-26 Thread GitBox
arjun4084346 commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-1110354907 > I ran the integration tests on branch-2.10 and that also has skipped tests. Test results are same for both the branch-2.10 and my PR in terms of tests run, failures,

[GitHub] [hadoop] goiri commented on a diff in pull request #4228: HDFS-16468. Define ssize_t for Windows

2022-04-26 Thread GitBox
goiri commented on code in PR #4228: URL: https://github.com/apache/hadoop/pull/4228#discussion_r859202738 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/x-platform/types.h: ## @@ -0,0 +1,39 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [hadoop] goiri commented on a diff in pull request #4228: HDFS-16468. Define ssize_t for Windows

2022-04-26 Thread GitBox
goiri commented on code in PR #4228: URL: https://github.com/apache/hadoop/pull/4228#discussion_r859200993 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/examples/cc/cat/cat.cc: ## @@ -62,7 +62,6 @@ int main(int argc, char *argv[]) { //wrapping

[jira] [Commented] (HADOOP-18187) Convert s3a prefetching to use JavaDoc for fields and enums

2022-04-26 Thread Bhalchandra Pandit (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528442#comment-17528442 ] Bhalchandra Pandit commented on HADOOP-18187: - [~dannycjones]  do you have a reference

[jira] [Assigned] (HADOOP-18187) Convert s3a prefetching to use JavaDoc for fields and enums

2022-04-26 Thread Bhalchandra Pandit (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhalchandra Pandit reassigned HADOOP-18187: --- Assignee: Bhalchandra Pandit > Convert s3a prefetching to use JavaDoc for

[jira] [Commented] (HADOOP-18182) S3File to store reference to active S3Object in a field.

2022-04-26 Thread Bhalchandra Pandit (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528440#comment-17528440 ] Bhalchandra Pandit commented on HADOOP-18182: - [~ste...@apache.org] I did not understand

[jira] [Assigned] (HADOOP-18182) S3File to store reference to active S3Object in a field.

2022-04-26 Thread Bhalchandra Pandit (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhalchandra Pandit reassigned HADOOP-18182: --- Assignee: Bhalchandra Pandit > S3File to store reference to active

[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=762537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762537 ] ASF GitHub Bot logged work on HADOOP-18069: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ashutoshcipher commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3

2022-04-26 Thread GitBox
ashutoshcipher commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1110235192 > commented. > > * checkstyle needs to be happy, along with javac. > * spotbugs still thinks there is a problem. what is it that it is warning about? > > i'm

[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4228: URL: https://github.com/apache/hadoop/pull/4228#issuecomment-1110234952 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=762514=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762514 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-04-26 Thread GitBox
mukund-thakur commented on code in PR #3964: URL: https://github.com/apache/hadoop/pull/3964#discussion_r859104404 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java: ## @@ -564,6 +568,23 @@ public void initialize(URI name, Configuration

[jira] [Commented] (HADOOP-18194) Public dataset class for S3A integration tests

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528370#comment-17528370 ] Steve Loughran commented on HADOOP-18194: - yeah > Public dataset class for S3A integration

[jira] [Resolved] (HADOOP-18216) Document "io.file.buffer.size" must be greater than zero

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18216. - Resolution: Fixed documentation is updated. PR discusses a followup with safety check

[jira] [Updated] (HADOOP-18216) Document "io.file.buffer.size" must be greater than zero

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18216: Summary: Document "io.file.buffer.size" must be greater than zero (was: Ensure

[jira] [Updated] (HADOOP-18216) Document "io.file.buffer.size" must be greater than zero

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18216: Priority: Minor (was: Major) > Document "io.file.buffer.size" must be greater than zero

[jira] [Updated] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18216: Fix Version/s: 3.4.0 > Ensure "io.file.buffer.size" is greater than zero. Otherwise, it

[jira] [Work logged] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762486=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762486 ] ASF GitHub Bot logged work on HADOOP-18216: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero

2022-04-26 Thread GitBox
steveloughran merged PR #4220: URL: https://github.com/apache/hadoop/pull/4220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762485=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762485 ] ASF GitHub Bot logged work on HADOOP-18216: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero

2022-04-26 Thread GitBox
steveloughran commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-1110139088 @Hexiaoqiao yes, you can add a hadoop.util.Precondition check where it is loaded, but do it in a new PR. let's merge this doc change in first, as it self contained and low risk to

[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=762481=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762481 ] ASF GitHub Bot logged work on HADOOP-18069: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3

2022-04-26 Thread GitBox
steveloughran commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1110137354 commented. * checkstyle needs to be happy, along with javac. * spotbugs still thinks there is a problem. what is it that it is warning about? i'm worried about adding

[jira] [Updated] (HADOOP-18167) Add metrics to track delegation token secret manager operations

2022-04-26 Thread Hector Sandoval Chaverri (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hector Sandoval Chaverri updated HADOOP-18167: -- Attachment: HADOOP-18167-branch-2.10-4.patch > Add metrics to track

[jira] [Commented] (HADOOP-18028) High performance S3A input stream with prefetching & caching

2022-04-26 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528330#comment-17528330 ] Ahmar Suhail commented on HADOOP-18028: --- [https://github.com/apache/hadoop/pull/4212] fixes the

[jira] [Resolved] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18177. - Resolution: Fixed > document use and architecture design of prefetching s3a input

[jira] [Updated] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18177: Fix Version/s: 3.4.0 > document use and architecture design of prefetching s3a input

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762448=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762448 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
steveloughran merged PR #4205: URL: https://github.com/apache/hadoop/pull/4205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18028) High performance S3A input stream with prefetching & caching

2022-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528323#comment-17528323 ] Steve Loughran commented on HADOOP-18028: - this branch is going to need rebasing. what

[jira] [Updated] (HADOOP-18167) Add metrics to track delegation token secret manager operations

2022-04-26 Thread Hector Sandoval Chaverri (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hector Sandoval Chaverri updated HADOOP-18167: -- Attachment: HADOOP-18167-branch-3.3.patch > Add metrics to track

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4228: HDFS-16468. Define ssize_t for Windows

2022-04-26 Thread GitBox
GauthamBanasandra commented on code in PR #4228: URL: https://github.com/apache/hadoop/pull/4228#discussion_r858971962 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c: ## @@ -487,10 +488,10 @@ static ssize_t wildcard_expandPath(const char*

[GitHub] [hadoop] huaxiangsun commented on pull request #4170: HDFS-16540 Data locality is lost when DataNode pod restarts in kubern…

2022-04-26 Thread GitBox
huaxiangsun commented on PR #4170: URL: https://github.com/apache/hadoop/pull/4170#issuecomment-1110051006 Any more comments? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hadoop-yetus commented on pull request #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4235: URL: https://github.com/apache/hadoop/pull/4235#issuecomment-1110048699 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4235: URL: https://github.com/apache/hadoop/pull/4235#issuecomment-1110046794 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=762417=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762417 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1110029320 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18167) Add metrics to track delegation token secret manager operations

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18167?focusedWorklogId=762413=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762413 ] ASF GitHub Bot logged work on HADOOP-18167: --- Author: ASF GitHub Bot

[GitHub] [hadoop] omalley merged pull request #4092: HADOOP-18167. Add metrics to track delegation token secret manager op…

2022-04-26 Thread GitBox
omalley merged PR #4092: URL: https://github.com/apache/hadoop/pull/4092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=762384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762384 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot

[GitHub] [hadoop] dannycjones commented on a diff in pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket

2022-04-26 Thread GitBox
dannycjones commented on code in PR #4140: URL: https://github.com/apache/hadoop/pull/4140#discussion_r858847636 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestConstants.java: ## @@ -108,6 +108,21 @@ */ String DEFAULT_REQUESTER_PAYS_FILE =

[GitHub] [hadoop] hadoop-yetus commented on pull request #4186: HDFS-16528. Reconfigure slow peer enable for Namenode

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1109875631 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762347=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762347 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4205: URL: https://github.com/apache/hadoop/pull/4205#issuecomment-1109860466 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762317=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762317 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4205: URL: https://github.com/apache/hadoop/pull/4205#issuecomment-1109822081 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] szilard-nemeth opened a new pull request, #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name

2022-04-26 Thread GitBox
szilard-nemeth opened a new pull request, #4235: URL: https://github.com/apache/hadoop/pull/4235 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=762312=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762312 ] ASF GitHub Bot logged work on HADOOP-18069: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1109809947 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18194) Public dataset class for S3A integration tests

2022-04-26 Thread Daniel Carl Jones (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528168#comment-17528168 ] Daniel Carl Jones commented on HADOOP-18194: [~ste...@apache.org], as you mention there are

[jira] [Work logged] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762299=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762299 ] ASF GitHub Bot logged work on HADOOP-18216: --- Author: ASF GitHub Bot

[GitHub] [hadoop] Hexiaoqiao commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero

2022-04-26 Thread GitBox
Hexiaoqiao commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-1109772315 Hi @steveloughran and @liever18 Sorry for late comments. I found that this pr just update comments, right? Is it better to improve the logic to guard the illegal configuration? Thanks.

[jira] [Work logged] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762276=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762276 ] ASF GitHub Bot logged work on HADOOP-18216: --- Author: ASF GitHub Bot

[GitHub] [hadoop] liever18 commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero

2022-04-26 Thread GitBox
liever18 commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-1109754227 > i am about to merge this. should i give the credit to Jingxuan Fu? yeap! "Jingxuan Fu" is my jira's account. -- This is an automated message from the Apache Git

[jira] [Work logged] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762273=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762273 ] ASF GitHub Bot logged work on HADOOP-18216: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero

2022-04-26 Thread GitBox
steveloughran commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-1109750693 i am about to merge this. should i give the credit to Jingxuan Fu? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762246 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4212: URL: https://github.com/apache/hadoop/pull/4212#issuecomment-1109697812 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762241=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762241 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4212: URL: https://github.com/apache/hadoop/pull/4212#issuecomment-1109686826 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762239=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762239 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] dannycjones commented on a diff in pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
dannycjones commented on code in PR #4205: URL: https://github.com/apache/hadoop/pull/4205#discussion_r858597582 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/prefetching.md: ## @@ -0,0 +1,192 @@ + + +# S3A Prefetching + +This document explains the

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762237=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762237 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4205: URL: https://github.com/apache/hadoop/pull/4205#issuecomment-1109670724 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762232=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762232 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4212: URL: https://github.com/apache/hadoop/pull/4212#issuecomment-1109660722 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4178: HDFS-16543. Keep default value of dfs.datanode.directoryscan.throttle.limit.ms.pe…

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4178: URL: https://github.com/apache/hadoop/pull/4178#issuecomment-1109647004 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762219=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762219 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4205: URL: https://github.com/apache/hadoop/pull/4205#issuecomment-1109635269 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762204=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762204 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] monthonk commented on a diff in pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
monthonk commented on code in PR #4212: URL: https://github.com/apache/hadoop/pull/4212#discussion_r858504016 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java: ## @@ -192,6 +197,12 @@ private boolean isObjectStreamOpen(FSDataInputStream

[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=762203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762203 ] ASF GitHub Bot logged work on HADOOP-18069: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1109576214 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762200=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762200 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] monthonk commented on a diff in pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
monthonk commented on code in PR #4212: URL: https://github.com/apache/hadoop/pull/4212#discussion_r858492522 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java: ## @@ -71,11 +72,16 @@ public void testRequesterPaysOptionSuccess() throws

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762199=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762199 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] monthonk commented on a diff in pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
monthonk commented on code in PR #4212: URL: https://github.com/apache/hadoop/pull/4212#discussion_r858489694 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java: ## @@ -192,6 +197,12 @@ private boolean isObjectStreamOpen(FSDataInputStream

[GitHub] [hadoop] hadoop-yetus commented on pull request #4104: HDFS-16520. Improve EC pread: avoid potential reading whole block

2022-04-26 Thread GitBox
hadoop-yetus commented on PR #4104: URL: https://github.com/apache/hadoop/pull/4104#issuecomment-1109548468 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18177) document use and architecture design of prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18177?focusedWorklogId=762184=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762184 ] ASF GitHub Bot logged work on HADOOP-18177: --- Author: ASF GitHub Bot

[GitHub] [hadoop] dannycjones commented on a diff in pull request #4205: HADOOP-18177. Document prefetching architecture.

2022-04-26 Thread GitBox
dannycjones commented on code in PR #4205: URL: https://github.com/apache/hadoop/pull/4205#discussion_r858455675 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/prefetching.md: ## @@ -14,94 +14,114 @@ # S3A Prefetching - This document explains the

[jira] [Work logged] (HADOOP-18175) test failures with prefetching s3a input stream

2022-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18175?focusedWorklogId=762181=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762181 ] ASF GitHub Bot logged work on HADOOP-18175: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ahmarsuhail commented on a diff in pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox
ahmarsuhail commented on code in PR #4212: URL: https://github.com/apache/hadoop/pull/4212#discussion_r858421027 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java: ## @@ -71,11 +72,16 @@ public void testRequesterPaysOptionSuccess()

[jira] [Assigned] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HADOOP-18216: Assignee: Jingxuan Fu > Ensure "io.file.buffer.size" is greater than zero. Otherwise,

[jira] [Commented] (HADOOP-18216) Ensure "io.file.buffer.size" is greater than zero. Otherwise, it will lead to data read/write blockage

2022-04-26 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17527943#comment-17527943 ] Xiaoqiao He commented on HADOOP-18216: -- [~fujx] Thanks for raising this issue. Add to contributor