[jira] [Comment Edited] (HDFS-15217) Add more information to longest write/read lock held log

2020-12-08 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246225#comment-17246225 ] Toshihiro Suzuki edited comment on HDFS-15217 at 12/9/20, 1:07 AM:

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-12-08 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246225#comment-17246225 ] Toshihiro Suzuki commented on HDFS-15217: - Sound good. Thank you [~ahussein]. >

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-12-08 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245749#comment-17245749 ] Toshihiro Suzuki commented on HDFS-15217: - {quote} The ops information can be ret

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-12-08 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245740#comment-17245740 ] Toshihiro Suzuki commented on HDFS-15217: - Thank you [~ahussein]. Are we sure the

[jira] [Commented] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

2020-06-08 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128753#comment-17128753 ] Toshihiro Suzuki commented on HDFS-15386: - [~sodonnell] I created a PR for branch

[jira] [Commented] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

2020-06-05 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126647#comment-17126647 ] Toshihiro Suzuki commented on HDFS-15386: - [~sodonnell] Thank you for merging the

[jira] [Updated] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

2020-06-03 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15386: Description: When removing volumes, we need to invalidate all the blocks in the volumes. In

[jira] [Work started] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

2020-06-03 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15386 started by Toshihiro Suzuki. --- > ReplicaNotFoundException keeps happening in DN after removing multiple D

[jira] [Updated] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

2020-06-03 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15386: Description: When removing volumes, we need to invalidate all the blocks in the volumes. In

[jira] [Updated] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

2020-06-03 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15386: Summary: ReplicaNotFoundException keeps happening in DN after removing multiple DN's data d

[jira] [Created] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data direcotries

2020-06-03 Thread Toshihiro Suzuki (Jira)
Toshihiro Suzuki created HDFS-15386: --- Summary: ReplicaNotFoundException keeps happening in DN after removing multiple DN's data direcotries Key: HDFS-15386 URL: https://issues.apache.org/jira/browse/HDFS-15386

[jira] [Commented] (HDFS-15298) Fix the findbugs warnings introduced in HDFS-15217

2020-04-27 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093355#comment-17093355 ] Toshihiro Suzuki commented on HDFS-15298: - Thank you for reviewing and committing

[jira] [Updated] (HDFS-15298) Fix the findbugs warnings introduced in HDFS-15217

2020-04-23 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15298: Description: We need to fix the findbugs warnings introduced in HDFS-15217: https://builds.

[jira] [Updated] (HDFS-15298) Fix the findbugs warnings introduced in HDFS-15217

2020-04-23 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15298: Summary: Fix the findbugs warnings introduced in HDFS-15217 (was: FIx the findbugs warning

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-04-23 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091055#comment-17091055 ] Toshihiro Suzuki commented on HDFS-15217: - Filed: https://issues.apache.org/jira/

[jira] [Created] (HDFS-15298) FIx the findbugs warnings introduced in HDFS-15217

2020-04-23 Thread Toshihiro Suzuki (Jira)
Toshihiro Suzuki created HDFS-15298: --- Summary: FIx the findbugs warnings introduced in HDFS-15217 Key: HDFS-15298 URL: https://issues.apache.org/jira/browse/HDFS-15298 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-04-23 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091054#comment-17091054 ] Toshihiro Suzuki commented on HDFS-15217: - [~ayushtkn] I checked the Report again

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-04-23 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090933#comment-17090933 ] Toshihiro Suzuki commented on HDFS-15217: - [~ayushtkn] [~elgoiri] I was aware of

[jira] [Comment Edited] (HDFS-15217) Add more information to longest write/read lock held log

2020-04-13 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081309#comment-17081309 ] Toshihiro Suzuki edited comment on HDFS-15217 at 4/14/20, 1:10 AM:

[jira] [Commented] (HDFS-15217) Add more information to longest write/read lock held log

2020-04-11 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081309#comment-17081309 ] Toshihiro Suzuki commented on HDFS-15217: - I created a PR for this. After this pa

[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-04-06 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076935#comment-17076935 ] Toshihiro Suzuki commented on HDFS-15249: - Thank you for reviewing and committing

[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-04-06 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076924#comment-17076924 ] Toshihiro Suzuki commented on HDFS-15249: - [~weichiu] How do we proceed with this

[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071456#comment-17071456 ] Toshihiro Suzuki commented on HDFS-15249: - Currently, we already have a synchroni

[jira] [Comment Edited] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071455#comment-17071455 ] Toshihiro Suzuki edited comment on HDFS-15249 at 3/31/20, 4:37 AM:

[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071455#comment-17071455 ] Toshihiro Suzuki commented on HDFS-15249: - [~weichiu] Thank you for taking look a

[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17070784#comment-17070784 ] Toshihiro Suzuki commented on HDFS-15249: - CC: [~arp] [~weichiu] > ThrottledAsyn

[jira] [Updated] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15249: Description: ThrottledAsyncChecker should be thread-safe because it can be used by multiple

[jira] [Created] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.

2020-03-30 Thread Toshihiro Suzuki (Jira)
Toshihiro Suzuki created HDFS-15249: --- Summary: ThrottledAsyncChecker is not thread-safe. Key: HDFS-15249 URL: https://issues.apache.org/jira/browse/HDFS-15249 Project: Hadoop HDFS Issue Typ

[jira] [Resolved] (HDFS-14503) ThrottledAsyncChecker throws NPE during block pool initialization

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HDFS-14503. - Resolution: Duplicate > ThrottledAsyncChecker throws NPE during block pool initialization

[jira] [Commented] (HDFS-14503) ThrottledAsyncChecker throws NPE during block pool initialization

2020-03-30 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17070738#comment-17070738 ] Toshihiro Suzuki commented on HDFS-14503: - It looks like duplicated by https://is

[jira] [Commented] (HDFS-15215) The Timestamp for longest write/read lock held log is wrong

2020-03-10 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056609#comment-17056609 ] Toshihiro Suzuki commented on HDFS-15215: - [~elgoiri] Thank you for reviewing thi

[jira] [Created] (HDFS-15217) Add more information to longest write/read lock held log

2020-03-10 Thread Toshihiro Suzuki (Jira)
Toshihiro Suzuki created HDFS-15217: --- Summary: Add more information to longest write/read lock held log Key: HDFS-15217 URL: https://issues.apache.org/jira/browse/HDFS-15217 Project: Hadoop HDFS

[jira] [Created] (HDFS-15215) The Timestamp for longest write/read lock held log is wrong

2020-03-10 Thread Toshihiro Suzuki (Jira)
Toshihiro Suzuki created HDFS-15215: --- Summary: The Timestamp for longest write/read lock held log is wrong Key: HDFS-15215 URL: https://issues.apache.org/jira/browse/HDFS-15215 Project: Hadoop HDFS

[jira] [Updated] (HDFS-15018) DataNode doesn't shutdown although the number of failed disks reaches dfs.datanode.failed.volumes.tolerated

2019-11-26 Thread Toshihiro Suzuki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15018: Description: In our case, we set dfs.datanode.failed.volumes.tolerated=0 but a DataNode di

[jira] [Created] (HDFS-15018) DataNode doesn't shutdown although the number of failed disks reaches dfs.datanode.failed.volumes.tolerated

2019-11-26 Thread Toshihiro Suzuki (Jira)
Toshihiro Suzuki created HDFS-15018: --- Summary: DataNode doesn't shutdown although the number of failed disks reaches dfs.datanode.failed.volumes.tolerated Key: HDFS-15018 URL: https://issues.apache.org/jira/brow

[jira] [Updated] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2019-03-06 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-14123: Attachment: HDFS-14123.01.patch > NameNode failover doesn't happen when running fsfreeze fo

[jira] [Updated] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2019-02-22 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-14123: Attachment: HDFS-14123.01.patch > NameNode failover doesn't happen when running fsfreeze fo

[jira] [Commented] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2018-12-04 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708902#comment-16708902 ] Toshihiro Suzuki commented on HDFS-14123: - I don't think the failures in the last

[jira] [Updated] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2018-12-04 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-14123: Attachment: HDFS-14123.01.patch > NameNode failover doesn't happen when running fsfreeze fo

[jira] [Commented] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2018-12-04 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708390#comment-16708390 ] Toshihiro Suzuki commented on HDFS-14123: - I attached a patch to check if the Nam

[jira] [Updated] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2018-12-04 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-14123: Status: Patch Available (was: Open) > NameNode failover doesn't happen when running fsfree

[jira] [Created] (HDFS-14123) NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir)

2018-12-04 Thread Toshihiro Suzuki (JIRA)
Toshihiro Suzuki created HDFS-14123: --- Summary: NameNode failover doesn't happen when running fsfreeze for the NameNode dir (dfs.namenode.name.dir) Key: HDFS-14123 URL: https://issues.apache.org/jira/browse/HDFS-

[jira] [Commented] (HDFS-13949) Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml

2018-10-10 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644571#comment-16644571 ] Toshihiro Suzuki commented on HDFS-13949: - Thank you for reviewing and committing

[jira] [Commented] (HDFS-13949) Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml

2018-10-09 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644253#comment-16644253 ] Toshihiro Suzuki commented on HDFS-13949: - Thank you very much for reviewing [~na

[jira] [Commented] (HDFS-13949) Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml

2018-10-01 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634843#comment-16634843 ] Toshihiro Suzuki commented on HDFS-13949: - I just attached a patch to correct the

[jira] [Updated] (HDFS-13949) Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml

2018-10-01 Thread Toshihiro Suzuki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-13949: Attachment: HDFS-13949.1.patch > Correct the description of dfs.datanode.disk.check.timeout

[jira] [Created] (HDFS-13949) Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml

2018-10-01 Thread Toshihiro Suzuki (JIRA)
Toshihiro Suzuki created HDFS-13949: --- Summary: Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml Key: HDFS-13949 URL: https://issues.apache.org/jira/browse/HDFS-13949 Pr