[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082108#comment-17082108 ] Masatake Iwasaki commented on HDFS-15048: - +1 on the HDFS-15048-branch-2.10.002.patch. > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Fix For: 3.3.0, 2.10.1 > > Attachments: HDFS-15048-branch-2.10.002.patch, HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082085#comment-17082085 ] Masatake Iwasaki commented on HDFS-15048: - [~seanlook] I'm going to commit both HDFS-14476-branch-2.10.02.patch and HDFS-15048-branch-2.10.002.patch after testing on my local. Thanks. > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Fix For: 3.3.0, 2.10.1 > > Attachments: HDFS-15048-branch-2.10.002.patch, HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082046#comment-17082046 ] Sean Chow commented on HDFS-15048: -- Hi [~iwasakims] , thanks for this patch for resolving HDFS-14476 . I think the patch also need to commit to branch-2.10. I've added the patch though it's exact the same with your orignal one. Hope it can be committed this week, as HDFS-14476 may can't catch the Hadoop 3.3.0 release. Sorry for the trouble.:P > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Fix For: 3.3.0, 2.10.1 > > Attachments: HDFS-15048-branch-2.10.002.patch, HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997393#comment-16997393 ] Takanobu Asanuma commented on HDFS-15048: - Thanks, [~iwasakims], [~ayushtkn] and [~weichiu]! > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997133#comment-16997133 ] Hudson commented on HDFS-15048: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17766 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17766/]) HDFS-15048. Fix findbug in DirectoryScanner. (iwasakims: rev dc6cf17b3405a5f03b75b1f7bf3b9e79663deaf1) * (edit) hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Attachments: HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997121#comment-16997121 ] Ayush Saxena commented on HDFS-15048: - +1 > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Attachments: HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995410#comment-16995410 ] Hadoop QA commented on HDFS-15048: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 43s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 36s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 17s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 31m 41s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:e573ea49085 | | JIRA Issue | HDFS-15048 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12988758/HDFS-15048.001.patch | | Optional Tests | dupname asflicense xml | | uname | Linux 2661c2f0cf0f 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 65c4660 | | maven | version: Apache Maven 3.3.9 | | Max. process+thread count | 346 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/28522/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Attachments: HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995388#comment-16995388 ] Masatake Iwasaki commented on HDFS-15048: - {quote}it is ok since there is supposed to be just one DirectoryScanner in a DataNode except for a test case. {quote} Sure. The problems is that {{DataNodeTestUtils#runDirectoryScanner}} calls {{DirectoryScanner#reconcile}} from outside. I'd not like to update the DirectoryScanner a lot just for test utils. Suppressing the warning should be enough. > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > Attachments: HDFS-15048.001.patch > > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15048) Fix findbug in DirectoryScanner
[ https://issues.apache.org/jira/browse/HDFS-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995244#comment-16995244 ] Wei-Chiu Chuang commented on HDFS-15048: It's a bad practice (blame me for not catching it) but from a correctness stand point it is ok since there is supposed to be just one DirectoryScanner in a DataNode except for a test case. It is good to fix it though. > Fix findbug in DirectoryScanner > --- > > Key: HDFS-15048 > URL: https://issues.apache.org/jira/browse/HDFS-15048 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Masatake Iwasaki >Priority: Major > > There is a findbug in DirectoryScanner. > {noformat} > Multithreaded correctness Warnings > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() calls > Thread.sleep() with a lock held > Bug type SWL_SLEEP_WITH_LOCK_HELD (click for details) > In class org.apache.hadoop.hdfs.server.datanode.DirectoryScanner > In method org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile() > At DirectoryScanner.java:[line 441] > {noformat} > https://builds.apache.org/job/PreCommit-HDFS-Build/28498/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org