[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494342#comment-16494342 ] Hudson commented on HADOOP-14946: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14312 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14312/]) HADOOP-14946 S3Guard testPruneCommandCLI can fail. Contributed by Gabor (fabbri: rev 30284d020d36c502dad5bdbae61ec48e9dfe9f8c) * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/AbstractS3GuardToolTestBase.java > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch, > HADOOP-14946.003.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494195#comment-16494195 ] genericqa commented on HADOOP-14946: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 13s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 41s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 25s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 17s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 51m 44s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-14946 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12925619/HADOOP-14946.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 1106a914ff62 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / e3236a9 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_162 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14703/testReport/ | | Max. process+thread count | 471 (vs. ulimit of 1) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14703/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL:
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494136#comment-16494136 ] Gabor Bota commented on HADOOP-14946: - Thank you [~fabbri]! > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch, > HADOOP-14946.003.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494099#comment-16494099 ] Aaron Fabbri commented on HADOOP-14946: --- I'll commit the v3 patch once yetus is clean. (same as v2, except use max_prune_age + 2 instead of + 1 for sleep). This one has been stable in testing for me in US West 2. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch, > HADOOP-14946.003.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492645#comment-16492645 ] Gabor Bota commented on HADOOP-14946: - Hi [~ste...@apache.org], is there anything thats needs to be fixed with this, or it can be committed? > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490507#comment-16490507 ] Steve Loughran commented on HADOOP-14946: - bq. Attempt at a joke / me grasping at an explanation for your last stack trace. No worries, didn't know NTP logged much. FWIW, if I'd been testing in a VM, you can get massive clock jumps, but its hard to defend against. bq. Aside: Ideally we'd have a Timer or Ticker "fake time source" we could inject into S3AFileSystem and DynamoDBMS it exists, just needs to be integrated, somehow org.apache.hadoop.util.FakeTimer > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489671#comment-16489671 ] Aaron Fabbri commented on HADOOP-14946: --- Thanks guys. {quote}NTP logs? {quote} Attempt at a joke / me grasping at an explanation for your last stack trace. :) A bogged-down system should result in "too many items pruned", (since "fresh" files become "stale") not "not enough items pruned" which is what was different about your last stack trace. I could only reproduce the former. {quote}the other one would be to dynamically change the asserts {quote} Yeah. We could assert an OR of the two possibilities. That at least gives some sanity check. Not sure how to do one exact value without a race. Anyways, will probably just commit this for now since it appears to work, and revisit if it comes up again–unless you have other desires [~ste...@apache.org]. (Aside: Ideally we'd have a Timer or Ticker "fake time source" we could inject into S3AFileSystem and DynamoDBMS so we could precisely control time without doing this sleep stuff. Another day). > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489230#comment-16489230 ] Gabor Bota commented on HADOOP-14946: - Another idea would be to run the test again if it fails - let's say 3 times max - if the prune times out. But it would cause the test to last 3 times more in the worst case, and it could still fail in the end. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489204#comment-16489204 ] Gabor Bota commented on HADOOP-14946: - Thanks, and +1 (non-binding) for the patch [~fabbri]. Measuring the timeout like this is a good idea. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488771#comment-16488771 ] Steve Loughran commented on HADOOP-14946: - NTP logs? Just assume I'd overloaded the macbook's 16GB of RAM with an IDE whose heap was 6GB; swapping will kill perf enough. I like the skip if overload detected strategy; the other one would be to dynamically change the asserts within range: current tests out of range: expect a different prune count > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488567#comment-16488567 ] genericqa commented on HADOOP-14946: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 54s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 23s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 35s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 58m 1s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-14946 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12924887/HADOOP-14946.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 1bc9fb26dc5c 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 7a87add | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_162 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14686/testReport/ | | Max. process+thread count | 301 (vs. ulimit of 1) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14686/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL:
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488508#comment-16488508 ] Aaron Fabbri commented on HADOOP-14946: --- I hope you don't mind [~gabor.bota], I've attached a v2 patch that is stable for me even with extreme load on my system. Differences from v1: * Add a big comment to {{testPruneCommand()}} explaining what it is doing. * If time between creation of "fresh" file and completion of prune command is > max path age, just warn() and skip the problematic assertion. I can only reproduce this if I torture my system with 18 test threads and some unrelated benchmarks running to slow down my system. * Use a constant. [~gabor.bota] if you like this patch I suggest we commit this and keep an eye out for the other case (Steve's most recent stack trace). I do not have more time to debug it now, but I would probably re-review the prune CLI handling of "-seconds 1" and if that looks good, maybe look at [~ste...@apache.org]'s NTP logs for big adjustments during his test run?? Other ideas? > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488430#comment-16488430 ] Aaron Fabbri commented on HADOOP-14946: --- Looking at this again.. since I was able to reproduce an earlier case even with the patch. My last comment above matches earlier stack trace, but not Steve's most recent one (edited for clarity): {quote}testPruneCommand:201->AbstractS3GuardToolTestBase.assertMetastoreListingCount:214->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 *Pruned children count* [ /test/testPruneCommandCLI/fresh; isDirectory=false; modification_time=152649906*9374*;, /test/testPruneCommandCLI/*stale*; isDirectory=false; modification_time=152649906*6615*;] *expected:<1> but was:<2>* {quote} This is a "not enough items pruned" error, (The "pruned children count" wording is confusing.) and I don't have an explanation. In this case, either (A) {{sleep( x )}} slept < x seconds, or (B) there is a timekeeping error somewhere. However, the testPruneCommandCLI/*stale* file should have been pruned: Note the time delta between the stale and fresh file is 2759 msec (~2.8 sec). This implies the existing sleep(2 sec) did sleep long enough (we know stale is at least 2.8 sec old so prune should have caught it). This points towards either an issue with the CLI interpreting "-seconds 1", or a different clock source or something? I *was* able to reproduce the earlier case (too many things pruned because i purposely fork bombed my system) and added some code that skips the assertion when the test is taking too long to get to the prune command, e.g.: {quote}AbstractS3GuardToolTestBase.java:testPruneCommand(250)) - Skipping an assertion: Test running too slowly (2539 msec) {quote} > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486803#comment-16486803 ] Aaron Fabbri commented on HADOOP-14946: --- Thank you for the patch [~gabor.bota]. +1 LGTM. I believe this will make the failure less likely. I think the issue is this: calling sleep() , for example: {noformat} foo(); sleep(x); bar(); {noformat} This guarantees that bar() will execute *no sooner* than x seconds after foo(), but doesn't guarantee when bar() will run. It could be x+1, x+2, or x+10, depending on what your system is doing. This is consistent with the stack traces showing there were *too many* items pruned (because by the time prune() ran, more files had become stale). I'll commit this after running the tests a couple of times under external load (in us-west-2). (If this failure comes back again, we can use timers to at least detect when too much time has passed (get time before and after prune() call, if *too many* files were pruned we can tell from the timers if the issue is that your system was just too slow to run the test and ignore the failure with a LOG.warn(). Even better would be to add a Ticker class like the google cache stuff uses to make the code testable without depending on real time.. that would be more work though.) > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480742#comment-16480742 ] genericqa commented on HADOOP-14946: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 34s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 57s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 38s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 47s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 59m 10s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-14946 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12924120/HADOOP-14946.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 95b9b7f89c1d 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 6e99686 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_162 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14657/testReport/ | | Max. process+thread count | 334 (vs. ulimit of 1) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14657/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL:
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480650#comment-16480650 ] Gabor Bota commented on HADOOP-14946: - I've also corrected a javadoc with my patch in AbstractS3GuardToolTestBase. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-14946.001.patch > > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480647#comment-16480647 ] Gabor Bota commented on HADOOP-14946: - I think increasing the timeout will solve this issue. Please test it, it passes for me in {{eu-west-1}} with {{mvn -Dparallel-tests -DtestsThreadCount=8 clean verify -Ds3guard -Ddynamo}} > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479066#comment-16479066 ] Steve Loughran commented on HADOOP-14946: - That last stack may be a real regression > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478997#comment-16478997 ] Gabor Bota commented on HADOOP-14946: - Sure I'll start working on this shortly. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478062#comment-16478062 ] Aaron Fabbri commented on HADOOP-14946: --- [~gabor.bota] do you mind looking at this one? If not reassign to me. Thank you. > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477970#comment-16477970 ] Steve Loughran commented on HADOOP-14946: - seeing on trunk, even in standalone tests with my HADOOP-15430 patch in (its the thing I'm working on, see). This the problem with failing tests, you can't see if new patches are regressions or not. {code} [INFO] [ERROR] Failures: [ERROR] ITestS3GuardToolLocal>AbstractS3GuardToolTestBase.testPruneCommandCLI:221->AbstractS3GuardToolTestBase.testPruneCommand:201->AbstractS3GuardToolTestBase.assertMetastoreListingCount:214->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 Pruned children count [PathMetadata{fileStatus=S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test/testPruneCommandCLI/fresh; isDirectory=false; length=100; replication=1; blocksize=512; modification_time=1526499069374; access_time=0; owner=hdfs; group=hdfs; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE; isEmptyDirectory=UNKNOWN; isDeleted=false}, PathMetadata{fileStatus=S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test/testPruneCommandCLI/stale; isDirectory=false; length=100; replication=1; blocksize=512; modification_time=1526499066615; access_time=0; owner=hdfs; group=hdfs; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE; isEmptyDirectory=UNKNOWN; isDeleted=false}] expected:<1> but was:<2> [ERROR] ITestS3GuardToolLocal>AbstractS3GuardToolTestBase.testPruneCommandConf:230->AbstractS3GuardToolTestBase.testPruneCommand:201->AbstractS3GuardToolTestBase.assertMetastoreListingCount:214->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 Pruned children count [PathMetadata{fileStatus=S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test/testPruneCommandConf/fresh; isDirectory=false; length=100; replication=1; blocksize=512; modification_time=1526499076808; access_time=0; owner=hdfs; group=hdfs; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE; isEmptyDirectory=UNKNOWN; isDeleted=false}, PathMetadata{fileStatus=S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test/testPruneCommandConf/stale; isDirectory=false; length=100; replication=1; blocksize=512; modification_time=1526499074053; access_time=0; owner=hdfs; group=hdfs; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE; isEmptyDirectory=UNKNOWN; isDeleted=false}] expected:<1> but was:<2> [INFO] [ERROR] Tests run: 21, Failures: 2, Errors: 0, Skipped: 0 {code} > S3Guard testPruneCommandCLI can fail > > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail in parallel runs
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476014#comment-16476014 ] Steve Loughran commented on HADOOP-14946: - stil recurring when the thread count is >= core count. Maybe: pull this specific test out and run in the serial phase > S3Guard testPruneCommandCLI can fail in parallel runs > - > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail in parallel runs
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338394#comment-16338394 ] Steve Loughran commented on HADOOP-14946: - Seen again. Another possible cause: sleep() time too tight, prune CLI command isn't finding entries to prune {code} ERROR] testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) Time elapsed: 15.704 s <<< FAILURE! java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.assertMetastoreListingCount(AbstractS3GuardToolTestBase.java:210) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testPruneCommand(AbstractS3GuardToolTestBase.java:199) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testPruneCommandCLI(AbstractS3GuardToolTestBase.java:217) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) [ERROR] testPruneCommandConf(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) Time elapsed: 14.833 s <<< FAILURE! java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.assertMetastoreListingCount(AbstractS3GuardToolTestBase.java:210) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testPruneCommand(AbstractS3GuardToolTestBase.java:199) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testPruneCommandConf(AbstractS3GuardToolTestBase.java:226) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) [INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 405.046 s - in org.apache.hadoop.fs.s3a.commit.staging.integrati {code} > S3Guard testPruneCommandCLI can fail in parallel runs > - > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Priority: Major > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail in parallel runs
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202557#comment-16202557 ] Steve Loughran commented on HADOOP-14946: - {code} Tests run: 43, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 116.336 sec - in org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract Running org.apache.hadoop.fs.s3a.yarn.ITestS3A Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 93.582 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) Time elapsed: 10.765 sec <<< FAILURE! java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.assertMetastoreListingCount(AbstractS3GuardToolTestBase.java:210) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testPruneCommand(AbstractS3GuardToolTestBase.java:199) at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testPruneCommandCLI(AbstractS3GuardToolTestBase.java:217) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) {code} > S3Guard testPruneCommandCLI can fail in parallel runs > - > > Key: HADOOP-14946 > URL: https://issues.apache.org/jira/browse/HADOOP-14946 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0 >Reporter: Steve Loughran > > The test of the S3Guard CLI prune can sometimes fail on parallel test runs. > Assumption: it is the parallelism which is causing the problem > {code} > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB > testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB) > Time elapsed: 10.765 sec <<< FAILURE! > java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:88) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org