[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275165#comment-15275165
]
Hadoop QA commented on HADOOP-13113:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
53s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 18s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 10s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 12s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 2s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12802806/HADOOP-13113.001.patch
|
| JIRA Issue | HADOOP-13113 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux 6fbbab01fa6e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275410#comment-15275410
]
Hadoop QA commented on HADOOP-13113:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 11s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 13s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 22s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12802848/HADOOP-13113.002.patch
|
| JIRA Issue | HADOOP-13113 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux ab94ac778226 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276944#comment-15276944
]
Chris Nauroth commented on HADOOP-13113:
One interesting finding from repeated parallel test runs is that I'm seeing a
greater occurrence of failures in
{{TestS3AContractRootDir#testListEmptyRootDirectory}}, despite the fact that
this is one of the test suites that I force to run in serial.
{code}
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.085 sec <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir
testListEmptyRootDirectory(org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir)
Time elapsed: 3.041 sec <<< FAILURE!
java.lang.AssertionError: Deleted file: unexpectedly found
s3a://cnauroth-test-aws-s3a/fork-2 as
S3AFileStatus{path=s3a://cnauroth-test-aws-s3a/fork-2; isDirectory=true;
modification_time=0; access_time=0; owner=; group=; permission=rwxrwxrwx;
isSymlink=false}
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertPathDoesNotExist(ContractTestUtils.java:706)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertDeleted(ContractTestUtils.java:566)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertDeleted(ContractTestUtils.java:544)
at
org.apache.hadoop.fs.contract.AbstractContractRootDirectoryTest.testListEmptyRootDirectory(AbstractContractRootDirectoryTest.java:134)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
I do see an intermittent failure of this test even without the parallel test
patch, so it's not something introduced by this patch. I think the test is
subject to failure because of the eventual consistency behavior of deletes
against S3. Reviewing debug logs, in the failure case, I get the "Deleting
fake empty directory" message, even though it's definitely not empty at this
point. If I put a retry loop around the test, sometimes it fails first and
then succeeds on the retry.
It might be interesting to explore if the order of operations in
{{S3AFileSystem#listStatus}} could be changed to help with this. Maybe the
common prefixes could be analyzed first before deciding the key represents a
fake empty directory. I'm not sure if it's completely solvable though, because
any kind of S3 listing modification is subject to eventual consistency effects.
I'm not planning to change anything related to this within the scope of the
current patch though.
> Enable parallel test execution for hadoop-aws.
> --
>
> Key: HADOOP-13113
> URL: https://issues.apache.org/jira/browse/HADOOP-13113
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch
>
>
> The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend
> most of their time blocked on network I/O with the S3 back-end, but they
> don't saturate the bandwidth of the NIC. We can improve overall execution
> time by enabling parallel test execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278034#comment-15278034 ] Steve Loughran commented on HADOOP-13113: - I think our tests generate race conditions more than most other bits of code do, as they are repeatedly working on the same bits of the objects store and trying to do lots of small operations. Looks like you are hitting the delayed-listing-consistency problem which S3 has —here on a delete, rather than an addition. why not have that test not fail immediately if the root dir isn't empty, but sleep and spin for a bit until the dir empties up. > Enable parallel test execution for hadoop-aws. > -- > > Key: HADOOP-13113 > URL: https://issues.apache.org/jira/browse/HADOOP-13113 > Project: Hadoop Common > Issue Type: Improvement > Components: test >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch > > > The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend > most of their time blocked on network I/O with the S3 back-end, but they > don't saturate the bandwidth of the NIC. We can improve overall execution > time by enabling parallel test execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278107#comment-15278107
]
Steve Loughran commented on HADOOP-13113:
-
other than that though, LGTM.
Do we need to rely on a new system property, or could we just rely on the test
class name being unique? that is {{this.getClass().getName()}} could provide
the path?
> Enable parallel test execution for hadoop-aws.
> --
>
> Key: HADOOP-13113
> URL: https://issues.apache.org/jira/browse/HADOOP-13113
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch
>
>
> The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend
> most of their time blocked on network I/O with the S3 back-end, but they
> don't saturate the bandwidth of the NIC. We can improve overall execution
> time by enabling parallel test execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279798#comment-15279798
]
Hadoop QA commented on HADOOP-13113:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
45s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 18s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} hadoop-tools/hadoop-aws: The patch generated 0 new
+ 1 unchanged - 10 fixed = 1 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 11s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 12s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 56s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803376/HADOOP-13113.003.patch
|
| JIRA Issue | HADOOP-13113 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux 4a30a38e7487 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281478#comment-15281478 ] Steve Loughran commented on HADOOP-13113: - +1 It's pretty painful to look at what you've had to do regarding test setup, but I don't see any other workaround. And JS is built into Java6+, so it'll work everywhere > Enable parallel test execution for hadoop-aws. > -- > > Key: HADOOP-13113 > URL: https://issues.apache.org/jira/browse/HADOOP-13113 > Project: Hadoop Common > Issue Type: Improvement > Components: test >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch, > HADOOP-13113.003.patch > > > The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend > most of their time blocked on network I/O with the S3 back-end, but they > don't saturate the bandwidth of the NIC. We can improve overall execution > time by enabling parallel test execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281481#comment-15281481 ] Steve Loughran commented on HADOOP-13113: - oh, can you tack in a couple of lines in the aws index about how to enable this? thanks > Enable parallel test execution for hadoop-aws. > -- > > Key: HADOOP-13113 > URL: https://issues.apache.org/jira/browse/HADOOP-13113 > Project: Hadoop Common > Issue Type: Improvement > Components: test >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch, > HADOOP-13113.003.patch > > > The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend > most of their time blocked on network I/O with the S3 back-end, but they > don't saturate the bandwidth of the NIC. We can improve overall execution > time by enabling parallel test execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282571#comment-15282571 ] Steve Loughran commented on HADOOP-13113: - LGTM +1 > Enable parallel test execution for hadoop-aws. > -- > > Key: HADOOP-13113 > URL: https://issues.apache.org/jira/browse/HADOOP-13113 > Project: Hadoop Common > Issue Type: Improvement > Components: test >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch, > HADOOP-13113.003.patch, HADOOP-13113.004.patch > > > The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend > most of their time blocked on network I/O with the S3 back-end, but they > don't saturate the bandwidth of the NIC. We can improve overall execution > time by enabling parallel test execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284700#comment-15284700 ] Steve Loughran commented on HADOOP-13113: - I should warn that I'm seeing errors here; I think inconsistencies may be surfing —but I don't see why that should happen with parallel tests. Maybe if writes are taking time in the background after a close(), the longer that write takes to complete, the more chance the next operation may fail —and with multiple threads trying to use the same network connection, that's generating the load > Enable parallel test execution for hadoop-aws. > -- > > Key: HADOOP-13113 > URL: https://issues.apache.org/jira/browse/HADOOP-13113 > Project: Hadoop Common > Issue Type: Improvement > Components: test >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 2.8.0 > > Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch, > HADOOP-13113.003.patch, HADOOP-13113.004.patch > > > The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend > most of their time blocked on network I/O with the S3 back-end, but they > don't saturate the bandwidth of the NIC. We can improve overall execution > time by enabling parallel test execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285053#comment-15285053
]
Chris Nauroth commented on HADOOP-13113:
bq. I should warn that I'm seeing errors here...
[~ste...@apache.org], here are my observations based on additional experiments.
# If I configure a dedicated bucket per file system (S3, S3N and S3A), then I
consistently get a full pass on parallel test runs. This was the setup I was
using when I developed the patch.
# All of my buckets are in AWS region US-west-2 (Oregon), which is very close
to my laptop physically.
# If I configure the same bucket to be used by the test suites for all 3 file
systems, then sometimes I see some flaky behavior. I don't know if this is due
to data problems, such as perhaps S3N tests leaving bits behind that confuse
S3A later, or if the increased activity on a shared bucket simply surfaces
eventual consistency problems with greater likelihood.
# One test run surfaced a bug, and I have submitted a patch for that on
HADOOP-13158.
# {{TestS3NContractRootDir#testListEmptyRootDirectory}} probably needs the same
retry loop that we put in place for the same test in {{TestS3AContractRootDir}}.
I'll continue to look for ways to stabilize this.
> Enable parallel test execution for hadoop-aws.
> --
>
> Key: HADOOP-13113
> URL: https://issues.apache.org/jira/browse/HADOOP-13113
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.8.0
>
> Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch,
> HADOOP-13113.003.patch, HADOOP-13113.004.patch
>
>
> The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend
> most of their time blocked on network I/O with the S3 back-end, but they
> don't saturate the bandwidth of the NIC. We can improve overall execution
> time by enabling parallel test execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286329#comment-15286329 ] Steve Loughran commented on HADOOP-13113: - I was only testing TestS3A*, so it's not the cross FS problem...something else > Enable parallel test execution for hadoop-aws. > -- > > Key: HADOOP-13113 > URL: https://issues.apache.org/jira/browse/HADOOP-13113 > Project: Hadoop Common > Issue Type: Improvement > Components: test >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 2.8.0 > > Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch, > HADOOP-13113.003.patch, HADOOP-13113.004.patch > > > The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend > most of their time blocked on network I/O with the S3 back-end, but they > don't saturate the bandwidth of the NIC. We can improve overall execution > time by enabling parallel test execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
