[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150575#comment-16150575 ] Steve Loughran commented on HADOOP-14068: - we should get on with this; marking it as a Scale test would make it clear its in the "slow test" category. Like you say, "big" tests > Add integration test version of TestMetadataStore for DynamoDB > -- > > Key: HADOOP-14068 > URL: https://issues.apache.org/jira/browse/HADOOP-14068 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Attachments: HADOOP-14068-HADOOP-13345.001.patch, > HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch, > HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch, > HADOOP-14068-HADOOP-13345.006.patch, HADOOP-14068-HADOOP-13345.007.patch, > HADOOP-14068-HADOOP-13345.008.patch > > > I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB > service (as opposed to the "local" edition). Several tests failed because of > minor variations in behavior. I think the differences that are clearly > possible are enough to warrant extending that class as an ITest (but > obviously keeping the existing test so 99% of the the coverage remains even > when not configured for actual DynamoDB usage). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063106#comment-16063106 ] Steve Loughran commented on HADOOP-14068: - where are we with this? Have you been back to it? > Add integration test version of TestMetadataStore for DynamoDB > -- > > Key: HADOOP-14068 > URL: https://issues.apache.org/jira/browse/HADOOP-14068 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Attachments: HADOOP-14068-HADOOP-13345.001.patch, > HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch, > HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch, > HADOOP-14068-HADOOP-13345.006.patch, HADOOP-14068-HADOOP-13345.007.patch > > > I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB > service (as opposed to the "local" edition). Several tests failed because of > minor variations in behavior. I think the differences that are clearly > possible are enough to warrant extending that class as an ITest (but > obviously keeping the existing test so 99% of the the coverage remains even > when not configured for actual DynamoDB usage). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890382#comment-15890382 ] Sean Mackrory commented on HADOOP-14068: Now conflicting with HADOOP-13130 changes. > Add integration test version of TestMetadataStore for DynamoDB > -- > > Key: HADOOP-14068 > URL: https://issues.apache.org/jira/browse/HADOOP-14068 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Attachments: HADOOP-14068-HADOOP-13345.001.patch, > HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch, > HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch, > HADOOP-14068-HADOOP-13345.006.patch > > > I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB > service (as opposed to the "local" edition). Several tests failed because of > minor variations in behavior. I think the differences that are clearly > possible are enough to warrant extending that class as an ITest (but > obviously keeping the existing test so 99% of the the coverage remains even > when not configured for actual DynamoDB usage). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889406#comment-15889406
]
Hadoop QA commented on HADOOP-14068:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} HADOOP-14068 does not apply to HADOOP-13345. Rebase required?
Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14068 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855283/HADOOP-14068-HADOOP-13345.006.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11738/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add integration test version of TestMetadataStore for DynamoDB
> --
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch,
> HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch,
> HADOOP-14068-HADOOP-13345.006.patch
>
>
> I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB
> service (as opposed to the "local" edition). Several tests failed because of
> minor variations in behavior. I think the differences that are clearly
> possible are enough to warrant extending that class as an ITest (but
> obviously keeping the existing test so 99% of the the coverage remains even
> when not configured for actual DynamoDB usage).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889322#comment-15889322 ] Aaron Fabbri commented on HADOOP-14068: --- +1, patch LGTM. I re-submitted patch here to kick Yetus.. Not sure it still applies cleanly though. > Add integration test version of TestMetadataStore for DynamoDB > -- > > Key: HADOOP-14068 > URL: https://issues.apache.org/jira/browse/HADOOP-14068 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Attachments: HADOOP-14068-HADOOP-13345.001.patch, > HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch, > HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch > > > I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB > service (as opposed to the "local" edition). Several tests failed because of > minor variations in behavior. I think the differences that are clearly > possible are enough to warrant extending that class as an ITest (but > obviously keeping the existing test so 99% of the the coverage remains even > when not configured for actual DynamoDB usage). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15883313#comment-15883313 ] Sean Mackrory commented on HADOOP-14068: There's a lot of things in those failures that are definitely unreleated. Yetus passed for me locally, and trying to rerun this seems to be hitting infra issues at the moment...: https://builds.apache.org/job/PreCommit-HADOOP-Build/11710/console > Add integration test version of TestMetadataStore for DynamoDB > -- > > Key: HADOOP-14068 > URL: https://issues.apache.org/jira/browse/HADOOP-14068 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Attachments: HADOOP-14068-HADOOP-13345.001.patch, > HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch, > HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch > > > I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB > service (as opposed to the "local" edition). Several tests failed because of > minor variations in behavior. I think the differences that are clearly > possible are enough to warrant extending that class as an ITest (but > obviously keeping the existing test so 99% of the the coverage remains even > when not configured for actual DynamoDB usage). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879515#comment-15879515
]
Hadoop QA commented on HADOOP-14068:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
40s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
48s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} hadoop-aws in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
12s{color} | {color:red} hadoop-aws in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 12s{color}
| {color:red} hadoop-aws in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
13s{color} | {color:red} hadoop-aws in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} hadoop-aws in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 14s{color}
| {color:red} hadoop-aws in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14068 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854093/HADOOP-14068-HADOOP-13345.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 55f3f563a7d8 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HADOOP-13345 / 4f7f2ba |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11692/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-aws.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11692/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11692/artifact/patchprocess/patch-compile-hadoop-tools_hadoop-aws.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11692/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-aws.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11692/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-aws.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11692/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-aws.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/1
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877093#comment-15877093
]
Hadoop QA commented on HADOOP-14068:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
46s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14068 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853823/HADOOP-14068-HADOOP-13345.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ce590fb0d4b3 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HADOOP-13345 / 2ff5812 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11679/testReport/ |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11679/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add integration test version of TestMetadataStore for DynamoDB
> --
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch,
> HADOOP-14068
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876895#comment-15876895
]
Aaron Fabbri commented on HADOOP-14068:
---
Looks good for the most part.. two things:
- Where did ITestDynamoDBMetadataStore.java go?
- I suggested resetting the interrupt flag on the thread following
InterruptedException, but I think the codebase consistently just rethrows
InterruptedIOException. We should follow that behavior IMO. e.g.:
{noformat}
} catch (InterruptedException e) {
LOG.error("Interrupted while reprovisioning I/O for table; " +
"may not have taken effect yet...");
throw new InterruptedIOException(""Interrupted while reprovisioning I/O
for table");
}
{noformat}
> Add integration test version of TestMetadataStore for DynamoDB
> --
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch
>
>
> I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB
> service (as opposed to the "local" edition). Several tests failed because of
> minor variations in behavior. I think the differences that are clearly
> possible are enough to warrant extending that class as an ITest (but
> obviously keeping the existing test so 99% of the the coverage remains even
> when not configured for actual DynamoDB usage).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876700#comment-15876700
]
Hadoop QA commented on HADOOP-14068:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
27s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14068 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853797/HADOOP-14068-HADOOP-13345.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 93b85369258d 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HADOOP-13345 / 2ff5812 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11678/testReport/ |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11678/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add integration test version of TestMetadataStore for DynamoDB
> --
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch
>
>
> I tweaked
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871890#comment-15871890 ] Steve Loughran commented on HADOOP-14068: - FWIW for the s3guard committer, I've seen diffs too, at least from clients of s3a+s3guard. For testing, it may just be best to switch to the real DDB endpoint everywhere. Removes any inconsistency (and tracking it down, working around, etc), and increases the likelihood that test runs will catch any transient failures due to network problems, so give us more stack traces to deal with. > Add integration test version of TestMetadataStore for DynamoDB > -- > > Key: HADOOP-14068 > URL: https://issues.apache.org/jira/browse/HADOOP-14068 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Attachments: HADOOP-14068-HADOOP-13345.001.patch, > HADOOP-14068-HADOOP-13345.002.patch > > > I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB > service (as opposed to the "local" edition). Several tests failed because of > minor variations in behavior. I think the differences that are clearly > possible are enough to warrant extending that class as an ITest (but > obviously keeping the existing test so 99% of the the coverage remains even > when not configured for actual DynamoDB usage). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870948#comment-15870948
]
Aaron Fabbri commented on HADOOP-14068:
---
Thanks for doing this. It is clear there is value in also running the
MetadataStore tests against the live service since it behaves differently.
{noformat}
@@ -733,12 +733,17 @@ void provisionTable(Long readCapacity, Long writeCapacity)
.withReadCapacityUnits(readCapacity)
.withWriteCapacityUnits(writeCapacity);
try {
- final ProvisionedThroughputDescription p =
- table.updateTable(toProvision).getProvisionedThroughput();
+ table.updateTable(toProvision).getProvisionedThroughput();
+ table.waitForActive();
+ final ProvisionedThroughputDescription p = table.getDescription()
+ .getProvisionedThroughput();
LOG.info("Provision table {} in region {}: readCapacityUnits={}, "
+ "writeCapacityUnits={}",
tableName, region, p.getReadCapacityUnits(),
p.getWriteCapacityUnits());
{noformat}
I assume this change is because the table moves out of the Active state after
changing throughput provisioning? So we wait for it to become Active before
proceeding? And I assume we never hit the issue with the local dynamoDB?
{noformat}
+} catch (InterruptedException e) {
+ LOG.error("Interrupted while reprovisioning I/O for table; " +
+ "may not have taken effect yet...");
{noformat}
Don't think we want to swallow this exception. How about resetting the
thread's interrupt flag then throwing an InterruptedIOException that wraps the
original?
{noformat}
conf.unset(Constants.S3GUARD_DDB_TABLE_NAME_KEY);
-conf.unset(Constants.S3GUARD_DDB_ENDPOINT_KEY);
+//conf.unset(Constants.S3GUARD_DDB_ENDPOINT_KEY);
try {
{noformat}
Just remove the line..
{noformat}
-try {
+/*try {
DynamoDBMetadataStore ddbms = new DynamoDBMetadataStore();
ddbms.initialize(conf);
fail("Should have failed because as the endpoint is not set!");
} catch (IllegalArgumentException ignored) {
-}
-// config endpoint
-conf.set(Constants.S3GUARD_DDB_ENDPOINT_KEY, ddbEndpoint);
-// config credentials
-conf.set(Constants.ACCESS_KEY, "dummy-access-key");
-conf.set(Constants.SECRET_KEY, "dummy-secret-key");
-conf.setBoolean(Constants.S3GUARD_DDB_TABLE_CREATE_KEY, true);
+}*/
+customizeConfigurationForDynamoDB(conf);
{noformat}
Ditto here.. looks like that code moved to the helper function.
> Add integration test version of TestMetadataStore for DynamoDB
> --
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch
>
>
> I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB
> service (as opposed to the "local" edition). Several tests failed because of
> minor variations in behavior. I think the differences that are clearly
> possible are enough to warrant extending that class as an ITest (but
> obviously keeping the existing test so 99% of the the coverage remains even
> when not configured for actual DynamoDB usage).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB
[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866021#comment-15866021
]
Hadoop QA commented on HADOOP-14068:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} HADOOP-14068 does not apply to HADOOP-13345. Rebase required?
Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14068 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852596/HADOOP-14068-HADOOP-13345.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11623/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add integration test version of TestMetadataStore for DynamoDB
> --
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch
>
>
> I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB
> service (as opposed to the "local" edition). Several tests failed because of
> minor variations in behavior. I think the differences that are clearly
> possible are enough to warrant extending that class as an ITest (but
> obviously keeping the existing test so 99% of the the coverage remains even
> when not configured for actual DynamoDB usage).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
