[
https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Mackrory updated HADOOP-14068:
-----------------------------------
Attachment: HADOOP-14068-HADOOP-13345.008.patch
I had been back to this, but after some previous refactoring it was more
difficult to override everything I needed to, and then some tests were failing
for unusual reasons. I now have all tests passing. Attaching a patch that does
a few things:
- Does a bit of refactoring in TestDynamoDBMetadataStore so the parts that deal
with the local instance can be overridden, allowing the test to fall back on
the integration test configuration.
- Provides an ITestDynamoDBMetadataStore class that overrides such things. It
is not safe to run concurrently with other tests - so unfortunately it added
about 8 minutes to a test run for me.
- Fixed the I/O reprovisioning test: previously, state changes happened
immediately after reprovisioning. Now, you have to wait for the table to return
to the active state.
- Fixed testInitialization to not assume the bucket and table names match.
My main concern is the time it takes. I wouldn't be opposed to hiding this
behind the -Dscale option, although technically it's not a scale test. It is a
*big* test, and is only useful for catching bugs not caught against the Local
version. Those have definitely existed before, and developing this test helped
me catch them, but I don't think the time is worth it for every little commit.
> Add integration test version of TestMetadataStore for DynamoDB
> --------------------------------------------------------------
>
> Key: HADOOP-14068
> URL: https://issues.apache.org/jira/browse/HADOOP-14068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Attachments: HADOOP-14068-HADOOP-13345.001.patch,
> HADOOP-14068-HADOOP-13345.002.patch, HADOOP-14068-HADOOP-13345.003.patch,
> HADOOP-14068-HADOOP-13345.004.patch, HADOOP-14068-HADOOP-13345.005.patch,
> HADOOP-14068-HADOOP-13345.006.patch, HADOOP-14068-HADOOP-13345.007.patch,
> HADOOP-14068-HADOOP-13345.008.patch
>
>
> I tweaked TestDynamoDBMetadataStore to run against the actual Amazon DynamoDB
> service (as opposed to the "local" edition). Several tests failed because of
> minor variations in behavior. I think the differences that are clearly
> possible are enough to warrant extending that class as an ITest (but
> obviously keeping the existing test so 99% of the the coverage remains even
> when not configured for actual DynamoDB usage).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
