[
https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488778#comment-16488778
]
Steve Loughran commented on HADOOP-14918:
-----------------------------------------
I'm not quite sure how we are meant to use this, but it broke all my other
tests by deleting the ddb table in a parallel run
{code}
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
16.016 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEC
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore
[ERROR] Tests run: 38, Failures: 0, Errors: 36, Skipped: 0, Time elapsed:
179.425 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore
[ERROR]
testBatchWrite(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)
Time elapsed: 100.009 s <<< ERROR!
java.lang.Exception: test timed out after 100000 milliseconds
at java.lang.Thread.sleep(Native Method)
at
com.amazonaws.waiters.FixedDelayStrategy.delayBeforeNextRetry(FixedDelayStrategy.java:45)
at
com.amazonaws.waiters.WaiterExecution.safeCustomDelay(WaiterExecution.java:118)
at
com.amazonaws.waiters.WaiterExecution.pollResource(WaiterExecution.java:82)
at com.amazonaws.waiters.WaiterImpl.run(WaiterImpl.java:88)
at
com.amazonaws.services.dynamodbv2.document.Table.waitForDelete(Table.java:502)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.destroy(DynamoDBMetadataStore.java:798)
at
org.apache.hadoop.fs.s3a.s3guard.MetadataStoreTestBase.tearDown(MetadataStoreTestBase.java:129)
at
org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore.tearDown(ITestDynamoDBMetadataStore.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[ERROR]
testProvisionTable(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)
Time elapsed: 0.366 s <<< ERROR!
java.io.FileNotFoundException: DynamoDB table 'hwdev-steve-ireland-new' is
being deleted in region eu-west-1
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:908)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:292)
at
org.apache.hadoop.fs.s3a.s3guard.MetadataStoreTestBase.setUp(MetadataStoreTestBase.java:121)
at
org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore.setUp(ITestDynamoDBMetadataStore.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
> remove the Local Dynamo DB test option
> --------------------------------------
>
> Key: HADOOP-14918
> URL: https://issues.apache.org/jira/browse/HADOOP-14918
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.9.0, 3.0.0
> Reporter: Steve Loughran
> Assignee: Gabor Bota
> Priority: Major
> Attachments: HADOOP-14918-001.patch, HADOOP-14918-002.patch,
> HADOOP-14918-003.patch, HADOOP-14918-004.patch
>
>
> I'm going to propose cutting out the localdynamo test option for s3guard
> * the local DDB JAR is unmaintained/lags the SDK We work with...eventually
> there'll be differences in API.
> * as the local dynamo DB is unshaded. it complicates classpath setup for the
> build. Remove it and there's no need to worry about versions of anything
> other than the shaded AWS
> * it complicates test runs. Now we need to test for both localdynamo *and*
> real dynamo
> * but we can't ignore real dynamo, because that's the one which matters
> While the local option promises to reduce test costs, really, it's just
> adding complexity. If you are testing with s3guard, you need to have a real
> table to test against., And with the exception of those people testing s3a
> against non-AWS, consistent endpoints, everyone should be testing with
> S3Guard.
> -Straightforward to remove.-
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
