[
https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802679#comment-15802679
]
Mingliang Liu commented on HADOOP-13931:
----------------------------------------
Thanks [~cnauroth] for review. The Jenkin report looks good.
Running the integration test against US-West-1 after rebasing from feature
branch:
{code}
$ mvn -Dtest=none -Dit.test='ITestS3A*' -q -Dscale clean verify
Results :
Failed tests:
ITestS3ADeleteFilesOneByOne>ITestS3ADeleteManyFiles.testBulkRenameAndDelete:102->Assert.assertEquals:542->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
expected:<2005> but was:<27>
ITestS3ADeleteManyFiles.testBulkRenameAndDelete:102->Assert.assertEquals:542->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
expected:<1042> but was:<109>
ITestS3ADirectoryPerformance.testListOperations:93->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
Files found in listFiles(recursive=true) created=48 directories and 6 files
listed=3 directories and 0 files expected:<6> but was:<0>
Tests in error:
ITestS3AAWSCredentialsProvider.testAnonymousProvider:133 » IO Failed to
instan...
ITestS3ACredentialsInURL.testInstantiateFromURL:86 » IO Failed to instantiate
...
ITestS3ACredentialsInURL.testInvalidCredentialsFail:127 » AccessDenied
s3a://m...
ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:669->FileSystemContractBaseTest.rename:525
» AWSServiceIO
ITestS3ADirectoryPerformance.testTimeToStatFile:163->timeToStatPath:182 »
FileNotFound
ITestS3ADirectoryPerformance.testTimeToStatNonEmptyDirectory:155->timeToStatPath:182
» FileNotFound
ITestS3ADirectoryPerformance.testTimeToStatEmptyDirectory:145->timeToStatPath:182
» FileNotFound
ITestS3AHugeFilesArrayBlocks>AbstractSTestS3AHugeFiles.test_100_renameHugeFile:370->AbstractSTestS3AHugeFiles.assumeHugeFileExists:287
» FileNotFound
Tests run: 321, Failures: 3, Errors: 8, Skipped: 8
{code}
I will find out why the scale tests are failing, or it's intermittent.
> S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()
> -----------------------------------------------------------
>
> Key: HADOOP-13931
> URL: https://issues.apache.org/jira/browse/HADOOP-13931
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Rajesh Balamohan
> Assignee: Mingliang Liu
> Priority: Minor
> Attachments: HADOOP-13931-HADOOP-13345.000.patch,
> HADOOP-13931-HADOOP-13345.001.patch
>
>
> Using {{batchWriteItem}} might be performant in
> {{DynamoDBMetadataStore#put(DirListingMetadata meta)}} and
> {{DynamoDBMetadataStore#put(PathMetadata meta)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
