[jira] [Commented] (HADOOP-13998) initial s3guard preview

2017-02-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877812#comment-15877812 ] Aaron Fabbri commented on HADOOP-13998: --- [~ste...@apache.org] probably merge in la

[jira] [Issue Comment Deleted] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14102: -- Comment: was deleted (was: Thanks for taking the time to do this [~liuml07]. Looks good to me

[jira] [Commented] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879118#comment-15879118 ] Aaron Fabbri commented on HADOOP-14102: --- Thanks for taking the time to do this [~l

[jira] [Commented] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879119#comment-15879119 ] Aaron Fabbri commented on HADOOP-14102: --- Thanks for taking the time to do this [~l

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-02-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879422#comment-15879422 ] Aaron Fabbri commented on HADOOP-13345: --- Works for me, thanks [~liuml07] > S3Guar

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.002.patch Attaching initial patch of my solution for thi

[jira] [Assigned] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13914: - Assignee: Aaron Fabbri (was: Mingliang Liu) > s3guard: improve S3AFileStatus#isEmptyDi

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887120#comment-15887120 ] Aaron Fabbri commented on HADOOP-13914: --- Currently running tests on a second versi

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.003.patch Attaching v3 patch. - Also fixes createFakeDi

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Status: Patch Available (was: Open) Tested in US-West-2: All unit and integration tests with

[jira] [Commented] (HADOOP-14130) Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888532#comment-15888532 ] Aaron Fabbri commented on HADOOP-14130: --- We discussed this in HADOOP-14090. We we

[jira] [Commented] (HADOOP-14130) Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888555#comment-15888555 ] Aaron Fabbri commented on HADOOP-14130: --- Also wanted to add: Thanks [~qmlmoon] fo

[jira] [Comment Edited] (HADOOP-14130) Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888555#comment-15888555 ] Aaron Fabbri edited comment on HADOOP-14130 at 2/28/17 5:54 PM: --

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888668#comment-15888668 ] Aaron Fabbri commented on HADOOP-13914: --- The findbugs warning is here: {code} t

[jira] [Commented] (HADOOP-14125) s3guard tool tests aren't isolated; can't run in parallel

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1513#comment-1513 ] Aaron Fabbri commented on HADOOP-14125: --- +1 LGTM. I will give it a test run when

[jira] [Commented] (HADOOP-14129) ITestS3ACredentialsInURL sometimes fails

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1522#comment-1522 ] Aaron Fabbri commented on HADOOP-14129: --- {code} +LOG.error("IMPL: " + impl); {

[jira] [Commented] (HADOOP-14130) Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889189#comment-15889189 ] Aaron Fabbri commented on HADOOP-14130: --- Thanks for the explanation [~liuml07]. I

[jira] [Comment Edited] (HADOOP-14130) Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889189#comment-15889189 ] Aaron Fabbri edited comment on HADOOP-14130 at 3/1/17 12:22 AM: --

[jira] [Commented] (HADOOP-14130) Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889201#comment-15889201 ] Aaron Fabbri commented on HADOOP-14130: --- Ok.. I read the rest of the patch.. I see

[jira] [Commented] (HADOOP-14036) S3Guard: intermittent duplicate item keys failure

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889260#comment-15889260 ] Aaron Fabbri commented on HADOOP-14036: --- Cool.. Haven't had time to dig into this

[jira] [Updated] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14068: -- Status: Patch Available (was: Open) > Add integration test version of TestMetadataStore for D

[jira] [Updated] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14068: -- Status: Open (was: Patch Available) > Add integration test version of TestMetadataStore for D

[jira] [Commented] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB

2017-02-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889322#comment-15889322 ] Aaron Fabbri commented on HADOOP-14068: --- +1, patch LGTM. I re-submitted patch her

[jira] [Commented] (HADOOP-14129) ITestS3ACredentialsInURL sometimes fails

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893177#comment-15893177 ] Aaron Fabbri commented on HADOOP-14129: --- +1 on v4 patch. One minor question: {co

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893332#comment-15893332 ] Aaron Fabbri commented on HADOOP-13914: --- Thanks for the review [~mackrorysd]. {qu

[jira] [Commented] (HADOOP-14094) Rethink S3GuardTool options

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893363#comment-15893363 ] Aaron Fabbri commented on HADOOP-14094: --- Ok.. I think we should fix the indent lev

[jira] [Created] (HADOOP-14144) s3guard: CLI import does not yield an empty diff.

2017-03-02 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-14144: - Summary: s3guard: CLI import does not yield an empty diff. Key: HADOOP-14144 URL: https://issues.apache.org/jira/browse/HADOOP-14144 Project: Hadoop Common

[jira] [Updated] (HADOOP-14144) s3guard: CLI diff non-empty after import on new table

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14144: -- Summary: s3guard: CLI diff non-empty after import on new table (was: s3guard: CLI import does

[jira] [Updated] (HADOOP-14144) s3guard: CLI diff non-empty after import on new table

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14144: -- Description: I expected the following steps to yield zero diff from `hadoop s3guard diff` com

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893755#comment-15893755 ] Aaron Fabbri commented on HADOOP-13914: --- Yep, those two failures are HADOOP-14129

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.004.patch Attaching v4 patch. Changes from previous pat

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15894806#comment-15894806 ] Aaron Fabbri commented on HADOOP-13914: --- Sigh.. I did run test-patch on this befor

[jira] [Commented] (HADOOP-14094) Rethink S3GuardTool options

2017-03-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15894814#comment-15894814 ] Aaron Fabbri commented on HADOOP-14094: --- {noformat} ``` -hadoop s3a import [-m UR

[jira] [Updated] (HADOOP-14094) Rethink S3GuardTool options

2017-03-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14094: -- Resolution: Fixed Fix Version/s: HADOOP-13345 Status: Resolved (was: Patch A

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.005.patch v5 patch addresses checkstyle warnings. > s3g

[jira] [Commented] (HADOOP-14129) ITestS3ACredentialsInURL sometimes fails

2017-03-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895214#comment-15895214 ] Aaron Fabbri commented on HADOOP-14129: --- [~liuml07] yes, v5 LGTM. +1 > ITestS3AC

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895268#comment-15895268 ] Aaron Fabbri commented on HADOOP-13914: --- Checkstyle is clean now. Compile failu

[jira] [Commented] (HADOOP-13736) Change PathMetadata to hold S3AFileStatus instead of FileStatus.

2017-03-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898237#comment-15898237 ] Aaron Fabbri commented on HADOOP-13736: --- Hi [~cnauroth]. Check out HADOOP-13914.

[jira] [Commented] (HADOOP-14027) Implicitly creating DynamoDB table ignores endpoint config

2017-03-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898508#comment-15898508 ] Aaron Fabbri commented on HADOOP-14027: --- +1 LGTM, assuming the tests are now passi

[jira] [Commented] (HADOOP-14145) Ensure GenericOptionParser is used for S3Guard CLI

2017-03-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898514#comment-15898514 ] Aaron Fabbri commented on HADOOP-14145: --- +1 on the patch. Can you describe what t

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898693#comment-15898693 ] Aaron Fabbri commented on HADOOP-13914: --- Thanks for the review [~liuml07]. Great

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900130#comment-15900130 ] Aaron Fabbri commented on HADOOP-13914: --- Yes [~rajesh.balamohan], I just noticed t

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.006.patch Attaching v6 patch. Changes from v5: - Use w

[jira] [Comment Edited] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900523#comment-15900523 ] Aaron Fabbri edited comment on HADOOP-13914 at 3/8/17 1:26 AM: ---

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.007.patch Attaching v7 patch. Same as v6 except removes

[jira] [Commented] (HADOOP-14145) Ensure GenericOptionParser is used for S3Guard CLI

2017-03-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902179#comment-15902179 ] Aaron Fabbri commented on HADOOP-14145: --- I'm ok with punting on automated testing

[jira] [Commented] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902191#comment-15902191 ] Aaron Fabbri commented on HADOOP-14154: --- Hi [~rajesh.balamohan]. Thanks for the p

[jira] [Commented] (HADOOP-14165) Add S3Guard.dirListingUnion in S3AFileSystem#listFiles, listLocatedStatus

2017-03-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902574#comment-15902574 ] Aaron Fabbri commented on HADOOP-14165: --- Dupe of HADOOP-13926? > Add S3Guard.dirL

[jira] [Commented] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904022#comment-15904022 ] Aaron Fabbri commented on HADOOP-14154: --- {quote} Would that isAuthoritative flag h

[jira] [Comment Edited] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904022#comment-15904022 ] Aaron Fabbri edited comment on HADOOP-14154 at 3/9/17 10:54 PM: --

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904055#comment-15904055 ] Aaron Fabbri commented on HADOOP-13914: --- Thanks for the review [~mackrorysd]. The

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: HADOOP-13914-HADOOP-13345.008.patch Attaching v8 patch.. Same as v7 except additio

[jira] [Updated] (HADOOP-14168) S3GuardTool tests should not run if S3Guard is not set up

2017-03-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14168: -- Attachment: HADOOP-14168-HADOOP-13345.003.patch Looks good to me. I removed the unused import

[jira] [Updated] (HADOOP-14168) S3GuardTool tests should not run if S3Guard is not set up

2017-03-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14168: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to HADOOP-13345 f

[jira] [Commented] (HADOOP-13811) s3a: getFileStatus fails with com.amazonaws.AmazonClientException: Failed to sanitize XML document destined for handler class

2017-03-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925084#comment-15925084 ] Aaron Fabbri commented on HADOOP-13811: --- Adding a related stack trace in case it h

[jira] [Assigned] (HADOOP-13811) s3a: getFileStatus fails with com.amazonaws.AmazonClientException: Failed to sanitize XML document destined for handler class

2017-03-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13811: - Assignee: Aaron Fabbri (was: Steve Loughran) > s3a: getFileStatus fails with com.amazo

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-03-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925119#comment-15925119 ] Aaron Fabbri commented on HADOOP-13345: --- {quote}What about the prefix "s3guard+" o

[jira] [Commented] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466764#comment-16466764 ] Aaron Fabbri commented on HADOOP-15420: --- Thank you for working on this issue [~gab

[jira] [Comment Edited] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466764#comment-16466764 ] Aaron Fabbri edited comment on HADOOP-15420 at 5/8/18 2:44 AM: ---

[jira] [Commented] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466798#comment-16466798 ] Aaron Fabbri commented on HADOOP-13649: --- Nice work [~gabor.bota], this looks good.

[jira] [Updated] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13649: -- Attachment: HADOOP-13649.003.patch > s3guard: implement time-based (TTL) expiry for LocalMetad

[jira] [Comment Edited] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466798#comment-16466798 ] Aaron Fabbri edited comment on HADOOP-13649 at 5/8/18 3:07 AM: ---

[jira] [Updated] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13649: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Availabl

[jira] [Updated] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-05-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15420: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Availabl

[jira] [Commented] (HADOOP-15356) Make HTTP timeout configurable in ADLS Connector

2018-05-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468245#comment-16468245 ] Aaron Fabbri commented on HADOOP-15356: --- Patch looks good to me, +1. One question

[jira] [Commented] (HADOOP-15356) Make HTTP timeout configurable in ADLS Connector

2018-05-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16469413#comment-16469413 ] Aaron Fabbri commented on HADOOP-15356: --- Thanks for adding the extra validation of

[jira] [Commented] (HADOOP-15460) S3A FS to add "s3a:no-existence-checks" to the builder file creation option set

2018-05-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16475164#comment-16475164 ] Aaron Fabbri commented on HADOOP-15460: --- Interesting. Any thoughts on mkdirs()?

[jira] [Commented] (HADOOP-15469) S3A directory committer commit job fails if _temporary directory created under dest

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478057#comment-16478057 ] Aaron Fabbri commented on HADOOP-15469: --- Interesting. The argument for this seems

[jira] [Assigned] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14946: - Assignee: Gabor Bota > S3Guard testPruneCommandCLI can fail > -

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478062#comment-16478062 ] Aaron Fabbri commented on HADOOP-14946: --- [~gabor.bota] do you mind looking at this

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478404#comment-16478404 ] Aaron Fabbri commented on HADOOP-15430: --- Great work here. Test code is very nice.

[jira] [Commented] (HADOOP-15469) S3A directory committer commit job fails if _temporary directory created under dest

2018-05-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479458#comment-16479458 ] Aaron Fabbri commented on HADOOP-15469: --- And even before change, files could arriv

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480961#comment-16480961 ] Aaron Fabbri commented on HADOOP-15480: --- I also reproduced this yesterday.  Apolog

[jira] [Commented] (HADOOP-15437) AzureBlobFS - Services

2018-05-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16484600#comment-16484600 ] Aaron Fabbri commented on HADOOP-15437: --- Thank you for the effort to split up the

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16486747#comment-16486747 ] Aaron Fabbri commented on HADOOP-15480: --- Looks good [~gabor.bota], glad the hint w

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16486803#comment-16486803 ] Aaron Fabbri commented on HADOOP-14946: --- Thank you for the patch [~gabor.bota]. +1

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-23 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488430#comment-16488430 ] Aaron Fabbri commented on HADOOP-14946: --- Looking at this again.. since I was able

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-23 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Attachment: HADOOP-14946.002.patch > S3Guard testPruneCommandCLI can fail > --

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-23 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488508#comment-16488508 ] Aaron Fabbri commented on HADOOP-14946: --- I hope you don't mind [~gabor.bota], I've

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16489671#comment-16489671 ] Aaron Fabbri commented on HADOOP-14946: --- Thanks guys. {quote}NTP logs? {quote} Att

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16489949#comment-16489949 ] Aaron Fabbri commented on HADOOP-15480: --- Thanks for the info [~ste...@apache.org]

[jira] [Commented] (HADOOP-15489) S3Guard to self update on directory listings of S3

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490281#comment-16490281 ] Aaron Fabbri commented on HADOOP-15489: --- Look at S3Guard#dirListingUnion(). We us

[jira] [Comment Edited] (HADOOP-15489) S3Guard to self update on directory listings of S3

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490281#comment-16490281 ] Aaron Fabbri edited comment on HADOOP-15489 at 5/25/18 6:08 AM: --

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490288#comment-16490288 ] Aaron Fabbri commented on HADOOP-14918: --- Thanks for making progress on this, [~gab

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.003.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when us

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491473#comment-16491473 ] Aaron Fabbri commented on HADOOP-15480: --- thanks for the update [~gabor.bota], good

[jira] [Comment Edited] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491473#comment-16491473 ] Aaron Fabbri edited comment on HADOOP-15480 at 5/26/18 2:38 AM: --

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Attachment: HADOOP-14946.003.patch > S3Guard testPruneCommandCLI can fail > -

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494099#comment-16494099 ] Aaron Fabbri commented on HADOOP-14946: --- I'll commit the v3 patch once yetus is c

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thank

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.004.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when u

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.005.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when u

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: (was: HADOOP-15480.005.patch) > AbstractS3GuardToolTestBase.testDiffCommand f

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.005.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when u

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494331#comment-16494331 ] Aaron Fabbri commented on HADOOP-15480: --- Thanks [~gabor.bota], and thank you for

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Availab

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-05-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495944#comment-16495944 ] Aaron Fabbri commented on HADOOP-14918: --- Shout if you need any help testing this

[jira] [Commented] (HADOOP-15507) Add MapReduce counters about EC bytes read

2018-06-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500916#comment-16500916 ] Aaron Fabbri commented on HADOOP-15507: --- +1 assuming you manually tested (i.e. yo

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-06-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503682#comment-16503682 ] Aaron Fabbri commented on HADOOP-15407: --- Thanks for the update. In terms of us ma

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-06-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503693#comment-16503693 ] Aaron Fabbri commented on HADOOP-15407: --- Thanks.. was still reading through lates

<    3   4   5   6   7   8   9   10   11   12   >