[jira] [Comment Edited] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876991#comment-15876991 ] Mingliang Liu edited comment on HADOOP-13345 at 2/22/17 1:34 AM: -

[jira] [Updated] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14102: --- Attachment: HADOOP-14102.000.patch > Relax error message assertion in S3A test > ITestS3AEn

[jira] [Updated] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14102: --- Status: Patch Available (was: Open) > Relax error message assertion in S3A test > ITestS3A

[jira] [Updated] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14102: --- Priority: Minor (was: Trivial) > Relax error message assertion in S3A test > ITestS3AEncry

[jira] [Updated] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14102: --- Description: [HADOOP-13075] added support for SSE-KMS and SSE-C in s3a filesystem, along wi

[jira] [Created] (HADOOP-14102) Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey

2017-02-21 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14102: -- Summary: Relax error message assertion in S3A test ITestS3AEncryptionSSEC::testCreateFileAndReadWithDifferentEncryptionKey Key: HADOOP-14102 URL: https://issues.apache.org/jir

[jira] [Updated] (HADOOP-14099) Split s3 testing documentation out into its own file

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14099: --- Attachment: HADOOP-14099-branch-2.002.patch > Split s3 testing documentation out into its ow

[jira] [Updated] (HADOOP-14099) Split s3 testing documentation out into its own file

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14099: --- Attachment: HADOOP-14099.002.patch Hi Steve, When I backport this into {{trunk}} branch, I

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876991#comment-15876991 ] Mingliang Liu commented on HADOOP-13345: Hi Aaron, I updated from feature branch

[jira] [Commented] (HADOOP-14099) Split s3 testing documentation out into its own file

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876903#comment-15876903 ] Mingliang Liu commented on HADOOP-14099: +1 Will commit shortly (with whitespac

[jira] [Updated] (HADOOP-14017) User friendly name for ADLS user and group

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14017: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 S

[jira] [Commented] (HADOOP-14017) User friendly name for ADLS user and group

2017-02-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876777#comment-15876777 ] Mingliang Liu commented on HADOOP-14017: +1 Will commit shortly. > User friendl

[jira] [Commented] (HADOOP-13321) Deprecate FileSystem APIs that promote inefficient call patterns.

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872945#comment-15872945 ] Mingliang Liu commented on HADOOP-13321: Thanks Steve, the point is very valid.

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872936#comment-15872936 ] Mingliang Liu commented on HADOOP-13345: Thanks [~fabbri] for prompt reviewing t

[jira] [Updated] (HADOOP-13321) Deprecate FileSystem APIs that promote inefficient call patterns.

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13321: --- Attachment: HADOOP-13321.003.patch > Deprecate FileSystem APIs that promote inefficient call

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872859#comment-15872859 ] Mingliang Liu commented on HADOOP-13345: Thanks for your comments. The summary

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872640#comment-15872640 ] Mingliang Liu commented on HADOOP-13914: Thanks for helping me out (and the doc/

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872582#comment-15872582 ] Mingliang Liu commented on HADOOP-13914: [~fabbri] I came back to work this week

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871450#comment-15871450 ] Mingliang Liu commented on HADOOP-13345: I propose we merge from trunk again. I

[jira] [Updated] (HADOOP-14040) Use shaded aws-sdk uber-JAR 1.11.86

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14040: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Commented] (HADOOP-14040) Use shaded aws-sdk uber-JAR 1.11.86

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870963#comment-15870963 ] Mingliang Liu commented on HADOOP-14040: +1 Will commit shortly. > Use shaded

[jira] [Updated] (HADOOP-14019) fix some typos in the s3a docs

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14019: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Availa

[jira] [Commented] (HADOOP-14019) fix some typos in the s3a docs

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870953#comment-15870953 ] Mingliang Liu commented on HADOOP-14019: +1 > fix some typos in the s3a docs >

[jira] [Updated] (HADOOP-13321) Deprecate FileSystem APIs that promote inefficient call patterns.

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13321: --- Affects Version/s: 3.0.0-alpha3 Status: Patch Available (was: Open) > Deprec

[jira] [Updated] (HADOOP-13321) Deprecate FileSystem APIs that promote inefficient call patterns.

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13321: --- Attachment: HADOOP-13321.002.patch Thanks [~ste...@apache.org] for your review. I found some

[jira] [Updated] (HADOOP-14049) Honour AclBit flag associated to file/folder permission for Azure datalake account

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14049: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 S

[jira] [Commented] (HADOOP-14049) Honour AclBit flag associated to file/folder permission for Azure datalake account

2017-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870846#comment-15870846 ] Mingliang Liu commented on HADOOP-14049: +1 > Honour AclBit flag associated to

[jira] [Commented] (HADOOP-14087) S3A typo in pom.xml test exclusions

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869001#comment-15869001 ] Mingliang Liu commented on HADOOP-14087: +1 not clean Jenkins... is the patch b

[jira] [Updated] (HADOOP-14087) S3A typo in pom.xml test exclusions

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14087: --- Description: Noticed a copy/paste typo in hadoop-tools/hadoop-aws/pom.xml: {code:xml}

[jira] [Assigned] (HADOOP-13930) Azure: Add Authorization support to WASB

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-13930: -- Assignee: Sivaguru Sankaridurg (was: Dushyanth) > Azure: Add Authorization support t

[jira] [Commented] (HADOOP-13930) Azure: Add Authorization support to WASB

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868768#comment-15868768 ] Mingliang Liu commented on HADOOP-13930: Assigning to [~sisan...@microsoft.com].

[jira] [Updated] (HADOOP-14085) Drop unnecessary type assertion and cast

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14085: --- Target Version/s: HADOOP-13345 +1 Specially, after [HADOOP-13914], we could all use plain F

[jira] [Commented] (HADOOP-13945) Azure: Add Kerberos and Delegation token support to WASB client.

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868742#comment-15868742 ] Mingliang Liu commented on HADOOP-13945: Thanks for updating the patch. I'll rev

[jira] [Comment Edited] (HADOOP-14017) User friendly name for ADLS user and group

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868678#comment-15868678 ] Mingliang Liu edited comment on HADOOP-14017 at 2/15/17 10:19 PM:

[jira] [Commented] (HADOOP-14017) User friendly name for ADLS user and group

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868678#comment-15868678 ] Mingliang Liu commented on HADOOP-14017: {code} The patch only support ls and ge

[jira] [Commented] (HADOOP-14049) Honour AclBit flag associated to file/folder permission for Azure datalake account

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868605#comment-15868605 ] Mingliang Liu commented on HADOOP-14049: Thanks for working on this. The patch L

[jira] [Commented] (HADOOP-14084) Shell.joinThread swallows InterruptedException

2017-02-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868578#comment-15868578 ] Mingliang Liu commented on HADOOP-14084: Thanks for reporting. Can you provide a

[jira] [Commented] (HADOOP-14079) Fix breaking link in s3guard.md

2017-02-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864721#comment-15864721 ] Mingliang Liu commented on HADOOP-14079: Thanks for your prompt review. > Fix b

[jira] [Commented] (HADOOP-14079) Fix breaking link in s3guard.md

2017-02-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864637#comment-15864637 ] Mingliang Liu commented on HADOOP-14079: This is trivial change. Ping [~ste...@a

[jira] [Updated] (HADOOP-14079) Fix breaking link in s3guard.md

2017-02-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14079: --- Attachment: HADOOP-14079-HADOOP-13345.000.patch > Fix breaking link in s3guard.md >

[jira] [Updated] (HADOOP-14079) Fix breaking link in s3guard.md

2017-02-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14079: --- Status: Patch Available (was: Open) > Fix breaking link in s3guard.md > ---

[jira] [Created] (HADOOP-14079) Fix breaking link in s3guard.md

2017-02-13 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14079: -- Summary: Fix breaking link in s3guard.md Key: HADOOP-14079 URL: https://issues.apache.org/jira/browse/HADOOP-14079 Project: Hadoop Common Issue Type: Bug

[jira] [Commented] (HADOOP-13945) Azure: Add Kerberos and Delegation token support to WASB client.

2017-02-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864241#comment-15864241 ] Mingliang Liu commented on HADOOP-13945: Can you provide a new patch to get a cl

[jira] [Commented] (HADOOP-13930) Azure: Add Authorization support to WASB

2017-02-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864227#comment-15864227 ] Mingliang Liu commented on HADOOP-13930: Hi [~dchickabasapa], do you have update

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816491#comment-15816491 ] Mingliang Liu commented on HADOOP-13345: I re-visited the retry logic and think

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816423#comment-15816423 ] Mingliang Liu commented on HADOOP-13589: We can address this in a separate JIRA.

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816416#comment-15816416 ] Mingliang Liu commented on HADOOP-13589: Yes I think it makes sense to enable in

[jira] [Commented] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816224#comment-15816224 ] Mingliang Liu commented on HADOOP-13876: Thanks [~ste...@apache.org] for clarify

[jira] [Assigned] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-13876: -- Assignee: Mingliang Liu > S3Guard: better support for multi-bucket access including r

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816135#comment-15816135 ] Mingliang Liu commented on HADOOP-13336: {quote} the complaint about an unused i

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813374#comment-15813374 ] Mingliang Liu commented on HADOOP-13589: I like this patch. {code:title=s3guard

[jira] [Commented] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813310#comment-15813310 ] Mingliang Liu commented on HADOOP-13908: Thanks [~cnauroth], [~ste...@apache.org

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813276#comment-15813276 ] Mingliang Liu commented on HADOOP-13914: Thanks. I saw the doc, it's very helpfu

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13914: --- Attachment: HADOOP-13914-HADOOP-13345.000.patch I upload the v0 patch for discussion. > s3g

[jira] [Updated] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13876: --- Status: Patch Available (was: Open) > S3Guard: better support for multi-bucket access inclu

[jira] [Updated] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13876: --- Attachment: HADOOP-13876-HADOOP-13345.000.patch For the goals of this JIRA, the v0 patch is

[jira] [Commented] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812974#comment-15812974 ] Mingliang Liu commented on HADOOP-13908: This is a good test report as the failu

[jira] [Updated] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13908: --- Attachment: HADOOP-13908-HADOOP-13345.006.patch The v6 patch rebases from feature branch and

[jira] [Comment Edited] (HADOOP-13959) S3guard: replace dynamo.describe() call in init with more efficient query

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812667#comment-15812667 ] Mingliang Liu edited comment on HADOOP-13959 at 1/9/17 7:51 PM: --

[jira] [Commented] (HADOOP-13959) S3guard: replace dynamo.describe() call in init with more efficient query

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812667#comment-15812667 ] Mingliang Liu commented on HADOOP-13959: +1 for the proposal. Thanks [~ste...@a

[jira] [Commented] (HADOOP-13960) Initialize DynamoDBMetadataStore without associated S3AFileSystem

2017-01-09 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812647#comment-15812647 ] Mingliang Liu commented on HADOOP-13960: Thanks [~eddyxu] for discussion, review

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810285#comment-15810285 ] Mingliang Liu commented on HADOOP-13650: I have filed [HADOOP-13960]. Can you ha

[jira] [Updated] (HADOOP-13960) Initialize DynamoDBMetadataStore without associated S3AFileSystem

2017-01-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13960: --- Status: Patch Available (was: Open) > Initialize DynamoDBMetadataStore without associated S

[jira] [Updated] (HADOOP-13960) Initialize DynamoDBMetadataStore without associated S3AFileSystem

2017-01-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13960: --- Attachment: HADOOP-13960-HADOOP-13345.000.patch > Initialize DynamoDBMetadataStore without a

[jira] [Created] (HADOOP-13960) Initialize DynamoDBMetadataStore without associated S3AFileSystem

2017-01-08 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-13960: -- Summary: Initialize DynamoDBMetadataStore without associated S3AFileSystem Key: HADOOP-13960 URL: https://issues.apache.org/jira/browse/HADOOP-13960 Project: Hado

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806403#comment-15806403 ] Mingliang Liu commented on HADOOP-13345: Yeah, the two points both make sense to

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806381#comment-15806381 ] Mingliang Liu commented on HADOOP-13336: +1 Nice work! I'll hold on commit in

[jira] [Comment Edited] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806095#comment-15806095 ] Mingliang Liu edited comment on HADOOP-13908 at 1/6/17 11:10 PM: -

[jira] [Updated] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13908: --- Attachment: HADOOP-13908-HADOOP-13345.005.patch > S3Guard: Existing tables may not be initia

[jira] [Commented] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806095#comment-15806095 ] Mingliang Liu commented on HADOOP-13908: Chris and I have checked the test failu

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15805942#comment-15805942 ] Mingliang Liu commented on HADOOP-13650: {quote} In the current DynamoDBMetadata

[jira] [Updated] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13877: --- Attachment: HADOOP-13877-HADOOP-13345.002.patch In current unit test, the cases that are not

[jira] [Commented] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15805702#comment-15805702 ] Mingliang Liu commented on HADOOP-13931: Thanks [~cnauroth] for your review and

[jira] [Comment Edited] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804093#comment-15804093 ] Mingliang Liu edited comment on HADOOP-13908 at 1/6/17 9:25 AM: --

[jira] [Commented] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804093#comment-15804093 ] Mingliang Liu commented on HADOOP-13908: When I tested the v4 patch when the tab

[jira] [Comment Edited] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803082#comment-15803082 ] Mingliang Liu edited comment on HADOOP-13931 at 1/6/17 8:58 AM: --

[jira] [Comment Edited] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803742#comment-15803742 ] Mingliang Liu edited comment on HADOOP-13877 at 1/6/17 6:36 AM: --

[jira] [Commented] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803742#comment-15803742 ] Mingliang Liu commented on HADOOP-13877: Thanks [~fabbri] for attaching the exce

[jira] [Commented] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803082#comment-15803082 ] Mingliang Liu commented on HADOOP-13931: [~cnauroth], I can not reproduce the t

[jira] [Assigned] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-13914: -- Assignee: Mingliang Liu > s3guard: improve S3AFileStatus#isEmptyDirectory handling >

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802781#comment-15802781 ] Mingliang Liu commented on HADOOP-13650: I also commented in the parent thread,

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802751#comment-15802751 ] Mingliang Liu commented on HADOOP-13345: Hi [~eddyxu], You're right that in the

[jira] [Commented] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802679#comment-15802679 ] Mingliang Liu commented on HADOOP-13931: Thanks [~cnauroth] for review. The Jenk

[jira] [Commented] (HADOOP-13934) S3Guard: DynamoDBMetadataStore#move() could be throwing exception due to BatchWriteItem limits

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802559#comment-15802559 ] Mingliang Liu commented on HADOOP-13934: Thanks for testing this with Hive and r

[jira] [Updated] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13931: --- Status: Patch Available (was: Open) > S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore

[jira] [Commented] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799977#comment-15799977 ] Mingliang Liu commented on HADOOP-13908: When disabling the parallel test, the t

[jira] [Updated] (HADOOP-13761) S3Guard: implement retries

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13761: --- Affects Version/s: HADOOP-13345 > S3Guard: implement retries > ---

[jira] [Assigned] (HADOOP-13761) S3Guard: implement retries

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-13761: -- Assignee: Mingliang Liu > S3Guard: implement retries > --- >

[jira] [Comment Edited] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799289#comment-15799289 ] Mingliang Liu edited comment on HADOOP-13908 at 1/5/17 1:16 AM: --

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799916#comment-15799916 ] Mingliang Liu commented on HADOOP-13345: Hi [~fabbri], happy new year! The list

[jira] [Commented] (HADOOP-13945) Azure: Add Kerberos and Delegation token support to WASB client.

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799840#comment-15799840 ] Mingliang Liu commented on HADOOP-13945: Thanks for updating the patch. Looking

[jira] [Commented] (HADOOP-13934) S3Guard: DynamoDBMetadataStore#move() could be throwing exception due to BatchWriteItem limits

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799826#comment-15799826 ] Mingliang Liu commented on HADOOP-13934: Thanks [~ste...@apache.org]. [HADOOP-9

[jira] [Comment Edited] (HADOOP-13934) S3Guard: DynamoDBMetadataStore#move() could be throwing exception due to BatchWriteItem limits

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799350#comment-15799350 ] Mingliang Liu edited comment on HADOOP-13934 at 1/4/17 10:19 PM: -

[jira] [Updated] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13931: --- Attachment: HADOOP-13931-HADOOP-13345.001.patch The patch is based on [HADOOP-13934] V5 patc

[jira] [Updated] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13931: --- Description: Using {{batchWriteItem}} might be performant in {{DynamoDBMetadataStore#put(Dir

[jira] [Updated] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13931: --- Status: Open (was: Patch Available) > S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore

[jira] [Updated] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13931: --- Summary: S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put() (was: S3AGuard: Use Ba

[jira] [Updated] (HADOOP-13934) S3Guard: DynamoDBMetadataStore#move() could be throwing exception due to BatchWriteItem limits

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13934: --- Attachment: HADOOP-13934-HADOOP-13345.005.patch The v5 patch is based on v3 patch; and it fo

[jira] [Commented] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799289#comment-15799289 ] Mingliang Liu commented on HADOOP-13908: For the 2nd comment, yes v4 patch follo

[jira] [Updated] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

2017-01-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13908: --- Attachment: HADOOP-13908-HADOOP-13345.004.patch Thanks [~ste...@apache.org] for your comment

<    5   6   7   8   9   10   11   12   13   14   >