[jira] [Commented] (HADOOP-15696) KMS performance regression due to too many open file descriptors after Jetty migration

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597076#comment-16597076 ] Xiao Chen commented on HADOOP-15696: Thanks for the work here [~jojochuang] and [~m

[jira] [Comment Edited] (HADOOP-15698) KMS log4j is not initialized properly at startup

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597067#comment-16597067 ] Xiao Chen edited comment on HADOOP-15698 at 8/30/18 5:11 AM:

[jira] [Updated] (HADOOP-15698) KMS log4j is not initialized properly at startup

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HADOOP-15698: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.2 3.0.4

[jira] [Updated] (HADOOP-15698) KMS log4j is not initialized properly at startup

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HADOOP-15698: --- Summary: KMS log4j is not initialized properly at startup (was: KMS startup logs don't show) > KM

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597030#comment-16597030 ] Surendra Singh Lilhore commented on HADOOP-15684: - [~trjianjianjiao] :

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596968#comment-16596968 ] Da Zhou commented on HADOOP-15663: -- Attaching HADOOP-15663-HADOOP-15407-005.patch. The

[jira] [Updated] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15663: - Attachment: HADOOP-15663-HADOOP-15407-005.patch > ABFS: Simplify configuration > --

[jira] [Comment Edited] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596843#comment-16596843 ] Da Zhou edited comment on HADOOP-15663 at 8/30/18 1:11 AM: --- T

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596884#comment-16596884 ] Da Zhou commented on HADOOP-15407: -- Yes, [~sunilg], it will be ready. We are now worki

[jira] [Commented] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596856#comment-16596856 ] Daniel Templeton commented on HADOOP-15705: --- Filed HADOOP-15706 for the other

[jira] [Created] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD

2018-08-29 Thread Daniel Templeton (JIRA)
Daniel Templeton created HADOOP-15706: - Summary: Typo in compatibility doc: SHOUD -> SHOULD Key: HADOOP-15706 URL: https://issues.apache.org/jira/browse/HADOOP-15706 Project: Hadoop Common

[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.2

[jira] [Commented] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2018-08-29 Thread Steve Jacobs (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596852#comment-16596852 ] Steve Jacobs commented on HADOOP-15628: --- I think this is fixed in 3.1, at least f

[jira] [Comment Edited] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596843#comment-16596843 ] Da Zhou edited comment on HADOOP-15663 at 8/29/18 8:56 PM: --- T

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596843#comment-16596843 ] Da Zhou commented on HADOOP-15663: -- Thanks for the feedback. Attaching patch: HADOOP-1

[jira] [Updated] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15663: - Attachment: HADOOP-15663-HADOOP-15407-004.patch > ABFS: Simplify configuration > --

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Thomas Marquardt (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596835#comment-16596835 ] Thomas Marquardt commented on HADOOP-15663: --- LGTM too.  Some comments: Confi

[jira] [Commented] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596828#comment-16596828 ] Wei-Chiu Chuang commented on HADOOP-15705: -- +1 thanks for clarification [~temp

[jira] [Commented] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596817#comment-16596817 ] Aaron Fabbri commented on HADOOP-14630: --- +1 on the patch as soon as you get a gre

[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Status: Patch Available (was: Open) > Typo in the definition of "stable" in the inte

[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Attachment: HADOOP-15705.001.patch > Typo in the definition of "stable" in the interf

[jira] [Created] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
Daniel Templeton created HADOOP-15705: - Summary: Typo in the definition of "stable" in the interface classification Key: HADOOP-15705 URL: https://issues.apache.org/jira/browse/HADOOP-15705 Projec

[jira] [Commented] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596784#comment-16596784 ] Aaron Fabbri commented on HADOOP-15107: --- +1 on the v4 patch. Code all looks good.

[jira] [Commented] (HADOOP-10948) SwiftNativeFileSystem's directory is incompatible with Swift and Horizon

2018-08-29 Thread Shyamsundar Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596780#comment-16596780 ] Shyamsundar Ramanathan commented on HADOOP-10948: - * We applied the pat

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread JIRA
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596704#comment-16596704 ] Íñigo Goiri commented on HADOOP-15684: -- I'll follow up in HDFS-13805 for the {{Tes

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread Rong Tang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596696#comment-16596696 ] Rong Tang commented on HADOOP-15684: [~elgoiri]    For test case TestJournalNodeSyn

[jira] [Commented] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596683#comment-16596683 ] Aaron Fabbri commented on HADOOP-15107: --- I don't want to rob others of the joys o

[jira] [Commented] (HADOOP-15658) Memory leak in S3AOutputStream

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596667#comment-16596667 ] Steve Loughran commented on HADOOP-15658: - I've just moved this to being a 3.3

[jira] [Resolved] (HADOOP-15022) s3guard IT tests increase R/W capacity of the test table by 1

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15022. - Resolution: Duplicate > s3guard IT tests increase R/W capacity of the test table by 1 >

[jira] [Updated] (HADOOP-15658) Memory leak in S3AOutputStream

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15658: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15620 > Memory leak in S3AOutputS

[jira] [Commented] (HADOOP-15226) Über-JIRA: S3Guard Phase III: Hadoop 3.2 features

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596665#comment-16596665 ] Steve Loughran commented on HADOOP-15226: - For people watching this: just moved

[jira] [Updated] (HADOOP-15667) FileSystemMultipartUploader should verify that UploadHandle has non-0 length

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15667: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15620 > FileSystemMultipartUpload

[jira] [Updated] (HADOOP-15667) FileSystemMultipartUploader should verify that UploadHandle has non-0 length

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15667: Affects Version/s: 3.2.0 > FileSystemMultipartUploader should verify that UploadHandle ha

[jira] [Updated] (HADOOP-15667) FileSystemMultipartUploader should verify that UploadHandle has non-0 length

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15667: Component/s: fs/s3 > FileSystemMultipartUploader should verify that UploadHandle has non-

[jira] [Updated] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15426: Attachment: HADOOP-15426-007.patch > Make S3guard client resilient to DDB throttle events

[jira] [Updated] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15426: Status: Patch Available (was: Open) > Make S3guard client resilient to DDB throttle even

[jira] [Updated] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15426: Status: Open (was: Patch Available) > Make S3guard client resilient to DDB throttle even

[jira] [Updated] (HADOOP-14159) Add some Java-8 friendly way to work with RemoteIterable, especially listings

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14159: Priority: Minor (was: Major) > Add some Java-8 friendly way to work with RemoteIterable,

[jira] [Updated] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14630: Status: Patch Available (was: Open) patch 004; rebased for trunk > Contract Tests to ve

[jira] [Updated] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14630: Attachment: HADOOP-14630-004.patch > Contract Tests to verify create, mkdirs and rename u

[jira] [Updated] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14630: Target Version/s: (was: 3.2.0) > Contract Tests to verify create, mkdirs and rename und

[jira] [Updated] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15628: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15220 > S3A Filesystem does not c

[jira] [Commented] (HADOOP-13278) S3AFileSystem mkdirs does not need to validate parent path components

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596636#comment-16596636 ] Steve Loughran commented on HADOOP-13278: - Moving to branch-3.3 As noted, we d

[jira] [Resolved] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14483. - Resolution: Won't Fix > increase default value of fs.s3a.multipart.size to 128M > -

[jira] [Updated] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14483: Parent Issue: HADOOP-15220 (was: HADOOP-15620) > increase default value of fs.s3a.multip

[jira] [Commented] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596621#comment-16596621 ] Steve Loughran commented on HADOOP-14483: - I'm going to close as WONTFIX. There

[jira] [Updated] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14483: Target Version/s: (was: 3.3.0) > increase default value of fs.s3a.multipart.size to 128

[jira] [Commented] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596618#comment-16596618 ] Steve Loughran commented on HADOOP-15107: - I want this to go in to Hadoop 3.2;

[jira] [Updated] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15107: Priority: Blocker (was: Major) > Stabilize/tune S3A committers; review correctness & doc

[jira] [Updated] (HADOOP-14833) Remove s3a user:secret authentication

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14833: Target Version/s: (was: 3.2.0) > Remove s3a user:secret authentication > --

[jira] [Updated] (HADOOP-14468) S3Guard: make short-circuit getFileStatus() configurable

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14468: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > S3Guard: make short-circuit getFileStat

[jira] [Updated] (HADOOP-15183) S3Guard store becomes inconsistent after partial failure of rename

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15183: Target Version/s: (was: 3.2.0) > S3Guard store becomes inconsistent after partial failu

[jira] [Updated] (HADOOP-15183) S3Guard store becomes inconsistent after partial failure of rename

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15183: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > S3Guard store becomes inconsistent afte

[jira] [Updated] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13936: Target Version/s: (was: 3.2.0) > S3Guard: DynamoDB can go out of sync with S3AFileSyste

[jira] [Updated] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13936: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > S3Guard: DynamoDB can go out of sync wi

[jira] [Updated] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15664: - Status: Patch Available (was: Open) > ABFS: Reduce test run time via parallelization and grouping > --

[jira] [Updated] (HADOOP-14158) Possible for modified configuration to leak into metadatastore in S3GuardTool

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14158: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > Possible for modified configuration to

[jira] [Commented] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596607#comment-16596607 ] Da Zhou commented on HADOOP-15664: -- Attaching HADOOP-15664-HADOOP-15407-002.patch: -

[jira] [Updated] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15664: - Attachment: HADOOP-15664-HADOOP-15407-002.patch > ABFS: Reduce test run time via parallelization and gr

[jira] [Commented] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596602#comment-16596602 ] Steve Loughran commented on HADOOP-15628: - [~steveatbat]: if you can do a patch

[jira] [Updated] (HADOOP-15244) s3guard uploads command to add a way to complete outstanding uploads

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15244: Parent Issue: HADOOP-15220 (was: HADOOP-15620) > s3guard uploads command to add a way to

[jira] [Updated] (HADOOP-15244) s3guard uploads command to add a way to complete outstanding uploads

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15244: Target Version/s: (was: 3.3.0) > s3guard uploads command to add a way to complete outst

[jira] [Updated] (HADOOP-14833) Remove s3a user:secret authentication

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14833: Parent: HADOOP-15620 (was: HADOOP-15220) > Remove s3a user:secret authentication > -

[jira] [Updated] (HADOOP-15409) S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15409: Parent: HADOOP-15620 (was: HADOOP-15220) > S3AFileSystem.verifyBucketExists to move to s

[jira] [Updated] (HADOOP-15651) NPE in S3AInputStream.read() in ITestS3AInconsistency.testOpenFailOnRead

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15651: Parent: HADOOP-15620 (was: HADOOP-15220) > NPE in S3AInputStream.read() in ITestS3AIncon

[jira] [Updated] (HADOOP-14833) Remove s3a user:secret authentication

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14833: Priority: Major (was: Blocker) > Remove s3a user:secret authentication > ---

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread JIRA
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596546#comment-16596546 ] Íñigo Goiri commented on HADOOP-15684: -- We would need a review from somebody famil

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Sunil Govindan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596532#comment-16596532 ] Sunil Govindan commented on HADOOP-15430: - Thanks [~ste...@apache.org]. > hado

[jira] [Commented] (HADOOP-15703) ABFS - Implement client-side throttling

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596527#comment-16596527 ] Steve Loughran commented on HADOOP-15703: - + it'd be good to get these metrics

[jira] [Assigned] (HADOOP-15703) ABFS - Implement client-side throttling

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-15703: --- Assignee: Sneha Varma > ABFS - Implement client-side throttling > ---

[jira] [Updated] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15430: Priority: Minor (was: Blocker) > hadoop fs -mkdir -p path-ending-with-slash/ fails with

[jira] [Updated] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15430: Status: Open (was: Patch Available) > hadoop fs -mkdir -p path-ending-with-slash/ fails

[jira] [Updated] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15430: Target Version/s: (was: 3.2.0) > hadoop fs -mkdir -p path-ending-with-slash/ fails with

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596520#comment-16596520 ] Steve Loughran commented on HADOOP-15430: - Lets downgrade this one; I can live

[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers

2018-08-29 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596518#comment-16596518 ] Vinayakumar B commented on HADOOP-15602: Hi [~daryn] , Wondering whether you g

[jira] [Commented] (HADOOP-15703) ABFS - Implement client-side throttling

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596519#comment-16596519 ] Steve Loughran commented on HADOOP-15703: - Looks good, excluding what I'm about

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596501#comment-16596501 ] Vishwajeet Dusane commented on HADOOP-15700: The scenario i am trying to ad

[jira] [Commented] (HADOOP-15676) Cleanup TestSSLHttpServer

2018-08-29 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596489#comment-16596489 ] Szilard Nemeth commented on HADOOP-15676: - Hey [~zsiegl]! Thanks for your comme

[jira] [Commented] (HADOOP-15676) Cleanup TestSSLHttpServer

2018-08-29 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596488#comment-16596488 ] Szilard Nemeth commented on HADOOP-15676: - Hey [~zsiegl]! Thanks for your comme

[jira] [Updated] (HADOOP-15676) Cleanup TestSSLHttpServer

2018-08-29 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated HADOOP-15676: Attachment: HADOOP-15676.002.patch > Cleanup TestSSLHttpServer >

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Thomas Marquardt (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596468#comment-16596468 ] Thomas Marquardt commented on HADOOP-15700: --- [~vishwajeet.dusane], who is goi

[jira] [Commented] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596445#comment-16596445 ] Steve Loughran commented on HADOOP-15664: - This is a really great speedup! Ever

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

2018-08-29 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596443#comment-16596443 ] Vishwajeet Dusane commented on HADOOP-15669: Thank you for clarification [~

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596442#comment-16596442 ] Steve Loughran commented on HADOOP-15663: - Looks good # the testing doc is goi

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596439#comment-16596439 ] Vishwajeet Dusane commented on HADOOP-15700: Precisely the scenario i was t

[jira] [Commented] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596437#comment-16596437 ] Steve Loughran commented on HADOOP-15702: - sounds good; all to easy to write te

[jira] [Updated] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15702: Component/s: fs/azure > ABFS: Increase timeout of ITestAbfsReadWriteAndSeek > ---

[jira] [Updated] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15702: Component/s: test > ABFS: Increase timeout of ITestAbfsReadWriteAndSeek > ---

[jira] [Updated] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15702: Affects Version/s: HADOOP-15407 > ABFS: Increase timeout of ITestAbfsReadWriteAndSeek > -

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596434#comment-16596434 ] Steve Loughran commented on HADOOP-15669: - (BTW: The other stores generally off

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596433#comment-16596433 ] Steve Loughran commented on HADOOP-15669: - bq. org.apache.hadoop.util.VersionIn

[jira] [Updated] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15700: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15407 > ABFS: Failure in OpenSSLP

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596430#comment-16596430 ] Steve Loughran commented on HADOOP-15700: - there's never any guarantee that a j

[jira] [Commented] (HADOOP-15697) memory leak in distcp run method

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596428#comment-16596428 ] Steve Loughran commented on HADOOP-15697: - Can you tag this JIRA with the hadoo

[jira] [Updated] (HADOOP-15697) memory leak in distcp run method

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15697: Component/s: tools/distcp > memory leak in distcp run method > --

[jira] [Commented] (HADOOP-15695) In wasbFS user group always set via core-site

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596424#comment-16596424 ] Steve Loughran commented on HADOOP-15695: - +[~tmarquardt] > In wasbFS user gro

[jira] [Commented] (HADOOP-15680) ITestNativeAzureFileSystemConcurrencyLive times out

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596417#comment-16596417 ] Steve Loughran commented on HADOOP-15680: - looks good. +1 > ITestNativeAzureF

[jira] [Commented] (HADOOP-15635) s3guard set-capacity command to fail fast if bucket is unguarded

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596397#comment-16596397 ] Steve Loughran commented on HADOOP-15635: - well, {{S3AFileSystem.hasMetadataSto

[jira] [Assigned] (HADOOP-14734) add option to tag DDB table(s) created

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-14734: --- Assignee: Abraham Fine (was: Gabor Bota) > add option to tag DDB table(s) created

[jira] [Commented] (HADOOP-15688) ABFS: InputStream wrapped in FSDataInputStream twice

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596378#comment-16596378 ] Steve Loughran commented on HADOOP-15688: - we should make supporting byte buffe

  1   2   >