[jira] [Commented] (HADOOP-11394) hadoop-aws documentation missing.

[ https://issues.apache.org/jira/browse/HADOOP-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242967#comment-14242967 ] Juan Yu commented on HADOOP-11394: -- +1, thanks for catching this Chris. hadoop-aws

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199717#comment-14199717 ] Juan Yu commented on HADOOP-10714: -- Thanks [~dsw] for the initial patch,

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.010.patch Revert the rename change, and comment the S3N extra large file

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169729#comment-14169729 ] Juan Yu commented on HADOOP-10714: -- I tried it. with 1G file upload, the test takes ~20

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169731#comment-14169731 ] Juan Yu commented on HADOOP-10714: -- I also ran all other s3a, s3n tests and they

[jira] [Commented] (HADOOP-10400) Incorporate new S3A FileSystem implementation

[ https://issues.apache.org/jira/browse/HADOOP-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167063#comment-14167063 ] Juan Yu commented on HADOOP-10400: -- I ran the seek test against s3a and s3n many times

[jira] [Commented] (HADOOP-10400) Incorporate new S3A FileSystem implementation

[ https://issues.apache.org/jira/browse/HADOOP-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167064#comment-14167064 ] Juan Yu commented on HADOOP-10400: -- correction, the test is

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153400#comment-14153400 ] Juan Yu commented on HADOOP-10714: -- Why don't we just use one authentication property

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.008.patch Let s3a use {{S3Credentials}} to get authentication info, same as s3

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151236#comment-14151236 ] Juan Yu commented on HADOOP-10714: -- New patch looks good. +1. Thanks

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150881#comment-14150881 ] Juan Yu commented on HADOOP-10714: -- Thanks [~ste...@apache.org]. I've run the contract

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147851#comment-14147851 ] Juan Yu commented on HADOOP-10714: -- [~ste...@apache.org], any comment about the latest

[jira] [Created] (HADOOP-11128) abstracting out the scale tests for FileSystem Contract tests

Juan Yu created HADOOP-11128: Summary: abstracting out the scale tests for FileSystem Contract tests Key: HADOOP-11128 URL: https://issues.apache.org/jira/browse/HADOOP-11128 Project: Hadoop Common

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144829#comment-14144829 ] Juan Yu commented on HADOOP-10714: -- the failed test is the new rename test

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.004.patch Since we cannot change HDFS's rename behavior, I guess the same for

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.005.patch Thanks [~ste...@apache.org]. I added a new ContractOption for the

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.006.patch Same patch as previous to trigger Jenkins build.

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145884#comment-14145884 ] Juan Yu commented on HADOOP-10714: -- I don't think the failed test is related with this

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.003.patch Thanks Charles and Steve. here is a new patch to address all

[jira] [Commented] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143871#comment-14143871 ] Juan Yu commented on HADOOP-10714: -- I modified the new test

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.001.patch This patch contains the following changes: - Added scale test for

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Status: Patch Available (was: Open) AmazonS3Client.deleteObjects() need to be limited to 1000 entries

[jira] [Updated] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10714: - Attachment: HADOOP-10714.002.patch Thanks [~clamb] and [~atm] for code reviewing. new patch to address

[jira] [Assigned] (HADOOP-10714) AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call

[ https://issues.apache.org/jira/browse/HADOOP-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu reassigned HADOOP-10714: Assignee: Juan Yu (was: David S. Wang) AmazonS3Client.deleteObjects() need to be limited to

[jira] [Commented] (HADOOP-11068) Match hadoop.auth cookie format to jetty output

[ https://issues.apache.org/jira/browse/HADOOP-11068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127733#comment-14127733 ] Juan Yu commented on HADOOP-11068: -- +1 LGTM. Match hadoop.auth cookie format to jetty

[jira] [Commented] (HADOOP-11074) Move s3-related FS connector code to hadoop-aws

[ https://issues.apache.org/jira/browse/HADOOP-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127802#comment-14127802 ] Juan Yu commented on HADOOP-11074: -- +1, patch LGTM. I applied the patch and added the

[jira] [Commented] (HADOOP-11074) Move s3-related FS connector code to hadoop-aws

[ https://issues.apache.org/jira/browse/HADOOP-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128060#comment-14128060 ] Juan Yu commented on HADOOP-11074: -- This is the jackson dependency issue, upgrade

[jira] [Created] (HADOOP-11076) SbNN Web UI shows #Under replicated blocks and #pending deletion blocks

Juan Yu created HADOOP-11076: Summary: SbNN Web UI shows #Under replicated blocks and #pending deletion blocks Key: HADOOP-11076 URL: https://issues.apache.org/jira/browse/HADOOP-11076 Project: Hadoop

[jira] [Updated] (HADOOP-11076) SbNN Web UI shows #Under replicated blocks and #pending deletion blocks

[ https://issues.apache.org/jira/browse/HADOOP-11076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-11076: - Component/s: ha SbNN Web UI shows #Under replicated blocks and #pending deletion blocks

[jira] [Created] (HADOOP-11030) Should define a jackson.version property in the POM instead of using explicit version

Juan Yu created HADOOP-11030: Summary: Should define a jackson.version property in the POM instead of using explicit version Key: HADOOP-11030 URL: https://issues.apache.org/jira/browse/HADOOP-11030

[jira] [Updated] (HADOOP-11030) Should define a jackson.version property in the POM instead of using explicit version

[ https://issues.apache.org/jira/browse/HADOOP-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-11030: - Attachment: HADOOP-11030.patch Should define a jackson.version property in the POM instead of using

[jira] [Updated] (HADOOP-11030) Should define a jackson.version property in the POM instead of using explicit version

[ https://issues.apache.org/jira/browse/HADOOP-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-11030: - Status: Patch Available (was: Open) Should define a jackson.version property in the POM instead of

[jira] [Commented] (HADOOP-11030) Define a variable jackson.version instead of using constant at multiple places

[ https://issues.apache.org/jira/browse/HADOOP-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116024#comment-14116024 ] Juan Yu commented on HADOOP-11030: -- Thanks Karthik. Define a variable jackson.version

[jira] [Commented] (HADOOP-10451) Remove unused field and imports from SaslRpcServer

[ https://issues.apache.org/jira/browse/HADOOP-10451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111606#comment-14111606 ] Juan Yu commented on HADOOP-10451: -- This breaks downstream components. Hive and Hue

[jira] [Commented] (HADOOP-10373) move s3 logic to own replaceable jar, hadoop-aws

[ https://issues.apache.org/jira/browse/HADOOP-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075788#comment-14075788 ] Juan Yu commented on HADOOP-10373: -- Patch looks good, thanks for the work

[jira] [Commented] (HADOOP-9361) Strictly define the expected behavior of filesystem APIs and write tests to verify compliance

[ https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050584#comment-14050584 ] Juan Yu commented on HADOOP-9361: - Tested two swift provider, Rackspace and HP Cloud.

[jira] [Commented] (HADOOP-9361) Strictly define the expected behavior of filesystem APIs and write tests to verify compliance

[ https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049299#comment-14049299 ] Juan Yu commented on HADOOP-9361: - I just tested this against S3 with s3n. all tests

[jira] [Commented] (HADOOP-9361) Strictly define the expected behavior of filesystem APIs and write tests to verify compliance

[ https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047733#comment-14047733 ] Juan Yu commented on HADOOP-9361: - I applied it to trunk. Strictly define the expected

[jira] [Commented] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047736#comment-14047736 ] Juan Yu commented on HADOOP-10710: -- [~tucu], Jenkins passed now after removing {code}

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.007.patch [~tucu], thanks for the review, new patch uploaded. hadoop.auth

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.006.patch remove isHttpOnly() hadoop.auth cookie is not properly

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: (was: HADOOP-10710.test-patch.log) hadoop.auth cookie is not properly constructed

[jira] [Commented] (HADOOP-9361) Strictly define the expected behavior of filesystem APIs and write tests to verify compliance

[ https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044077#comment-14044077 ] Juan Yu commented on HADOOP-9361: - seems ChecksumFileSystem.java missing the following

[jira] [Commented] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042173#comment-14042173 ] Juan Yu commented on HADOOP-10710: -- oh I see. I'll modify the unit tests to not use

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.005.patch Thanks tucu for the review. I still don't know why the build fails.

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.test-patch.log [~tucu], I tried mvn clean test -DskipTests

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.002.patch Hadoop QA doesn't give any detail error message. everything

[jira] [Commented] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037874#comment-14037874 ] Juan Yu commented on HADOOP-10710: -- that one doesn't support httponly flag. I could

[jira] [Commented] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037986#comment-14037986 ] Juan Yu commented on HADOOP-10710: -- I tried to use it before, but the

[jira] [Commented] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037987#comment-14037987 ] Juan Yu commented on HADOOP-10710: -- that's Servlet 3.0 API, Hadoop uses 2.5, doesn't

[jira] [Commented] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038016#comment-14038016 ] Juan Yu commented on HADOOP-10710: -- [~tucu], I tried, toString() still doesn't print

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.003.patch Thanks tucu, new patch uploaded. hadoop.auth cookie is not

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.004.patch revert the new unit test and some changes in

[jira] [Work started] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-10710 started by Juan Yu. hadoop.auth cookie is not properly constructed according to RFC2109

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Attachment: HADOOP-10710.001.patch I use HttpCookie class to get the cookie string so it will be

[jira] [Updated] (HADOOP-10710) hadoop.auth cookie is not properly constructed according to RFC2109

[ https://issues.apache.org/jira/browse/HADOOP-10710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HADOOP-10710: - Status: Patch Available (was: In Progress) hadoop.auth cookie is not properly constructed according

[jira] [Commented] (HADOOP-10610) Upgrade S3n s3.fs.buffer.dir to suppoer multi directories

[ https://issues.apache.org/jira/browse/HADOOP-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13999109#comment-13999109 ] Juan Yu commented on HADOOP-10610: -- Just wondering why don't cache the tmp dir list? do