[jira] [Updated] (HDFS-6591) while loop is executed tens of thousands of times in Hedged Read

2014-06-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-6591: Target Version/s: 3.0.0, 2.5.0 Hadoop Flags: Reviewed Thanks for the explanation. Yes, I

[jira] [Commented] (HDFS-6591) while loop is executed tens of thousands of times in Hedged Read

2014-06-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045632#comment-14045632 ] Hadoop QA commented on HDFS-6591: - {color:green}+1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-6596) Improve InputStream when read spans two blocks

2014-06-27 Thread Zesheng Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zesheng Wu updated HDFS-6596: - Attachment: HDFS-6596.2.patch Uploaded a polished version. Improve InputStream when read spans two

[jira] [Updated] (HDFS-6556) Refine XAttr permissions

2014-06-27 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-6556: -- Attachment: refinedPermissions-HDFS-6556-3.patch Thanks a lot, Chris for the review!

[jira] [Commented] (HDFS-6596) Improve InputStream when read spans two blocks

2014-06-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045789#comment-14045789 ] Hadoop QA commented on HDFS-6596: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-6527) Edit log corruption due to defered INode removal

2014-06-27 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-6527: - Attachment: HDFS-6527-addendum-test.patch Hey folks, I've been looking into this a bit and have

[jira] [Commented] (HDFS-6572) Add an option to the NameNode that prints the software and on-disk image versions

2014-06-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045819#comment-14045819 ] Hudson commented on HDFS-6572: -- FAILURE: Integrated in Hadoop-Yarn-trunk #596 (See

[jira] [Commented] (HDFS-6556) Refine XAttr permissions

2014-06-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045922#comment-14045922 ] Hadoop QA commented on HDFS-6556: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6572) Add an option to the NameNode that prints the software and on-disk image versions

2014-06-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045977#comment-14045977 ] Hudson commented on HDFS-6572: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1787 (See

[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change

2014-06-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045990#comment-14045990 ] Kihwal Lee commented on HDFS-6601: -- [~jingzhao] Would you review the patch? We've tested

[jira] [Commented] (HDFS-6572) Add an option to the NameNode that prints the software and on-disk image versions

2014-06-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046035#comment-14046035 ] Hudson commented on HDFS-6572: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1814 (See

[jira] [Commented] (HDFS-6391) Get the Key/IV from the NameNode for encrypted files in DFSClient

2014-06-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046043#comment-14046043 ] Charles Lamb commented on HDFS-6391: LGTM, +1 Sorry about my nits, but I appreciate

[jira] [Commented] (HDFS-6556) Refine XAttr permissions

2014-06-27 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046055#comment-14046055 ] Uma Maheswara Rao G commented on HDFS-6556: --- Above test failure should be

[jira] [Created] (HDFS-6604) Disk space leak with shortcircuit

2014-06-27 Thread Giuseppe Reina (JIRA)
Giuseppe Reina created HDFS-6604: Summary: Disk space leak with shortcircuit Key: HDFS-6604 URL: https://issues.apache.org/jira/browse/HDFS-6604 Project: Hadoop HDFS Issue Type: Bug

[jira] [Comment Edited] (HDFS-6556) Refine XAttr permissions

2014-06-27 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045788#comment-14045788 ] Uma Maheswara Rao G edited comment on HDFS-6556 at 6/27/14 3:38 PM:

[jira] [Updated] (HDFS-6556) Refine XAttr permissions

2014-06-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-6556: Hadoop Flags: Reviewed +1 for the patch. Thanks, Uma! Yi, thank you for code reviewing. Refine

[jira] [Commented] (HDFS-6591) while loop is executed tens of thousands of times in Hedged Read

2014-06-27 Thread Andrew Purtell (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046146#comment-14046146 ] Andrew Purtell commented on HDFS-6591: -- {quote} bq. Is it actually useful to add the

[jira] [Updated] (HDFS-6604) Disk space leak with shortcircuit

2014-06-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6604: - Priority: Critical (was: Major) Disk space leak with shortcircuit -

[jira] [Commented] (HDFS-6527) Edit log corruption due to defered INode removal

2014-06-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046173#comment-14046173 ] Jing Zhao commented on HDFS-6527: - Thanks [~atm]! The analysis makes sense to me. Let's

[jira] [Commented] (HDFS-6527) Edit log corruption due to defered INode removal

2014-06-27 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046194#comment-14046194 ] Aaron T. Myers commented on HDFS-6527: -- bq. In the meanwhile, in FSNamesystem#delete,

[jira] [Commented] (HDFS-6591) while loop is executed tens of thousands of times in Hedged Read

2014-06-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046193#comment-14046193 ] Chris Nauroth commented on HDFS-6591: - I agree with Andrew. The problem with

[jira] [Updated] (HDFS-6591) while loop is executed tens of thousands of times in Hedged Read

2014-06-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-6591: Hadoop Flags: (was: Reviewed) while loop is executed tens of thousands of times in Hedged Read

[jira] [Commented] (HDFS-6527) Edit log corruption due to defered INode removal

2014-06-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046198#comment-14046198 ] Jing Zhao commented on HDFS-6527: - Yeah, let's merge the fix back to 2.4.1 with unit test

[jira] [Commented] (HDFS-2856) Fix block protocol so that Datanodes don't require root or jsvc

2014-06-27 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046201#comment-14046201 ] Jitendra Nath Pandey commented on HDFS-2856: - For the specialized encrypted

[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change

2014-06-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046205#comment-14046205 ] Jing Zhao commented on HDFS-6601: - The patch looks good to me. +1. Thanks [~kihwal]!

[jira] [Commented] (HDFS-5851) Support memory as a storage medium

2014-06-27 Thread Henry Saputra (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046247#comment-14046247 ] Henry Saputra commented on HDFS-5851: - HI [~sanjay.radia], I was looking at the JIRA

[jira] [Commented] (HDFS-5851) Support memory as a storage medium

2014-06-27 Thread Andrew Purtell (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046280#comment-14046280 ] Andrew Purtell commented on HDFS-5851: -- bq. In term of discardability, what is the

[jira] [Commented] (HDFS-2856) Fix block protocol so that Datanodes don't require root or jsvc

2014-06-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046317#comment-14046317 ] Chris Nauroth commented on HDFS-2856: - Jitendra, thank you for taking a look at this

[jira] [Commented] (HDFS-6389) Rename restrictions for encryption zones

2014-06-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046352#comment-14046352 ] Colin Patrick McCabe commented on HDFS-6389: Thanks. Looks better. I would

Re: Anyone know how to mock a secured hdfs for unit test?

2014-06-27 Thread Chris Nauroth
Hi David and Kai, There are a couple of challenges with this, but I just figured out a pretty decent setup while working on HDFS-2856. That code isn't committed yet, but if you open patch version 5 attached to that issue and look for the TestSaslDataTransfer class, then you'll see how it works.

[jira] [Updated] (HDFS-6391) Get the Key/IV from the NameNode for encrypted files in DFSClient

2014-06-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6391: -- Attachment: hdfs-6391.004.patch New patch fixes the whitespace issues and I moved around imports in

[jira] [Resolved] (HDFS-6391) Get the Key/IV from the NameNode for encrypted files in DFSClient

2014-06-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-6391. --- Resolution: Fixed Fix Version/s: fs-encryption (HADOOP-10150 and HDFS-6134) Committed to

[jira] [Created] (HDFS-6605) Client server negotiation of cipher suite

2014-06-27 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6605: - Summary: Client server negotiation of cipher suite Key: HDFS-6605 URL: https://issues.apache.org/jira/browse/HDFS-6605 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-6605) Client server negotiation of cipher suite

2014-06-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046386#comment-14046386 ] Andrew Wang commented on HDFS-6605: --- There was some discussion on HDFS-6391 with

[jira] [Updated] (HDFS-6389) Rename restrictions for encryption zones

2014-06-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6389: --- Attachment: HDFS-6389.003.patch [~cmccabe], You read my mind. I was working on the 2nd test when you

[jira] [Resolved] (HDFS-6389) Rename restrictions for encryption zones

2014-06-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb resolved HDFS-6389. Resolution: Fixed Fix Version/s: fs-encryption (HADOOP-10150 and HDFS-6134) [~cmccabe],

[jira] [Commented] (HDFS-6527) Edit log corruption due to defered INode removal

2014-06-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046441#comment-14046441 ] Jing Zhao commented on HDFS-6527: - [~atm], do you already have a updated patch for 2.4.1?

[jira] [Commented] (HDFS-6441) Add ability to exclude/include few datanodes while balancing

2014-06-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046451#comment-14046451 ] Arpit Agarwal commented on HDFS-6441: - HDFS-6133 excludes by file path, whereas this

[jira] [Updated] (HDFS-2856) Fix block protocol so that Datanodes don't require root or jsvc

2014-06-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-2856: Attachment: HDFS-2856.6.patch Here is patch version 6. This incorporates Jitendra's feedback as I

[jira] [Updated] (HDFS-4685) Implementation of ACLs in HDFS

2014-06-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-4685: -- Assignee: Chris Nauroth (was: Andrew Wang) Implementation of ACLs in HDFS

[jira] [Work started] (HDFS-6604) Disk space leak with shortcircuit

2014-06-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6604 started by Colin Patrick McCabe. Disk space leak with shortcircuit - Key: HDFS-6604

[jira] [Updated] (HDFS-6604) Disk space leak with shortcircuit

2014-06-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6604: --- Attachment: HDFS-6604.001.patch The CacheCleaner thread is supposed to time out

[jira] [Assigned] (HDFS-6604) Disk space leak with shortcircuit

2014-06-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reassigned HDFS-6604: -- Assignee: Colin Patrick McCabe Disk space leak with shortcircuit

[jira] [Updated] (HDFS-6604) Disk space leak with shortcircuit

2014-06-27 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-6604: - Target Version/s: 2.5.0 Disk space leak with shortcircuit -

[jira] [Commented] (HDFS-6527) Edit log corruption due to defered INode removal

2014-06-27 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046521#comment-14046521 ] Aaron T. Myers commented on HDFS-6527: -- Hi Jing, I haven't currently create that

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046571#comment-14046571 ] Tsz Wo Nicholas Sze commented on HDFS-6134: --- Correct me if I am wrong: The

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-27 Thread Alejandro Abdelnur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046575#comment-14046575 ] Alejandro Abdelnur commented on HDFS-6134: -- afaik, the only way you can protect

[jira] [Commented] (HDFS-2856) Fix block protocol so that Datanodes don't require root or jsvc

2014-06-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046573#comment-14046573 ] Hadoop QA commented on HDFS-2856: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-27 Thread Andrew Purtell (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046577#comment-14046577 ] Andrew Purtell commented on HDFS-6134: -- bq. The current design does not prevent a

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-27 Thread Alejandro Abdelnur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046584#comment-14046584 ] Alejandro Abdelnur commented on HDFS-6134: -- and even then, without getting the

[jira] [Commented] (HDFS-6441) Add ability to exclude/include few datanodes while balancing

2014-06-27 Thread Benoy Antony (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046694#comment-14046694 ] Benoy Antony commented on HDFS-6441: I am yet to get a response from [~carp84]. Add

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-27 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046711#comment-14046711 ] Todd Lipcon commented on HDFS-6134: --- I think the solution to this issue is by