[jira] [Resolved] (HDFS-5264) Consider renaming StorageID

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-5264. - Resolution: Won't Fix Punting - it will cause unnecessary code churn. > Consider renaming StorageI

[jira] [Resolved] (HDFS-5178) DataTransferProtocol changes for supporting mutiple storages per DN

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-5178. - Resolution: Duplicate Also resolving this as a duplicate of HDFS-6702. [~djp], my apologies once a

[jira] [Commented] (HDFS-3443) Unable to catch up edits during standby to active switch due to NPE

2015-01-20 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285234#comment-14285234 ] Vinayakumar B commented on HDFS-3443: - {quote}Need to add checkNNStartup() for the rpc

[jira] [Resolved] (HDFS-4989) Balancer needs to consider storage type in balancing

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-4989. - Resolution: Duplicate Resolving a duplicate of HDFS-6685 since that one is fixed. I am sorry about

[jira] [Updated] (HDFS-3443) Unable to catch up edits during standby to active switch due to NPE

2015-01-20 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-3443: Attachment: HDFS-3443-007.patch Attached the changes for AtomicBoolean for {{NameNode.started}} > Un

[jira] [Updated] (HDFS-7584) Enable Quota Support for Storage Types (SSD)

2015-01-20 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-7584: - Issue Type: New Feature (was: Sub-task) Parent: (was: HDFS-5682) > Enable Quota Support for St

[jira] [Commented] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285227#comment-14285227 ] Arpit Agarwal commented on HDFS-7643: - Thanks for the review [~xyao]. > Test case to e

[jira] [Commented] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285224#comment-14285224 ] Yi Liu commented on HDFS-7643: -- Thanks Arpit and Xiaoyu for review and commit. > Test case to

[jira] [Commented] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285223#comment-14285223 ] Hudson commented on HDFS-7643: -- FAILURE: Integrated in Hadoop-trunk-Commit #6900 (See [https:

[jira] [Updated] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-7643: Resolution: Fixed Target Version/s: (was: 2.7.0) Hadoop Flags: Reviewed

[jira] [Commented] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285219#comment-14285219 ] Xiaoyu Yao commented on HDFS-7643: -- LGTM. +1 (non-binding) > Test case to ensure lazy per

[jira] [Commented] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285206#comment-14285206 ] Hadoop QA commented on HDFS-7643: - {color:green}+1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-7650) Snapshot Corruption for OPENFORWRITE files

2015-01-20 Thread Prateek Gaur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prateek Gaur updated HDFS-7650: --- Attachment: hdfs_snapshot.cpp > Snapshot Corruption for OPENFORWRITE files > --

[jira] [Updated] (HDFS-7650) Snapshot Corruption for OPENFORWRITE files

2015-01-20 Thread Prateek Gaur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prateek Gaur updated HDFS-7650: --- Attachment: hdfs_config > Snapshot Corruption for OPENFORWRITE files >

[jira] [Created] (HDFS-7650) Snapshot Corruption for OPENFORWRITE files

2015-01-20 Thread Prateek Gaur (JIRA)
Prateek Gaur created HDFS-7650: -- Summary: Snapshot Corruption for OPENFORWRITE files Key: HDFS-7650 URL: https://issues.apache.org/jira/browse/HDFS-7650 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-7647) DatanodeManager.sortLocatedBlocks() sorts DatanodeIDs but not StorageIDs

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285184#comment-14285184 ] Arpit Agarwal commented on HDFS-7647: - Thanks for reporting this [~milandesai] and list

[jira] [Updated] (HDFS-7559) Create unit test to automatically compare HDFS related classes and hdfs-default.xml

2015-01-20 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated HDFS-7559: - Attachment: HDFS-7559.002.patch - Add two more Configuration properties to exceptions - Turn on flagging er

[jira] [Updated] (HDFS-7566) Remove obsolete entries from hdfs-default.xml

2015-01-20 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated HDFS-7566: - Status: Patch Available (was: Open) Submit for testing > Remove obsolete entries from hdfs-default.xml >

[jira] [Updated] (HDFS-7566) Remove obsolete entries from hdfs-default.xml

2015-01-20 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated HDFS-7566: - Attachment: HDFS-7566.001.patch First attempt at removing properties > Remove obsolete entries from hdfs-d

[jira] [Commented] (HDFS-7566) Remove obsolete entries from hdfs-default.xml

2015-01-20 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285169#comment-14285169 ] Ray Chiang commented on HDFS-7566: -- What I've found so far: dfs.https.enable - Removed in

[jira] [Commented] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285167#comment-14285167 ] Hadoop QA commented on HDFS-7641: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Updated] (HDFS-7649) Multihoming docs should emphasize using hostnames in configurations

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-7649: Issue Type: Improvement (was: Bug) > Multihoming docs should emphasize using hostnames in configurat

[jira] [Created] (HDFS-7649) Multihoming docs should emphasize using hostnames in configurations

2015-01-20 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-7649: --- Summary: Multihoming docs should emphasize using hostnames in configurations Key: HDFS-7649 URL: https://issues.apache.org/jira/browse/HDFS-7649 Project: Hadoop HDFS

[jira] [Updated] (HDFS-7649) Multihoming docs should emphasize using hostnames in configurations

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-7649: Component/s: documentation > Multihoming docs should emphasize using hostnames in configurations > --

[jira] [Commented] (HDFS-7610) Fix removal of dynamically added DN volumes

2015-01-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285146#comment-14285146 ] Hudson commented on HDFS-7610: -- FAILURE: Integrated in Hadoop-trunk-Commit #6899 (See [https:

[jira] [Updated] (HDFS-7610) Fix removal of dynamically added DN volumes

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7610: --- Resolution: Fixed Fix Version/s: 2.7.0 Status: Resolved (was: Patch A

[jira] [Updated] (HDFS-7610) Fix removal of dynamically added DN volumes

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7610: --- Summary: Fix removal of dynamically added DN volumes (was: Should use StorageDirectory

[jira] [Commented] (HDFS-7610) Should use StorageDirectory.getCurrentDIr() to construct FsVolumeImpl

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285136#comment-14285136 ] Colin Patrick McCabe commented on HDFS-7610: +1. Thanks, [~eddyxu]. > Should

[jira] [Commented] (HDFS-7575) NameNode not handling heartbeats properly after HDFS-2832

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285135#comment-14285135 ] Colin Patrick McCabe commented on HDFS-7575: Just to be clear, I'd like to see

[jira] [Commented] (HDFS-7623) Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285132#comment-14285132 ] Colin Patrick McCabe commented on HDFS-7623: Thanks, Yi! > Add htrace configur

[jira] [Resolved] (HDFS-7646) HDFS truncate may remove data from the supposedly read-only previous/ directory during an upgrade

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-7646. Resolution: Not a Problem > HDFS truncate may remove data from the supposedly read-on

[jira] [Commented] (HDFS-7646) HDFS truncate may remove data from the supposedly read-only previous/ directory during an upgrade

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285130#comment-14285130 ] Colin Patrick McCabe commented on HDFS-7646: Yes, you're right. Sorry. I can

[jira] [Updated] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode by reference-counting the volume instances

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7496: --- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode by reference-counting the volume instances

2015-01-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285119#comment-14285119 ] Hudson commented on HDFS-7496: -- SUCCESS: Integrated in Hadoop-trunk-Commit #6898 (See [https:

[jira] [Commented] (HDFS-7411) Refactor and improve decommissioning logic into DecommissionManager

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285117#comment-14285117 ] Colin Patrick McCabe commented on HDFS-7411: Sorry, there's been a bunch of big

[jira] [Commented] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285086#comment-14285086 ] Hudson commented on HDFS-7641: -- FAILURE: Integrated in Hadoop-trunk-Commit #6897 (See [https:

[jira] [Updated] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode by reference-counting the volume instances

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7496: --- Summary: Fix FsVolume removal race conditions on the DataNode by reference-counting the

[jira] [Commented] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285081#comment-14285081 ] Colin Patrick McCabe commented on HDFS-7496: +1. Thanks, [~eddyxu]. > Fix FsV

[jira] [Commented] (HDFS-3519) Checkpoint upload may interfere with a concurrent saveNamespace

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285077#comment-14285077 ] Hadoop QA commented on HDFS-3519: - {color:green}+1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7641: - Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-3689) Add support for variable length block

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285063#comment-14285063 ] Colin Patrick McCabe commented on HDFS-3689: Thanks for working on this, Jing.

[jira] [Commented] (HDFS-7577) Add additional headers that includes need by Windows

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285057#comment-14285057 ] Colin Patrick McCabe commented on HDFS-7577: This looks pretty good. {{os/wind

[jira] [Commented] (HDFS-7575) NameNode not handling heartbeats properly after HDFS-2832

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285048#comment-14285048 ] Colin Patrick McCabe commented on HDFS-7575: bq. Layout version defines layout

[jira] [Commented] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285049#comment-14285049 ] Arpit Agarwal commented on HDFS-7643: - +1 pending Jenkins. Thank you for the prompt fi

[jira] [Commented] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285043#comment-14285043 ] Yi Liu commented on HDFS-7641: -- Thanks Jing for the review, will commit it shortly. > Update

[jira] [Updated] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7643: - Status: Patch Available (was: Open) > Test case to ensure lazy persist files cannot be truncated > ---

[jira] [Updated] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7643: - Attachment: HDFS-7643.001.patch Agree that we'd better to have a test. Sorry for not having one for HDFS-7634. Att

[jira] [Assigned] (HDFS-7643) Test case to ensure lazy persist files cannot be truncated

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reassigned HDFS-7643: Assignee: Yi Liu > Test case to ensure lazy persist files cannot be truncated >

[jira] [Commented] (HDFS-7623) Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285035#comment-14285035 ] Hadoop QA commented on HDFS-7623: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7339) Allocating and persisting block groups in NameNode

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285013#comment-14285013 ] Hadoop QA commented on HDFS-7339: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7623) Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace

2015-01-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285007#comment-14285007 ] Hudson commented on HDFS-7623: -- SUCCESS: Integrated in Hadoop-trunk-Commit #6896 (See [https:

[jira] [Updated] (HDFS-7623) Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7623: - Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284993#comment-14284993 ] Jing Zhao commented on HDFS-7641: - Thanks Yi! The patch looks good to me. +1 > Update arch

[jira] [Updated] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7641: - Attachment: HDFS-7641.002.patch > Update archival storage user doc for list/set/get block storage policies > --

[jira] [Updated] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7641: - Attachment: (was: HDFS-7641.002.patch) > Update archival storage user doc for list/set/get block storage polici

[jira] [Updated] (HDFS-7641) Update archival storage user doc for list/set/get block storage policies

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7641: - Attachment: HDFS-7641.002.patch Thanks Jing for review. You are right, I missed that. Update the patch to address

[jira] [Commented] (HDFS-7647) DatanodeManager.sortLocatedBlocks() sorts DatanodeIDs but not StorageIDs

2015-01-20 Thread Milan Desai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284980#comment-14284980 ] Milan Desai commented on HDFS-7647: --- There are a couple of ways to go about fixing this.

[jira] [Commented] (HDFS-7575) NameNode not handling heartbeats properly after HDFS-2832

2015-01-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284973#comment-14284973 ] Tsz Wo Nicholas Sze commented on HDFS-7575: --- We should log the old (invalid) stor

[jira] [Updated] (HDFS-7623) Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7623: - Attachment: HDFS-7623.002.patch No problem:) Thanks a lot for review, Colin. Update the patch to address the commen

[jira] [Commented] (HDFS-7548) Corrupt block reporting delayed until datablock scanner thread detects it

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284960#comment-14284960 ] Hadoop QA commented on HDFS-7548: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-6833) DirectoryScanner should not register a deleting block with memory of DataNode

2015-01-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284959#comment-14284959 ] Tsz Wo Nicholas Sze commented on HDFS-6833: --- Sure, will review the patch. > Dire

[jira] [Created] (HDFS-7648) Verify the datanode directory layout

2015-01-20 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7648: - Summary: Verify the datanode directory layout Key: HDFS-7648 URL: https://issues.apache.org/jira/browse/HDFS-7648 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-7610) Should use StorageDirectory.getCurrentDIr() to construct FsVolumeImpl

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284950#comment-14284950 ] Hadoop QA commented on HDFS-7610: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284942#comment-14284942 ] Haohui Mai commented on HDFS-6673: -- Just to recap. The current approaches are (please corr

[jira] [Commented] (HDFS-7575) NameNode not handling heartbeats properly after HDFS-2832

2015-01-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284937#comment-14284937 ] Tsz Wo Nicholas Sze commented on HDFS-7575: --- Layout version defines layout format

[jira] [Commented] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284913#comment-14284913 ] Hadoop QA commented on HDFS-7496: - {color:green}+1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-7647) DatanodeManager.sortLocatedBlocks() sorts DatanodeIDs but not StorageIDs

2015-01-20 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-7647: -- Target Version/s: 2.7.0 Affects Version/s: (was: 3.0.0) 2.

[jira] [Commented] (HDFS-7634) Disallow truncation of Lazy persist files

2015-01-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284906#comment-14284906 ] Yi Liu commented on HDFS-7634: -- Thanks a lot for [~arpitagarwal] and [~shv] for the review and

[jira] [Commented] (HDFS-7646) HDFS truncate may remove data from the supposedly read-only previous/ directory during an upgrade

2015-01-20 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284897#comment-14284897 ] Konstantin Shvachko commented on HDFS-7646: --- In place truncate is disabled during

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284889#comment-14284889 ] Lei (Eddy) Xu commented on HDFS-6673: - I went back to check the sizes of two LevelDB me

[jira] [Commented] (HDFS-7646) HDFS truncate may remove data from the supposedly read-only previous/ directory during an upgrade

2015-01-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284708#comment-14284708 ] Tsz Wo Nicholas Sze commented on HDFS-7646: --- Do we have copy-on-truncate? We dis

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284696#comment-14284696 ] Lei (Eddy) Xu commented on HDFS-6673: - [~wheat9] To provide more background, I describe

[jira] [Commented] (HDFS-7575) NameNode not handling heartbeats properly after HDFS-2832

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284681#comment-14284681 ] Colin Patrick McCabe commented on HDFS-7575: So there are two approaches here:

[jira] [Updated] (HDFS-3519) Checkpoint upload may interfere with a concurrent saveNamespace

2015-01-20 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-3519: -- Attachment: HDFS-3519-3.patch Thanks, Chris. Good point. Here is the updated patch with your suggestions. > Chec

[jira] [Commented] (HDFS-7623) Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284654#comment-14284654 ] Colin Patrick McCabe commented on HDFS-7623: bq. using the open source tracing

[jira] [Commented] (HDFS-7611) TestFileTruncate.testTruncateEditLogLoad times out waiting for Mini HDFS Cluster to start

2015-01-20 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284651#comment-14284651 ] Konstantin Shvachko commented on HDFS-7611: --- Was looking at {{TestOpenFilesWithSn

[jira] [Commented] (HDFS-7005) DFS input streams do not timeout

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284645#comment-14284645 ] Colin Patrick McCabe commented on HDFS-7005: [~zsl2007], it appears that the Da

[jira] [Created] (HDFS-7647) DatanodeManager.sortLocatedBlocks() sorts DatanodeIDs but not StorageIDs

2015-01-20 Thread Milan Desai (JIRA)
Milan Desai created HDFS-7647: - Summary: DatanodeManager.sortLocatedBlocks() sorts DatanodeIDs but not StorageIDs Key: HDFS-7647 URL: https://issues.apache.org/jira/browse/HDFS-7647 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284600#comment-14284600 ] Lei (Eddy) Xu commented on HDFS-6673: - bq. For this particular purpose you don't necess

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284596#comment-14284596 ] Haohui Mai commented on HDFS-6673: -- bq. Thank you very much to pointing this out. In your

[jira] [Commented] (HDFS-7587) Edit log corruption can happen if append fails with a quota violation

2015-01-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284594#comment-14284594 ] Tsz Wo Nicholas Sze commented on HDFS-7587: --- > ... The quotas aren't going to cha

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284590#comment-14284590 ] Lei (Eddy) Xu commented on HDFS-6673: - One more thing, we'd also like to have the full

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284580#comment-14284580 ] Lei (Eddy) Xu commented on HDFS-6673: - Hi, [~wheat9] Thank you very much to pointing t

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284572#comment-14284572 ] Haohui Mai commented on HDFS-6673: -- Here is a link to the related comments from HDFS-6293:

[jira] [Updated] (HDFS-7339) Allocating and persisting block groups in NameNode

2015-01-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7339: Attachment: HDFS-7339-005.patch Fix findbugs issues. > Allocating and persisting block groups in NameNode >

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284564#comment-14284564 ] Haohui Mai commented on HDFS-6673: -- bq. By IN || parent_id || localName, do you mean conca

[jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool

2015-01-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284506#comment-14284506 ] Andrew Wang commented on HDFS-6673: --- Hi Eddy, thanks for doing these benchmarks. I only h

[jira] [Updated] (HDFS-7548) Corrupt block reporting delayed until datablock scanner thread detects it

2015-01-20 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-7548: - Status: Patch Available (was: Open) > Corrupt block reporting delayed until datablock scanner thre

[jira] [Updated] (HDFS-7548) Corrupt block reporting delayed until datablock scanner thread detects it

2015-01-20 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-7548: - Attachment: HDFS-7548-v4.patch Attaching a new patch addressing Daryn's comments. > Corrupt block

[jira] [Updated] (HDFS-7610) Should use StorageDirectory.getCurrentDIr() to construct FsVolumeImpl

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-7610: Attachment: HDFS-7610.001.patch [~cmccabe] Thanks for reviewing. I have updated the patch to address

[jira] [Created] (HDFS-7646) HDFS truncate may remove data from the supposedly read-only previous/ directory during an upgrade

2015-01-20 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7646: -- Summary: HDFS truncate may remove data from the supposedly read-only previous/ directory during an upgrade Key: HDFS-7646 URL: https://issues.apache.org/jira/browse/HD

[jira] [Updated] (HDFS-7548) Corrupt block reporting delayed until datablock scanner thread detects it

2015-01-20 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-7548: - Status: Open (was: Patch Available) Cancelling the patch to address Daryn's comment. > Corrupt bl

[jira] [Commented] (HDFS-7645) Rolling upgrade is restoring blocks from trash multiple times

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284445#comment-14284445 ] Colin Patrick McCabe commented on HDFS-7645: I think we should get rid of trash

[jira] [Updated] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode

2015-01-20 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-7496: Attachment: HDFS-7496.007.patch HDFS-5631 introduced {{ExternalVolumeImpl}} and {{ExternalDatasetImpl

[jira] [Commented] (HDFS-7411) Refactor and improve decommissioning logic into DecommissionManager

2015-01-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284438#comment-14284438 ] Andrew Wang commented on HDFS-7411: --- I think these test failures are unrelated. HDFS-7527

[jira] [Commented] (HDFS-7496) Fix FsVolume removal race conditions on the DataNode

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284429#comment-14284429 ] Hadoop QA commented on HDFS-7496: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7339) Allocating and persisting block groups in NameNode

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284427#comment-14284427 ] Hadoop QA commented on HDFS-7339: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Created] (HDFS-7645) Rolling upgrade is restoring blocks from trash multiple times

2015-01-20 Thread Nathan Roberts (JIRA)
Nathan Roberts created HDFS-7645: Summary: Rolling upgrade is restoring blocks from trash multiple times Key: HDFS-7645 URL: https://issues.apache.org/jira/browse/HDFS-7645 Project: Hadoop HDFS

[jira] [Commented] (HDFS-4173) If the NameNode has already been formatted, but a QuroumJournal has not, auto-format it on startup

2015-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284418#comment-14284418 ] Hadoop QA commented on HDFS-4173: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7610) Should use StorageDirectory.getCurrentDIr() to construct FsVolumeImpl

2015-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284413#comment-14284413 ] Colin Patrick McCabe commented on HDFS-7610: {code} + try { +volum

  1   2   >