[jira] [Updated] (HDFS-17404) Add Namenode info to log message when setting block keys from active nn

2024-03-05 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17404: Fix Version/s: 3.3.7 3.4.1 > Add Namenode info to log message when setting block

[jira] [Resolved] (HDFS-17404) Add Namenode info to log message when setting block keys from active nn

2024-03-05 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo resolved HDFS-17404. - Resolution: Fixed > Add Namenode info to log message when setting block keys from active nn > ---

[jira] [Updated] (HDFS-17404) Add Namenode info to log message when setting block keys from active nn

2024-03-05 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17404: Fix Version/s: 3.5.0 > Add Namenode info to log message when setting block keys from active nn > --

[jira] [Updated] (HDFS-17404) Add Namenode info to log message when setting block keys from active nn

2024-03-05 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17404: Description: Add Namenode info to log message when setting block keys from active nn (was: When BP

[jira] [Updated] (HDFS-17404) Add Namenode info to log message when setting block keys from active nn

2024-03-05 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17404: Summary: Add Namenode info to log message when setting block keys from active nn (was: Add NN Sock

[jira] [Updated] (HDFS-17404) Add NN Socket Address to log when processing command from active NN

2024-03-05 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17404: Summary: Add NN Socket Address to log when processing command from active NN (was: Record the BPSe

[jira] [Updated] (HDFS-17404) Add Namenode info to log message when setting block keys

2024-03-01 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17404: Summary: Add Namenode info to log message when setting block keys (was: Update DN to report which

[jira] [Moved] (HDFS-17404) Update DN to report which NN has provided the updated block keys

2024-02-28 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo moved HADOOP-19094 to HDFS-17404: -- Key: HDFS-17404 (was: HADOOP-19094) Project: Hadoop HDFS (was

[jira] [Resolved] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-19 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo resolved HDFS-17332. - Resolution: Fixed > DFSInputStream: avoid logging stacktrace until when we really need to fail a

[jira] [Updated] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-19 Thread Chris Trezzo (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-17332: Fix Version/s: 3.5.0 > DFSInputStream: avoid logging stacktrace until when we really need to fail a

[jira] [Commented] (HDFS-11035) Better documentation for maintenace mode and upgrade domain

2017-09-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170926#comment-16170926 ] Chris Trezzo commented on HDFS-11035: - +1 HDFS-11035-2.patch looks good to me. I look

[jira] [Comment Edited] (HDFS-11035) Better documentation for maintenace mode and upgrade domain

2017-09-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170926#comment-16170926 ] Chris Trezzo edited comment on HDFS-11035 at 9/19/17 12:05 AM: -

[jira] [Commented] (HDFS-8789) Block Placement policy migrator

2017-05-08 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001194#comment-16001194 ] Chris Trezzo commented on HDFS-8789: [~kihwal] Thanks for the comment! As I said, this

[jira] [Comment Edited] (HDFS-8789) Block Placement policy migrator

2017-05-03 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995424#comment-15995424 ] Chris Trezzo edited comment on HDFS-8789 at 5/3/17 6:51 PM: [~k

[jira] [Commented] (HDFS-8789) Block Placement policy migrator

2017-05-03 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995424#comment-15995424 ] Chris Trezzo commented on HDFS-8789: [~kihwal] [~zhz] I am not sure what your plans are

[jira] [Updated] (HDFS-8874) Add DN metrics for balancer and other block movement scenarios

2017-03-30 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8874: --- Status: In Progress (was: Patch Available) > Add DN metrics for balancer and other block movement scen

[jira] [Commented] (HDFS-8874) Add DN metrics for balancer and other block movement scenarios

2017-03-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948211#comment-15948211 ] Chris Trezzo commented on HDFS-8874: Woops. Sorry about that. Will post a new version.

[jira] [Issue Comment Deleted] (HDFS-8874) Add DN metrics for balancer and other block movement scenarios

2017-03-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8874: --- Comment: was deleted (was: Woops. Sorry about that. Will post a new version.) > Add DN metrics for bal

[jira] [Commented] (HDFS-8874) Add DN metrics for balancer and other block movement scenarios

2017-03-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948209#comment-15948209 ] Chris Trezzo commented on HDFS-8874: Woops. Sorry about that. Will post a new version.

[jira] [Updated] (HDFS-8874) Add DN metrics for balancer and other block movement scenarios

2017-03-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8874: --- Attachment: HDFS-8874-trunk.006.patch Attached v6 trunk patch to address whitespace and checkstyle. >

[jira] [Updated] (HDFS-8874) Add DN metrics for balancer and other block movement scenarios

2017-03-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8874: --- Attachment: HDFS-8874-trunk.005.patch Rebased and attached v5 for trunk. > Add DN metrics for balancer

[jira] [Work stopped] (HDFS-8875) Optimize the wait time in Balancer for federation scenario

2017-01-20 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8875 stopped by Chris Trezzo. -- > Optimize the wait time in Balancer for federation scenario > ---

[jira] [Work stopped] (HDFS-8876) Make hard coded parameters used by balancer and other tools configurable

2017-01-20 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8876 stopped by Chris Trezzo. -- > Make hard coded parameters used by balancer and other tools configurable > -

[jira] [Updated] (HDFS-7530) Allow renaming of encryption zone roots

2016-09-20 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-7530: --- Target Version/s: 2.7.0, 2.6.5 (was: 2.7.0) Adding 2.6.5 to the target versions. We are looking to bac

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-09-20 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507817#comment-15507817 ] Chris Trezzo commented on HDFS-9688: Thanks for the pointer to HDFS-7530 [~xyao]! > Te

[jira] [Comment Edited] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-09-20 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507419#comment-15507419 ] Chris Trezzo edited comment on HDFS-9688 at 9/20/16 7:56 PM: - H

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-09-20 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507419#comment-15507419 ] Chris Trezzo commented on HDFS-9688: Hi [~zhz] [~andrew.wang]. TestNestedEncryptionZone

[jira] [Updated] (HDFS-10495) Block should be marked as missing if the all the replicas are on Decommissioned nodes.

2016-09-16 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-10495: Target Version/s: 2.8.0, 2.7.4, 2.6.6 (was: 2.8.0, 2.6.5, 2.7.4) > Block should be marked as missin

[jira] [Updated] (HDFS-8870) Lease is leaked on write failure

2016-09-16 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8870: --- Target Version/s: 2.7.4, 2.6.6 (was: 2.6.5, 2.7.4) Moving this issue to 2.6.6. Please move back if you

[jira] [Commented] (HDFS-10495) Block should be marked as missing if the all the replicas are on Decommissioned nodes.

2016-09-16 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15497386#comment-15497386 ] Chris Trezzo commented on HDFS-10495: - Moving this issue to 2.6.6. Please move back if

[jira] [Commented] (HDFS-10178) Permanent write failures can happen if pipeline recoveries occur for the first packet

2016-09-13 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488823#comment-15488823 ] Chris Trezzo commented on HDFS-10178: - Thanks! > Permanent write failures can happen

[jira] [Commented] (HDFS-9669) TcpPeerServer should respect ipc.server.listen.queue.size

2016-09-13 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488775#comment-15488775 ] Chris Trezzo commented on HDFS-9669: Thanks! > TcpPeerServer should respect ipc.server

[jira] [Commented] (HDFS-8845) DiskChecker should not traverse the entire tree

2016-09-13 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488765#comment-15488765 ] Chris Trezzo commented on HDFS-8845: Thanks! > DiskChecker should not traverse the ent

[jira] [Resolved] (HDFS-9028) Create accessor methods for DataNode#data and DataNode#isBlockTokenEnabled

2016-08-24 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo resolved HDFS-9028. Resolution: Won't Fix The original motivation for creating these accessor methods is no longer valid.

[jira] [Updated] (HDFS-9028) Create accessor methods for DataNode#data and DataNode#isBlockTokenEnabled

2016-08-24 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9028: --- Assignee: (was: Chris Trezzo) > Create accessor methods for DataNode#data and DataNode#isBlockToken

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-23 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433785#comment-15433785 ] Chris Trezzo commented on HDFS-9696: [~kihwal] Do you think this is worth backporting t

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-23 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433751#comment-15433751 ] Chris Trezzo commented on HDFS-10763: - [~kihwal] do you think this is worth backportin

[jira] [Updated] (HDFS-10178) Permanent write failures can happen if pipeline recoveries occur for the first packet

2016-08-22 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-10178: Target Version/s: 2.7.3, 2.6.5 (was: 2.7.3) > Permanent write failures can happen if pipeline recov

[jira] [Commented] (HDFS-10178) Permanent write failures can happen if pipeline recoveries occur for the first packet

2016-08-22 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431746#comment-15431746 ] Chris Trezzo commented on HDFS-10178: - Adding 2.6.5 to the target versions with the in

[jira] [Commented] (HDFS-9669) TcpPeerServer should respect ipc.server.listen.queue.size

2016-08-22 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431672#comment-15431672 ] Chris Trezzo commented on HDFS-9669: Adding 2.6.5 to the target versions with the inten

[jira] [Updated] (HDFS-9669) TcpPeerServer should respect ipc.server.listen.queue.size

2016-08-22 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9669: --- Target Version/s: 2.7.3, 2.6.5 (was: 2.7.3) > TcpPeerServer should respect ipc.server.listen.queue.siz

[jira] [Updated] (HDFS-8845) DiskChecker should not traverse the entire tree

2016-08-22 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8845: --- Target Version/s: 2.8.0, 2.6.5 (was: 2.8.0) > DiskChecker should not traverse the entire tree > --

[jira] [Commented] (HDFS-8845) DiskChecker should not traverse the entire tree

2016-08-22 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431594#comment-15431594 ] Chris Trezzo commented on HDFS-8845: Adding 2.6.5 to the target versions with the inten

[jira] [Updated] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-06-14 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9922: --- Attachment: HDFS-9922-trunk-v3.patch V3 attached. # Fixed import issues in TestUpgradeDomainBlockPlacem

[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-06-14 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330646#comment-15330646 ] Chris Trezzo commented on HDFS-9922: Also, thank you [~mingma] for the review and help

[jira] [Updated] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-06-14 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9922: --- Attachment: HDFS-9922-trunk-v2.patch V2 attached. # Spoke with [~mingma] offline and will not modify th

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229360#comment-15229360 ] Chris Trezzo commented on HDFS-8791: Thanks [~kihwal]! > block ID-based DN storage lay

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225258#comment-15225258 ] Chris Trezzo commented on HDFS-8791: [~busbey] That being said, I did manually verify t

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225244#comment-15225244 ] Chris Trezzo commented on HDFS-8791: bq. My understanding was that this patch adds a ne

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225242#comment-15225242 ] Chris Trezzo commented on HDFS-8791: bq. Please expand the release note to state that s

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Release Note: HDFS-8791 introduces a new datanode layout format. This layout is identical to the previo

[jira] [Updated] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-03-08 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9922: --- Status: Patch Available (was: In Progress) > Upgrade Domain placement policy status marks a good block

[jira] [Updated] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-03-08 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9922: --- Attachment: HDFS-9922-trunk-v1.patch [~mingma] v1 patch attached. > Upgrade Domain placement policy st

[jira] [Updated] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-03-08 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9922: --- Issue Type: Sub-task (was: Bug) Parent: HDFS-7541 > Upgrade Domain placement policy status mar

[jira] [Created] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-03-08 Thread Chris Trezzo (JIRA)
Chris Trezzo created HDFS-9922: -- Summary: Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes Key: HDFS-9922 URL: https://issues.apache.org/jira/browse/HDFS-9922

[jira] [Work started] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-03-08 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9922 started by Chris Trezzo. -- > Upgrade Domain placement policy status marks a good block in violation when > there

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-03-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Release Note: HDFS-8791 introduces a new datanode layout format. This layout is identical to the previo

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-03-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174509#comment-15174509 ] Chris Trezzo commented on HDFS-8791: [~kihwal] Should I add a blurb in the jira release

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-03-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174499#comment-15174499 ] Chris Trezzo commented on HDFS-8791: Thanks [~kihwal] and everyone else! > block ID-ba

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-02-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172951#comment-15172951 ] Chris Trezzo commented on HDFS-8791: Thanks for the question [~szetszwo]! To echo [~kih

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-02-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: HDFS-8791-trunk-v3-bin.patch Attaching rebased v3-bin patch. Now that HDFS-8578 is committ

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172705#comment-15172705 ] Chris Trezzo commented on HDFS-8578: Woops. Nevermind. I see the commits there now. Sor

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-29 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172696#comment-15172696 ] Chris Trezzo commented on HDFS-8578: [~szetszwo] [~vinayrpet] This patch also needs to

[jira] [Updated] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8923: --- Attachment: HDFS-8923-trunk-v4.patch [~szetszwo] sounds good! Attached is v4. > Add -source flag to ba

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153250#comment-15153250 ] Chris Trezzo commented on HDFS-8578: Thanks [~szetszwo]! I now see that the boolean ens

[jira] [Commented] (HDFS-8789) Block Placement policy migrator

2016-02-05 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135414#comment-15135414 ] Chris Trezzo commented on HDFS-8789: Also, here is the usage message for the command: {

[jira] [Updated] (HDFS-8789) Block Placement policy migrator

2016-02-05 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8789: --- Attachment: HDFS-8789-trunk-STRAWMAN-v1.patch Attached is a v1 strawman patch based on trunk. Note: Thi

[jira] [Commented] (HDFS-9761) Rebalancer sleeps too long between iterations

2016-02-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133362#comment-15133362 ] Chris Trezzo commented on HDFS-9761: +1. The patch looks good to me. I agree, I think t

[jira] [Updated] (HDFS-9761) Rebalancer sleeps too long between iterations

2016-02-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-9761: --- Assignee: (was: Chris Trezzo) > Rebalancer sleeps too long between iterations > ---

[jira] [Assigned] (HDFS-9761) Rebalancer sleeps too long between iterations

2016-02-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo reassigned HDFS-9761: -- Assignee: Chris Trezzo (was: Mingliang Liu) > Rebalancer sleeps too long between iterations > -

[jira] [Commented] (HDFS-9761) Rebalancer sleeps too long between iterations

2016-02-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133320#comment-15133320 ] Chris Trezzo commented on HDFS-9761: [~cnauroth] will take a look. Thanks! > Rebalance

[jira] [Updated] (HDFS-8923) Add -source flag to balancer usage message

2016-02-03 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8923: --- Attachment: HDFS-8923-trunk-v3.patch Thanks [~xyao]! Attached is a rebased v3 of the patch. > Add -sou

[jira] [Commented] (HDFS-9730) Storage ID update does not happen when there is a layout change

2016-02-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127089#comment-15127089 ] Chris Trezzo commented on HDFS-9730: (As well as 2.7.3) > Storage ID update does not h

[jira] [Commented] (HDFS-9730) Storage ID update does not happen when there is a layout change

2016-02-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127085#comment-15127085 ] Chris Trezzo commented on HDFS-9730: Thanks [~kihwal] for the catch. I am not quite sur

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126809#comment-15126809 ] Chris Trezzo commented on HDFS-8578: [~szetszwo] [~vinayrpet] One major issue I see wi

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122366#comment-15122366 ] Chris Trezzo commented on HDFS-8578: Looking through the patch now. > On upgrade, Data

[jira] [Commented] (HDFS-9654) Code refactoring for HDFS-8578

2016-01-27 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119965#comment-15119965 ] Chris Trezzo commented on HDFS-9654: Sounds good! +1 on the patch. Thanks [~szetszwo].

[jira] [Commented] (HDFS-9654) Code refactoring for HDFS-8578

2016-01-26 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118203#comment-15118203 ] Chris Trezzo commented on HDFS-9654: Thanks [~szetszwo] for the updated patch. Overall

[jira] [Commented] (HDFS-1312) Re-balance disks within a Datanode

2016-01-13 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096827#comment-15096827 ] Chris Trezzo commented on HDFS-1312: I will dial into the call as well. Thanks for post

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082039#comment-15082039 ] Chris Trezzo commented on HDFS-8578: [~szetszwo] Also, we might want to break this into

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15081991#comment-15081991 ] Chris Trezzo commented on HDFS-8578: Hi [~szetszwo], thanks for the updated patch! One

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-15 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058566#comment-15058566 ] Chris Trezzo commented on HDFS-8791: Thanks [~kihwal] for the info! For the sake of com

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2015-12-10 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051472#comment-15051472 ] Chris Trezzo commented on HDFS-8578: I agree with [~szetszwo] let's fix the OOM part in

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2015-12-07 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045900#comment-15045900 ] Chris Trezzo commented on HDFS-8578: One possible solution to limit memory growth is to

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2015-12-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042020#comment-15042020 ] Chris Trezzo commented on HDFS-8578: [~vinayrpet] Woops, sorry I missed the part about

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2015-12-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042012#comment-15042012 ] Chris Trezzo commented on HDFS-8578: One big spot where I see the memory footprint pote

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2015-12-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041946#comment-15041946 ] Chris Trezzo commented on HDFS-8578: [~jrottinghuis] bq. With 12 disks and 3 namespaces

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-04 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041902#comment-15041902 ] Chris Trezzo commented on HDFS-8791: Reviewing HDFS-8578. > block ID-based DN storage

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037393#comment-15037393 ] Chris Trezzo commented on HDFS-8791: As a side note: I see that there are already multi

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037383#comment-15037383 ] Chris Trezzo commented on HDFS-8791: [~kihwal] For the sake of completeness, we upgrade

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: test-node-upgrade.txt > block ID-based DN storage layout can be very slow for datanode on e

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: test-node-upgrade.txt > block ID-based DN storage layout can be very slow for datanode on e

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: (was: test-node-upgrade.txt) > block ID-based DN storage layout can be very slow for da

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036701#comment-15036701 ] Chris Trezzo commented on HDFS-8791: Thanks [~kihwal] for the testing info! I definitel

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036168#comment-15036168 ] Chris Trezzo commented on HDFS-8791: If I have my Twitter hat on, like [~jrottinghuis]

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034662#comment-15034662 ] Chris Trezzo commented on HDFS-8791: I re-kicked another build: https://builds.apache.

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Status: Patch Available (was: Open) > block ID-based DN storage layout can be very slow for datanode o

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-01 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Status: Open (was: Patch Available) > block ID-based DN storage layout can be very slow for datanode o

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-11-30 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Status: Patch Available (was: Open) > block ID-based DN storage layout can be very slow for datanode o

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-11-30 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: hadoop-56-layout-datanode-dir.tgz HDFS-8791-trunk-v2.patch [~cmccabe] Attac

  1   2   >