[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HDFS-15288: -- Component/s: block placement > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: New Feature > Components: block placement >Affects Versions: 3.2.2, 3.4.0 >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Fix For: 3.4.0, 3.3.2 > > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch, HDFS-15288-Addendum-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HDFS-15288: -- Affects Version/s: 3.2.2 3.4.0 > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.2.2, 3.4.0 >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Fix For: 3.4.0, 3.3.2 > > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch, HDFS-15288-Addendum-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Fix Version/s: 3.3.2 > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Fix For: 3.4.0, 3.3.2 > > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch, HDFS-15288-Addendum-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Release Note: Added a new BlockPlacementPolicy: "AvailableSpaceRackFaultTolerantBlockPlacementPolicy" which uses the same optimization logic as the AvailableSpaceBlockPlacementPolicy along with spreading the replicas across maximum number of racks, similar to BlockPlacementPolicyRackFaultTolerant. The BPP can be configured by setting the blockplacement policy class as org.apache.hadoop.hdfs.server.blockmanagement.AvailableSpaceRackFaultTolerantBlockPlacementPolicy Issue Type: New Feature (was: Improvement) > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Fix For: 3.4.0 > > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch, HDFS-15288-Addendum-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Attachment: HDFS-15288-Addendum-01.patch > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Fix For: 3.4.0 > > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch, HDFS-15288-Addendum-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Fix For: 3.4.0 > > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Attachment: HDFS-15288-03.patch > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch, > HDFS-15288-03.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Attachment: HDFS-15288-02.patch > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-15288-01.patch, HDFS-15288-02.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Attachment: HDFS-15288-01.patch > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-15288-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Attachment: (was: HDFS-15288-01.patch) > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Attachment: HDFS-15288-01.patch > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-15288-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15288) Add Available Space Rack Fault Tolerant BPP
[ https://issues.apache.org/jira/browse/HDFS-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15288: Status: Patch Available (was: Open) > Add Available Space Rack Fault Tolerant BPP > --- > > Key: HDFS-15288 > URL: https://issues.apache.org/jira/browse/HDFS-15288 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-15288-01.patch > > > The Present {{AvailableSpaceBlockPlacementPolicy}} extends the Default Block > Placement policy, which makes it apt for Replicated files. But not very > efficient for EC files, which by default use. > {{BlockPlacementPolicyRackFaultTolerant}}. So propose a to add new BPP having > similar optimization as ASBPP where as keeping the spread of Blocks to max > racks, i.e as RackFaultTolerantBPP. > This could extend {{BlockPlacementPolicyRackFaultTolerant}}, rather than the > {{BlockPlacementPOlicyDefault}} like ASBPP and keep other logics of > optimization same as ASBPP -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org