[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Hadoop Flags: Reviewed > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Fix For: 3.1.0 > > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch, HDFS-12591-HDFS-9806.007.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Resolution: Fixed Status: Resolved (was: Patch Available) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch, HDFS-12591-HDFS-9806.007.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HDFS-12591: - Attachment: HDFS-12591-HDFS-9806.007.patch Really minor tweak, so the opt can be set programmatically without also setting it in the config (moves the throw to {{createDB}}) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch, HDFS-12591-HDFS-9806.007.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Status: Patch Available (was: Open) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Attachment: (was: HDFS-12591-HDFS-9806.006.patch) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.006.patch Thanks for taking a look [~chris.douglas]. Posting a modified patch (v006) based on the comments -- it throws an {{IllegalArgumentException}} in {{LevelDBOptions.setConf}}, if {{DFS_PROVIDED_ALIASMAP_LEVELDB_PATH}} is not set correctly. Also, I removed the {{@VisibleForTesting}} for {{createDB}}, made it static and added a {{boolean}} parameter to determine if the db has to be created (when being written) or not (when being read). > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.006.patch > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Status: Open (was: Patch Available) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch, > HDFS-12591-HDFS-9806.006.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Status: Patch Available (was: Open) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Status: Open (was: Patch Available) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.005.patch Fixing the checkstyle and findbugs issues in patch v4 in patch v5. > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Status: Patch Available (was: Open) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Status: Open (was: Patch Available) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch, HDFS-12591-HDFS-9806.005.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.004.patch Posting a new patch (v4) that rebases v3 on the latest feature branch, and removes the shared reference to the leveldb between {{LevelDBReader}} and {{LevelDBWriter}}. The reader and writer are not intended to be used simultaneously as provided storage just supports read-only as of now (hence, removed {{testSimultaneousWriteAndRead}}). [~ehiggs], can you take a look at this? > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch, > HDFS-12591-HDFS-9806.004.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.003.patch Attaching a patch rebased on HDFS-12665 so the two LevelDB alias maps (file based and NN in-memory) use the same code for reading and writing data. > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Assignee: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch, HDFS-12591-HDFS-9806.003.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.002.patch Posting a modified version of [~ehiggs]'s patch, rebased on the latest HDFS-9806 feature branch. This applies on top of the HDFS-11902 v009 patch. I also fixed the memory leak issues pointed out by [~jeagles]. Thanks [~jeagles]! > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch, > HDFS-12591-HDFS-9806.002.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.001.patch Attaching work I've previously done for this. It needs to be rebased onto the HDFS-9806 branch. > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12591: -- Status: Patch Available (was: Open) > [READ] Implement LevelDBFileRegionFormat > > > Key: HDFS-12591 > URL: https://issues.apache.org/jira/browse/HDFS-12591 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Ewan Higgs >Priority: Minor > Attachments: HDFS-12591-HDFS-9806.001.patch > > > The existing work for HDFS-9806 uses an implementation of the {{FileRegion}} > read from a csv file. This is good for testability and diagnostic purposes, > but it is not very efficient for larger systems. > There should be a version that is similar to the {{TextFileRegionFormat}} > that instead uses LevelDB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org