[jira] [Commented] (DRILL-3492) Add support for encoding of Drill data types into byte ordered format
[ https://issues.apache.org/jira/browse/DRILL-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647932#comment-14647932 ] Smidth Panchamia commented on DRILL-3492: - Hi Aditya, Can you please review the change for this jira? This is the change in my drill fork. https://github.com/spanchamiamapr/drill/commit/70e1f3b2ca0410748b9872535bb205651e86d6c9 -- Smidth On Tue, Jul 14, 2015 at 5:11 PM, Aditya Kishore (JIRA) j...@apache.org Add support for encoding of Drill data types into byte ordered format - Key: DRILL-3492 URL: https://issues.apache.org/jira/browse/DRILL-3492 Project: Apache Drill Issue Type: New Feature Reporter: Smidth Panchamia Assignee: Smidth Panchamia Attachments: 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch, 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch, 0001-DRILL-3492-merged.patch The following JIRA added this functionality in HBase: https://issues.apache.org/jira/browse/HBASE-8201 We need to port this functionality in Drill so as to allow filtering and pruning of rows during scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3492) Add support for encoding of Drill data types into byte ordered format
[ https://issues.apache.org/jira/browse/DRILL-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3492: Attachment: 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch This change allows encoding/decoding of data from/to 'double', 'float', 'bigint', 'int' and 'utf8' data types to/from OrderedBytes format. It also allows for OrderedByte encoded row-keys to be stored in ascending as well as descending order. The following JIRA added the OrderedBytes encoding to HBase: https://issues.apache.org/jira/browse/HBASE-8201 This encoding scheme will preserve the sort-order of the native data-type when it is stored as sorted byte arrays on disk. Thus, it will help the HBase storage plugin if the row-keys have been encoded in OrderedBytes format. This functionality allows us to prune the scan ranges, thus reading much lesser data from the server. Testing Done: Added a new unit-test class TestOrderedBytesConvertFunctions.java which derives from TestConvertFunctions.java class. Also add new test cases to TestHBaseFilterPushDown class that will test if we were able to push-down filters correctly and if the results are correct. Add support for encoding of Drill data types into byte ordered format - Key: DRILL-3492 URL: https://issues.apache.org/jira/browse/DRILL-3492 Project: Apache Drill Issue Type: New Feature Reporter: Smidth Panchamia Assignee: Smidth Panchamia Attachments: 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch, 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch The following JIRA added this functionality in HBase: https://issues.apache.org/jira/browse/HBASE-8201 We need to port this functionality in Drill so as to allow filtering and pruning of rows during scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3364: Attachment: 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch Combined patch attached. Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: 0001-Add-convert_from-and-convert_to-methods-for-TIMESTAM.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3364: Attachment: 0001-PATCH-DRILL-3364-Prune-scan-range-if-the-filter-is-o.patch Attached a new combined patch after updating as per Aditya's comments. Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: 0001-Add-convert_from-and-convert_to-methods-for-TIMESTAM.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-PATCH-DRILL-3364-Prune-scan-range-if-the-filter-is-o.patch, composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3492) Add support for encoding of Drill data types into byte ordered format
[ https://issues.apache.org/jira/browse/DRILL-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3492: Attachment: 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch Combined patch to support OrderedBytes encoding of double, long, bigint and int type data. Add support for encoding of Drill data types into byte ordered format - Key: DRILL-3492 URL: https://issues.apache.org/jira/browse/DRILL-3492 Project: Apache Drill Issue Type: New Feature Reporter: Smidth Panchamia Assignee: Smidth Panchamia Attachments: 0001-DRILL-3492-Add-support-for-encoding-decoding-of-to-f.patch The following JIRA added this functionality in HBase: https://issues.apache.org/jira/browse/HBASE-8201 We need to port this functionality in Drill so as to allow filtering and pruning of rows during scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3364: Attachment: 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch The change adds some framework to handle conditions on row-key prefix. It also adds support to perform row-key range pruning when the row-key prefix is interpretted as DATE_EPOCH_BE encoded. Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3364: Attachment: 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch The change adds support to perform row-key range pruning when the row-key prefix is interpretted as TIME_EPOCH_BE, TIMESTAMP_EPOCH_BE or UINT8_BE encoded. Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: 0001-Add-convert_from-and-convert_to-methods-for-TIMESTAM.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3364: Attachment: 0001-Add-convert_from-and-convert_to-methods-for-TIMESTAM.patch Add convert_from and convert_to methods for TIMESTAMP type. This will help in scan range pruning when the query is on leading bytes of row-key and it needs to be interpreted as timestamp. Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: 0001-Add-convert_from-and-convert_to-methods-for-TIMESTAM.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, 0001-DRILL-3364-Prune-scan-range-if-the-filter-is-on-the-.patch, composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634368#comment-14634368 ] Smidth Panchamia commented on DRILL-3364: - Oh, let me test it out and update the change. Thanks! Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3492) Add support for encoding of Drill data types into byte ordered format
[ https://issues.apache.org/jira/browse/DRILL-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630333#comment-14630333 ] Smidth Panchamia commented on DRILL-3492: - Uploaded the diff to reviewboard: https://reviews.apache.org/r/36558/ Add support for encoding of Drill data types into byte ordered format - Key: DRILL-3492 URL: https://issues.apache.org/jira/browse/DRILL-3492 Project: Apache Drill Issue Type: New Feature Reporter: Smidth Panchamia Assignee: Smidth Panchamia The following JIRA added this functionality in HBase: https://issues.apache.org/jira/browse/HBASE-8201 We need to port this functionality in Drill so as to allow filtering and pruning of rows during scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628889#comment-14628889 ] Smidth Panchamia commented on DRILL-3364: - ReviewBoard link: https://reviews.apache.org/r/35893/ Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3364) Prune scan range if the filter is on the leading field with byte comparable encoding
[ https://issues.apache.org/jira/browse/DRILL-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Smidth Panchamia updated DRILL-3364: Attachment: composite.jun26.diff Patch attached. Prune scan range if the filter is on the leading field with byte comparable encoding Key: DRILL-3364 URL: https://issues.apache.org/jira/browse/DRILL-3364 Project: Apache Drill Issue Type: Sub-task Components: Storage - HBase Reporter: Aditya Kishore Assignee: Smidth Panchamia Fix For: 1.2.0 Attachments: composite.jun26.diff -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-3492) Add support for encoding of Drill data types into byte ordered format
Smidth Panchamia created DRILL-3492: --- Summary: Add support for encoding of Drill data types into byte ordered format Key: DRILL-3492 URL: https://issues.apache.org/jira/browse/DRILL-3492 Project: Apache Drill Issue Type: New Feature Reporter: Smidth Panchamia The following JIRA added this functionality in HBase: https://issues.apache.org/jira/browse/HBASE-8201 We need to port this functionality in Drill so as to allow filtering and pruning of rows during scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
