Khurram Faraaz created DRILL-3881: ------------------------------------- Summary: Rowkey filter does not get pushed into Scan Key: DRILL-3881 URL: https://issues.apache.org/jira/browse/DRILL-3881 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Affects Versions: 1.2.0 Reporter: Khurram Faraaz Assignee: Smidth Panchamia
Rowkey filter does not get pushed down into Scan 4 node cluster CentOS Drill master commit ID: b9afcf8f case 1) Rowkey filter does not get pushed into Scan {code} 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE ROW_KEY = CAST('3.0838087E38' AS FLOAT); +------+------+ | text | json | +------+------+ | 00-00 Screen 00-01 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))]) 00-02 SelectionVectorRemover 00-03 Filter(condition=[=($0, CAST('3.0838087E38'):FLOAT NOT NULL)]) 00-04 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=null, stopRow=null, filter=null], columns=[`*`]]]) {code} case 2) Rowkey filter does not get pushed into Scan {code} 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' order by CONVERT_FROM(ROW_KEY,'FLOAT_OB') ASC; +------+------+ | text | json | +------+------+ | 00-00 Screen 00-01 Project(RK=[$0], EXPR$1=[$1]) 00-02 SelectionVectorRemover 00-03 Sort(sort0=[$0], dir0=[ASC]) 00-04 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))]) 00-05 SelectionVectorRemover 00-06 Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))]) 00-07 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]]) {code} Same as case (2) just that ASC is missing in order by clause. {code} 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' order by CONVERT_FROM(ROW_KEY,'FLOAT_OB'); +------+------+ | text | json | +------+------+ | 00-00 Screen 00-01 Project(RK=[$0], EXPR$1=[$1]) 00-02 SelectionVectorRemover 00-03 Sort(sort0=[$0], dir0=[ASC]) 00-04 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))]) 00-05 SelectionVectorRemover 00-06 Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))]) 00-07 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]]) {code} Snippet that creates and inserts data into HBase table. {code} public static void main(String args[]) throws IOException { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.property.clientPort","5181"); HBaseAdmin admin = new HBaseAdmin(conf); if (admin.tableExists("flt_Tbl")) { admin.disableTable("flt_Tbl"); admin.deleteTable("flt_Tbl"); } HTableDescriptor tableDesc = new HTableDescriptor(TableName.valueOf("flt_Tbl")); tableDesc.addFamily(new HColumnDescriptor("colfam1")); admin.createTable(tableDesc); HTable table = new HTable(conf, "flt_Tbl"); //for (float i = (float)0.5; i <= 100.00; i += 0.75) { for (float i = (float)1.4E-45; i <= Float.MAX_VALUE; i += Float.MAX_VALUE / 64) { byte[] bytes = new byte[5]; org.apache.hadoop.hbase.util.PositionedByteRange br = new org.apache.hadoop.hbase.util.SimplePositionedByteRange(bytes, 0, 5); org.apache.hadoop.hbase.util.OrderedBytes.encodeFloat32(br, i, org.apache.hadoop.hbase.util.Order.DESCENDING); Put p = new Put(bytes); p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),String.format("value %05f", i).getBytes()); table.put(p); } table.flushCommits(); table.close(); } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)