[ https://issues.apache.org/jira/browse/DRILL-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16771544#comment-16771544 ]
weijie.tong commented on DRILL-6871: ------------------------------------ [~kkhatua] please make sure whether it has been fixed now and close it if it has been fixed now. > Enabling runtime filter eliminates more incoming rows than it should. > --------------------------------------------------------------------- > > Key: DRILL-6871 > URL: https://issues.apache.org/jira/browse/DRILL-6871 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow > Affects Versions: 1.15.0 > Reporter: Kunal Khatua > Assignee: weijie.tong > Priority: Major > Fix For: 1.16.0 > > Attachments: 24036b35-7c41-12c9-8c08-9327c8d8fdf2.sys.drill, > 24036bbf-ede2-690a-93b7-9b43c28eed52.sys.drill, > 24036c15-54a9-7b5d-07be-4d6de4f63731.sys.drill, > 24036c22-7f39-ae8b-f8f4-000ecc2572cc.sys.drill > > > When testing with the following combination on TPC-H dataset (scale factor > 100) using a 4 node setup... > {code:java} > exec.hashjoin.bloom_filter.fpp=0.2 > exec.hashjoin.enable.runtime_filter=true > exec.hashjoin.runtime_filter.max.waiting.time=20000 > {code} > It was observed that the filter eliminates more rows than it should. > > {code:java} > 0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem > l, supplier s where l.l_suppkey = s.s_suppkey and s.s_acctbal <1000); > +---------+ > | EXPR$0 | > +---------+ > | 405566 | > +---------+ > 1 row selected (10.565 seconds) > 0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem > l, supplier s where l.l_suppkey = s.s_suppkey and s.s_acctbal <1000); > +---------+ > | EXPR$0 | > +---------+ > | 405769 | > +---------+ > 1 row selected (9.845 seconds) > {code} > The expected row count for the above (broadcast-join) query should have been > *109307880* > > {code:java} > 0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem > l, orders o where o.o_orderkey = l.l_orderkey and o.o_totalprice < 100000); > +-----------+ > | EXPR$0 | > +-----------+ > | 37338355 | > +-----------+ > 1 row selected (44.698 seconds) > 0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem > l, orders o where o.o_orderkey = l.l_orderkey and o.o_totalprice < 100000); > +-----------+ > | EXPR$0 | > +-----------+ > | 38044874 | > +-----------+ > 1 row selected (44.871 seconds) > {code} > The expected row count for the above (hash partition-join) query should have > been *96176495* -- This message was sent by Atlassian JIRA (v7.6.3#76005)