[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13547940#comment-13547940 ] Hudson commented on HIVE-1538: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-2791: filter is still removed due to regression of HIVE-1538 althougth HIVE-2344 (binlijin via hashutosh) (Revision 1291916) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1291916 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java * /hive/trunk/ql/src/test/queries/clientpositive/ppd2.q * /hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.8.0 Attachments: patch-1538-1.txt, patch-1538-2.txt, patch-1538-3.txt, patch-1538-4.txt, patch-1538.txt With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13212987#comment-13212987 ] Hudson commented on HIVE-1538: -- Integrated in Hive-trunk-h0.21 #1268 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1268/]) HIVE-2791: filter is still removed due to regression of HIVE-1538 althougth HIVE-2344 (binlijin via hashutosh) (Revision 1291916) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1291916 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java * /hive/trunk/ql/src/test/queries/clientpositive/ppd2.q * /hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.8.0 Attachments: patch-1538-1.txt, patch-1538-2.txt, patch-1538-3.txt, patch-1538-4.txt, patch-1538.txt With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13083847#comment-13083847 ] Hudson commented on HIVE-1538: -- Integrated in Hive-trunk-h0.21 #889 (See [https://builds.apache.org/job/Hive-trunk-h0.21/889/]) HIVE-1538. filter is removed due to regression of HIVE-1538 (Amareshwari Sriramadasu via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1156787 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java * /hive/trunk/ql/src/test/results/clientpositive/ppd_udf_col.q.out * /hive/trunk/ql/src/test/queries/clientpositive/ppd_udf_col.q * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.8.0 Attachments: patch-1538-1.txt, patch-1538-2.txt, patch-1538-3.txt, patch-1538-4.txt, patch-1538.txt With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064933#comment-13064933 ] Hudson commented on HIVE-1538: -- Integrated in Hive-trunk-h0.21 #825 (See [https://builds.apache.org/job/Hive-trunk-h0.21/825/]) HIVE-1538. FilterOperator is applied twice with ppd on. (Amareshwari Sriramadasu via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1146129 Files : * /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out * /hive/trunk/ql/src/test/results/clientpositive/bucket3.q.out * /hive/trunk/ql/src/test/results/clientpositive/sample6.q.out * /hive/trunk/ql/src/test/results/clientpositive/nullgroup4.q.out * /hive/trunk/ql/src/test/results/clientpositive/join9.q.out * /hive/trunk/ql/src/test/queries/clientpositive/ppd1.q * /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out * /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part13.q.out * /hive/trunk/ql/src/test/results/clientpositive/having.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out * /hive/trunk/ql/src/test/queries/clientpositive/ppd_join2.q * /hive/trunk/ql/src/test/results/clientpositive/join26.q.out * /hive/trunk/ql/src/test/results/clientpositive/ppd_gby2.q.out * /hive/trunk/ql/src/test/results/clientpositive/merge_dynamic_partition2.q.out * /hive/trunk/ql/src/test/results/clientpositive/input9.q.out * /hive/trunk/ql/src/test/results/clientpositive/split_sample.q.out * /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml * /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out * /hive/trunk/ql/src/test/results/clientpositive/notable_alias1.q.out * /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml * /hive/trunk/ql/src/test/results/clientpositive/input11_limit.q.out * /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml * /hive/trunk/ql/src/test/results/clientpositive/udf_10_trims.q.out * /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out * /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out * /hive/trunk/ql/src/test/results/clientpositive/udf_second.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java * /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml * /hive/trunk/ql/src/test/results/clientpositive/index_auto_unused.q.out * /hive/trunk/ql/src/test/results/clientpositive/udf_size.q.out * /hive/trunk/ql/src/test/results/clientpositive/join12.q.out * /hive/trunk/ql/src/test/queries/clientpositive/ppd_transform.q * /hive/trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out * /hive/trunk/ql/src/test/results/clientpositive/sample3.q.out * /hive/trunk/ql/src/test/results/clientpositive/join6.q.out * /hive/trunk/ql/src/test/results/clientpositive/input26.q.out * /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part10.q.out * /hive/trunk/ql/src/test/results/clientpositive/cast1.q.out * /hive/trunk/ql/src/test/queries/clientpositive/ppd_gby.q * /hive/trunk/ql/src/test/results/clientpositive/input2_limit.q.out * /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out * /hive/trunk/ql/src/test/results/clientpositive/join40.q.out * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml * /hive/trunk/ql/src/test/results/clientpositive/join23.q.out * /hive/trunk/ql/src/test/results/clientpositive/input6.q.out * /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out * /hive/trunk/ql/src/test/results/clientpositive/subq2.q.out * /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml * /hive/trunk/hbase-handler/src/test/results/hbase_queries.q.out * /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml * /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out * /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out * /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml * /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out * /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out * /hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out * /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out * /hive/trunk/hbase-handler/src/test/results/hbase_pushdown.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_multiple.q.out * /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml * /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part4.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out *
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13064355#comment-13064355 ] jirapos...@reviews.apache.org commented on HIVE-1538: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/550/ --- (Updated 2011-07-13 04:40:26.685437) Review request for hive, Yongqiang He and namit jain. Changes --- Patch re-based to trunk Summary --- Patch updated to trunk with newly added configuration var hive.ppd.remove.duplicatefilters This addresses bug HIVE-1538. https://issues.apache.org/jira/browse/HIVE-1538 Diffs (updated) - trunk/ql/src/test/results/clientpositive/sample7.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample9.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample8.q.out 1145463 trunk/ql/src/test/results/clientpositive/semijoin.q.out 1145463 trunk/ql/src/test/results/clientpositive/set_processor_namespaces.q.out 1145463 trunk/ql/src/test/results/clientpositive/skewjoin.q.out 1145463 trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1145463 trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1145463 trunk/ql/src/test/results/clientpositive/split_sample.q.out 1145463 trunk/ql/src/test/results/clientpositive/stats11.q.out 1145463 trunk/ql/src/test/results/clientpositive/stats2.q.out 1145463 trunk/ql/src/test/results/clientpositive/subq.q.out 1145463 trunk/ql/src/test/results/clientpositive/subq2.q.out 1145463 trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out 1145463 trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf1.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf9.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_10_trims.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_hour.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_isnull_isnotnull.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_like.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_lower.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_minute.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_notequal.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_parse_url.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_second.q.out 1145463 trunk/ql/src/test/results/clientpositive/udf_size.q.out 1145463 trunk/ql/src/test/results/clientpositive/union.q.out 1145463 trunk/ql/src/test/results/clientpositive/union20.q.out 1145463 trunk/ql/src/test/results/clientpositive/union22.q.out 1145463 trunk/ql/src/test/results/clientpositive/union_ppr.q.out 1145463 trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml 1145463 trunk/ql/src/test/results/compiler/plan/cast1.q.xml 1145463 trunk/ql/src/test/results/clientpositive/sample6.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample5.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample3.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample4.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample2.q.out 1145463 trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample1.q.out 1145463 trunk/ql/src/test/results/clientpositive/sample10.q.out 1145463 trunk/ql/src/test/results/clientpositive/regexp_extract.q.out 1145463 trunk/ql/src/test/results/clientpositive/regex_col.q.out 1145463 trunk/ql/src/test/results/clientpositive/reduce_deduplicate.q.out 1145463 trunk/ql/src/test/results/clientpositive/rcfile_null_value.q.out 1145463 trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out 1145463 trunk/ql/src/test/results/clientpositive/query_result_fileformat.q.out 1145463 trunk/ql/src/test/results/clientpositive/quote1.q.out 1145463 trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1145463 trunk/ql/src/test/results/clientpositive/ppd_union.q.out 1145463 trunk/ql/src/test/results/clientpositive/ppr_pushdown3.q.out 1145463 trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out 1145463 trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out 1145463 trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out 1145463 trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 1145463 trunk/ql/src/test/results/clientpositive/case_sensitivity.q.out 1145463 trunk/ql/src/test/results/clientpositive/cast1.q.out 1145463 trunk/ql/src/test/results/clientpositive/cluster.q.out 1145463 trunk/ql/src/test/results/clientpositive/combine2.q.out 1145463
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13061648#comment-13061648 ] John Sichi commented on HIVE-1538: -- Sorry for the repeated rebase requests; we'll make sure the next attempt gets committed! FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-1538-1.txt, patch-1538-2.txt, patch-1538-3.txt, patch-1538.txt With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015943#comment-13015943 ] jirapos...@reviews.apache.org commented on HIVE-1538: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/550/ --- Review request for hive, Yongqiang He and namit jain. Summary --- Patch updated to trunk with newly added configuration var hive.ppd.remove.duplicatefilters This addresses bug HIVE-1538. https://issues.apache.org/jira/browse/HIVE-1538 Diffs - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1088944 trunk/contrib/src/test/results/clientpositive/dboutput.q.out 1088944 trunk/contrib/src/test/results/clientpositive/serde_typedbytes4.q.out 1088944 trunk/hbase-handler/src/test/results/hbase_pushdown.q.out 1088944 trunk/hbase-handler/src/test/results/hbase_queries.q.out 1088944 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1088944 trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java 1088944 trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java 1088944 trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1088944 trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpWalkerInfo.java 1088944 trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicatePushDown.java 1088944 trunk/ql/src/test/queries/clientpositive/ppd1.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_clusterby.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_constant_expr.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_gby.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_gby2.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_gby_join.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_join.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_join2.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_join3.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_multi_insert.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_outer_join1.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_outer_join2.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_outer_join3.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_outer_join4.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_random.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_transform.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_udf_case.q 1088944 trunk/ql/src/test/queries/clientpositive/ppd_union.q 1088944 trunk/ql/src/test/results/clientpositive/auto_join0.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join11.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join12.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join13.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join14.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join19.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join20.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join21.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join23.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join27.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join4.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join5.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join6.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join7.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1088944 trunk/ql/src/test/results/clientpositive/auto_join9.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucket2.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucket3.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucket4.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucket_groupby.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out 1088944 trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 1088944 trunk/ql/src/test/results/clientpositive/case_sensitivity.q.out 1088944 trunk/ql/src/test/results/clientpositive/cast1.q.out 1088944 trunk/ql/src/test/results/clientpositive/cluster.q.out 1088944 trunk/ql/src/test/results/clientpositive/combine2.q.out 1088944 trunk/ql/src/test/results/clientpositive/create_view.q.out 1088944 trunk/ql/src/test/results/clientpositive/disable_merge_for_bucketing.q.out 1088944
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13009483#comment-13009483 ] Namit Jain commented on HIVE-1538: -- @Amareshwari, would it be possible to add a configuration variable to turn this off ? Just a fallbac, in case we missed something FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-1538-1.txt, patch-1538.txt With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006347#comment-13006347 ] Amareshwari Sriramadasu commented on HIVE-1538: --- Yongqiang, Can you have a look at the patch before i update it to trunk? Thanks FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-1538.txt With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[
https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12965508#action_12965508
]
He Yongqiang commented on HIVE-1538:
I am not familiar with the ppd code, just a few questions:
In ExprWalkerProcFactory.java line 291,
{noformat}
} else if (!FunctionRegistry.isOpAnd(expr)) {
{noformat}
why this is needed? For example, what will happen if it is a OpNot?
In OpProcFactory, why there is a createFilter in ScriptPPD, JoinPPD, and
DefaultPPD?
FilterOperator is applied twice with ppd on.
Key: HIVE-1538
URL: https://issues.apache.org/jira/browse/HIVE-1538
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Amareshwari Sriramadasu
Assignee: Amareshwari Sriramadasu
Fix For: 0.7.0
Attachments: patch-1538.txt
With hive.optimize.ppd set to true, FilterOperator is applied twice. And it
seems second operator is always filtering zero rows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[
https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12965509#action_12965509
]
He Yongqiang commented on HIVE-1538:
also in OpProcFactory
{noformat}
if (!((FilterOperator)op).getConf().getIsSamplingPred()) {
{noformat}
can we just look at the parent to see if it is a tso?
FilterOperator is applied twice with ppd on.
Key: HIVE-1538
URL: https://issues.apache.org/jira/browse/HIVE-1538
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Amareshwari Sriramadasu
Assignee: Amareshwari Sriramadasu
Fix For: 0.7.0
Attachments: patch-1538.txt
With hive.optimize.ppd set to true, FilterOperator is applied twice. And it
seems second operator is always filtering zero rows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12931447#action_12931447 ] Namit Jain commented on HIVE-1538: -- Talked to Amareshwari offline - the proposal looks good. As a follow-up, we can merge consecutive filters in a follow-up task FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929976#action_12929976 ] Amareshwari Sriramadasu commented on HIVE-1538: --- There are a couple of issues in removing the original filter operator. # All the expressions in the filter predicate may not be pushed. ** I'm planning to create a filter operator with non-final candidates as a child of the original filter op and mark the original filter op for deletion. # The candidate predicates may not pushed past some operators. For ex. Outer Join operator does not allow candidates for all aliases; LIMIT/SCRIPT/UDTF operators do not push any predicates. ** I'm planning to create a filter operator with unpushed predicates, as a child of the operator through which the predicates could not be pushed. Finally, remove the original filter operators which are marked for deletion. Thoughts? Any suggestions? FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12925777#action_12925777 ] Amareshwari Sriramadasu commented on HIVE-1538: --- bq. I think the solution is to collect the operators who are contributing the predicates for final candidates of predicare pushdown and remove them from the final operator graph. This does not work as I thought earlier, because all the predicates in the FilterOperator may not be pushed. We might have to reconstruct the FilterOperator with un-pushed predicates. FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12921964#action_12921964 ] Amareshwari Sriramadasu commented on HIVE-1538: --- Namit, I can take this up once I'm done with HIVE-474 i.e. mostly after a week. FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12921966#action_12921966 ] Namit Jain commented on HIVE-1538: -- Thanks, That will be great. It can lead to substantial improvement (10-15%) on the map-side for a large range of queries FilterOperator is applied twice with ppd on. Key: HIVE-1538 URL: https://issues.apache.org/jira/browse/HIVE-1538 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
