[
https://issues.apache.org/jira/browse/HIVE-948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13623036#comment-13623036
]
Hudson commented on HIVE-948:
-----------------------------
Integrated in Hive-trunk-hadoop2 #138 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2/138/])
HIVE-948: more query plan optimization rules (Navis via Ashutosh Chauhan)
(Revision 1449981)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1449981
Files :
* /hive/trunk/contrib/src/test/results/clientpositive/serde_typedbytes.q.out
* /hive/trunk/contrib/src/test/results/clientpositive/serde_typedbytes5.q.out
* /hive/trunk/hbase-handler/src/test/results/positive/hbase_queries.q.out
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/NonBlockingOpDeDupProc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicateTransitivePropagate.java
* /hive/trunk/ql/src/test/queries/clientpositive/nonblock_op_deduplicate.q
* /hive/trunk/ql/src/test/results/clientnegative/bucket_mapjoin_mismatch1.q.out
*
/hive/trunk/ql/src/test/results/clientnegative/sortmerge_mapjoin_mismatch_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/alias_casted_column.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ambiguous_col.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join14_hadoop20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join17.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join19.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/binarysortable_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucket_groupby.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucket_map_join_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucket_map_join_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out
* /hive/trunk/ql/src/test/results/clientpositive/create_view.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby10.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby1_map.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby1_map_nomap.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby1_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby1_noskew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby2_map.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby2_map_multi_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby2_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby2_noskew.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby2_noskew_multi_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby3_map.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby3_map_multi_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby3_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby3_noskew.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby3_noskew_multi_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby4_map.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby4_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby5_map.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby5_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby5_noskew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby7_map.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby7_map_multi_single_reducer.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby7_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby7_noskew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby8_map.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby8_map_skew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby8_noskew.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_cube1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_grouping_sets5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby_multi_single_reducer2.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby_mutli_insert_common_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_position.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_rollup1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_self_join.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_compression.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/infer_bucket_sort_dyn_part.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/infer_bucket_sort_map_operators.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/infer_bucket_sort_num_buckets.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input12_hadoop20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input14.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input14_limit.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input17.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input18.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input22.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input30.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input31.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input32.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input34.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input35.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input36.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_testsequencefile.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join14_hadoop20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join17.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join22.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join25.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join26.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join27.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join29.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join30.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join31.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join35.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join36.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join37.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join38.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join39.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join40.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join_view.q.out
* /hive/trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out
* /hive/trunk/ql/src/test/results/clientpositive/lineage1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/mapjoin1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/mapjoin_distinct.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out
* /hive/trunk/ql/src/test/results/clientpositive/merge1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/merge2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/multigroupby_singlemr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/noalias_subq1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nonblock_op_deduplicate.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nonmr_fetch.q.out
* /hive/trunk/ql/src/test/results/clientpositive/notable_alias1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/notable_alias2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/parallel.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_constant_expr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_gby.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_gby2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_multi_insert.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_random.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_udf_col.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_union.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out
* /hive/trunk/ql/src/test/results/clientpositive/quote1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rcfile_merge1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rcfile_merge2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rcfile_null_value.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sample9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/select_as_omitted.q.out
* /hive/trunk/ql/src/test/results/clientpositive/select_transform_hint.q.out
* /hive/trunk/ql/src/test/results/clientpositive/semijoin.q.out
* /hive/trunk/ql/src/test/results/clientpositive/skewjoin.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_10.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_14.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_15.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_16.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/split_sample.q.out
* /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/subq.q.out
* /hive/trunk/ql/src/test/results/clientpositive/subq2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/type_widening.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_to_unix_timestamp.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union10.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union25.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union28.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union29.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union30.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union31.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union32.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union33.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union_lateralview.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union_view.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
> more query plan optimization rules
> -----------------------------------
>
> Key: HIVE-948
> URL: https://issues.apache.org/jira/browse/HIVE-948
> Project: Hive
> Issue Type: Improvement
> Reporter: Ning Zhang
> Assignee: Navis
> Fix For: 0.11.0
>
> Attachments: HIVE-948.D8463.1.patch, HIVE-948.D8463.2.patch,
> HIVE-948.D8463.3.patch, HIVE-948.D8463.3.patch, HIVE-948.D8463.4.patch,
> HIVE-948.D8463.5.patch, HIVE-948.testresult_only.1.txt
>
>
> Many query plans are not optimal in that they contain redundant operators.
> Some examples are unnecessary select operators (select followed by select,
> select output being the same as input etc.). Even though these operators are
> not very expensive, they could account for around 10% of CPU time in some
> simple queries. It seems they are low-hanging fruits that we should pick
> first.
> BTW, it seems these optimization rules should be added at the last stage of
> the physical optimization phase since some redundant operators are added to
> facilitate physical plan generation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
