----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40590/ -----------------------------------------------------------
Review request for hive and Ashutosh Chauhan. Repository: hive-git Description ------- Following HIVE-12005, HIVE-12164, we have removed jdbc and hbase stats collection mechanism. Now we are targeting counter based stats collection mechanism. The main advantages are as follows (1) counter based stats has limitation on the length of the counter itself, if it is too long, MD5 will be applied. (2) when there are a large number of partitions and columns, we need to create a large number of counters in memory. This will put a heavy load on the M/R AM or Tez AM etc. FS based stats will do a better job. Diffs ----- common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 0a44bde common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f48403b data/conf/llap/hive-site.xml 9e15eda data/conf/spark/standalone/hive-site.xml 1a45274 data/conf/spark/yarn-client/hive-site.xml 346c248 data/conf/tez/hive-site.xml 9e8b04f ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregator.java 9b66024 ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregatorSpark.java 7ac01a7 ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregatorTez.java bb51fea ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsPublisher.java ab3d3cf ql/src/java/org/apache/hadoop/hive/ql/stats/StatsFactory.java a53fcc0 ql/src/test/queries/clientpositive/index_bitmap3.q ed29af9 ql/src/test/queries/clientpositive/index_bitmap_auto.q 2d434d1 ql/src/test/queries/clientpositive/stats_counter.q 3c1f132 ql/src/test/queries/clientpositive/stats_counter_partitioned.q e1274c0 ql/src/test/results/clientpositive/llap/stats_counter.q.out 8b3dcea ql/src/test/results/clientpositive/llap/stats_counter_partitioned.q.out 626dcff ql/src/test/results/clientpositive/spark/stats_counter.q.out 8b3dcea ql/src/test/results/clientpositive/spark/stats_counter_partitioned.q.out 12e1fbe ql/src/test/results/clientpositive/stats_counter.q.out 8b3dcea ql/src/test/results/clientpositive/stats_counter_partitioned.q.out 626dcff ql/src/test/results/clientpositive/tez/stats_counter.q.out 8b3dcea ql/src/test/results/clientpositive/tez/stats_counter_partitioned.q.out 626dcff Diff: https://reviews.apache.org/r/40590/diff/ Testing ------- Thanks, pengcheng xiong
