Author: vikram Date: Fri Jan 23 19:39:56 2015 New Revision: 1654344 URL: http://svn.apache.org/r1654344 Log: HIVE-9235: Turn off Parquet Vectorization until all data types work: DECIMAL, DATE, TIMESTAMP, CHAR, and VARCHAR (Matt McCline via Vikram Dixit K)
Modified: hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out Modified: hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java URL: http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java?rev=1654344&r1=1654343&r2=1654344&view=diff ============================================================================== --- hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java (original) +++ hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java Fri Jan 23 19:39:56 2015 @@ -31,9 +31,10 @@ import parquet.hadoop.ParquetInputFormat * * A Parquet InputFormat for Hive (with the deprecated package mapred) * + * NOTE: With HIVE-9235 we removed "implements VectorizedParquetInputFormat" since all data types + * are not currently supported. Removing the interface turns off vectorization. */ -public class MapredParquetInputFormat extends FileInputFormat<Void, ArrayWritable> - implements VectorizedInputFormatInterface { +public class MapredParquetInputFormat extends FileInputFormat<Void, ArrayWritable> { private static final Log LOG = LogFactory.getLog(MapredParquetInputFormat.class); Modified: hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out URL: http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out?rev=1654344&r1=1654343&r2=1654344&view=diff ============================================================================== --- hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out (original) +++ hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out Fri Jan 23 19:39:56 2015 @@ -149,7 +149,6 @@ STAGE PLANS: Map-reduce partition columns: _col0 (type: tinyint) Statistics: Num rows: 12288 Data size: 73728 Basic stats: COMPLETE Column stats: NONE value expressions: _col1 (type: int), _col2 (type: smallint), _col3 (type: bigint), _col4 (type: struct<count:bigint,sum:double,input:float>), _col5 (type: struct<count:bigint,sum:double,variance:double>) - Execution mode: vectorized Reducer 2 Reduce Operator Tree: Group By Operator Modified: hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out URL: http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out?rev=1654344&r1=1654343&r2=1654344&view=diff ============================================================================== --- hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out (original) +++ hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out Fri Jan 23 19:39:56 2015 @@ -84,7 +84,6 @@ STAGE PLANS: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - Execution mode: vectorized Stage: Stage-0 Fetch Operator @@ -161,7 +160,6 @@ STAGE PLANS: Map-reduce partition columns: _col0 (type: tinyint) Statistics: Num rows: 12288 Data size: 73728 Basic stats: COMPLETE Column stats: NONE value expressions: _col1 (type: int), _col2 (type: smallint), _col3 (type: bigint), _col4 (type: struct<count:bigint,sum:double,input:float>), _col5 (type: struct<count:bigint,sum:double,variance:double>) - Execution mode: vectorized Reduce Operator Tree: Group By Operator aggregations: max(VALUE._col0), min(VALUE._col1), count(VALUE._col2), avg(VALUE._col3), stddev_pop(VALUE._col4)