Author: vikram
Date: Fri Jan 23 19:39:56 2015
New Revision: 1654344
URL: http://svn.apache.org/r1654344
Log:
HIVE-9235: Turn off Parquet Vectorization until all data types work: DECIMAL,
DATE, TIMESTAMP, CHAR, and VARCHAR (Matt McCline via Vikram Dixit K)
Modified:
hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out
hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out
Modified:
hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
URL:
http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java?rev=1654344&r1=1654343&r2=1654344&view=diff
==============================================================================
---
hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
(original)
+++
hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
Fri Jan 23 19:39:56 2015
@@ -31,9 +31,10 @@ import parquet.hadoop.ParquetInputFormat
*
* A Parquet InputFormat for Hive (with the deprecated package mapred)
*
+ * NOTE: With HIVE-9235 we removed "implements VectorizedParquetInputFormat"
since all data types
+ * are not currently supported. Removing the interface turns off
vectorization.
*/
-public class MapredParquetInputFormat extends FileInputFormat<Void,
ArrayWritable>
- implements VectorizedInputFormatInterface {
+public class MapredParquetInputFormat extends FileInputFormat<Void,
ArrayWritable> {
private static final Log LOG =
LogFactory.getLog(MapredParquetInputFormat.class);
Modified:
hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out
URL:
http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out?rev=1654344&r1=1654343&r2=1654344&view=diff
==============================================================================
---
hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out
(original)
+++
hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out
Fri Jan 23 19:39:56 2015
@@ -149,7 +149,6 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint)
Statistics: Num rows: 12288 Data size: 73728 Basic
stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: int), _col2 (type:
smallint), _col3 (type: bigint), _col4 (type:
struct<count:bigint,sum:double,input:float>), _col5 (type:
struct<count:bigint,sum:double,variance:double>)
- Execution mode: vectorized
Reducer 2
Reduce Operator Tree:
Group By Operator
Modified:
hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out
URL:
http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out?rev=1654344&r1=1654343&r2=1654344&view=diff
==============================================================================
---
hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out
(original)
+++
hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out
Fri Jan 23 19:39:56 2015
@@ -84,7 +84,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde:
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Execution mode: vectorized
Stage: Stage-0
Fetch Operator
@@ -161,7 +160,6 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint)
Statistics: Num rows: 12288 Data size: 73728 Basic stats:
COMPLETE Column stats: NONE
value expressions: _col1 (type: int), _col2 (type:
smallint), _col3 (type: bigint), _col4 (type:
struct<count:bigint,sum:double,input:float>), _col5 (type:
struct<count:bigint,sum:double,variance:double>)
- Execution mode: vectorized
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0), min(VALUE._col1),
count(VALUE._col2), avg(VALUE._col3), stddev_pop(VALUE._col4)
