-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35950/
-----------------------------------------------------------
(Updated June 28, 2015, 12:29 a.m.)
Review request for hive, Ryan Blue, cheng xu, and Dong Chen.
Bugs: HIVE-11131
https://issues.apache.org/jira/browse/HIVE-11131
Repository: hive-git
Description
-------
Implemented data type writers that will be created before the first Hive row is
written to Parquet. These writers contain information about object inspectors
and schema of a specific data type, and calls the specific addXXXX() method
used by Parquet for each data type.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java
c195c3ec3ddae19bf255fc2c9633f8bf4390f428
Diff: https://reviews.apache.org/r/35950/diff/
Testing (updated)
-------
Tests from TestDataWritableWriter run OK.
I run other tests with micro-becnhmarks, and I got some better results from
this new implemntation:
Using repeated rows across the file, this is the throughput increase using 1
million records:
bigint boolean double float int string
7.598 7.491 7.488 7.588 7.53 0.270 (before)
10.137 11.511 10.155 10.297 10.242 0.286 (after)
Using random rows across the file, the is the throughput increase using 1
million records:
bigint boolean double float int string
5.268 7.723 4.107 4.173 4.729 0.20 (before)
6.236 10.466 5.944 4.749 5.234 0.22 (after)
Thanks,
Sergio Pena
