> On Nov. 2, 2014, 6:30 p.m., Brock Noland wrote: > > Hi, > > > > Nice work! Have you verified that some of the other parquet tools can read > > the data generated by this patch? ex. Impala or MapReduce?
I run some tests on Impala, but it does not support complex data types, such as maps or arrays. See http://www.cloudera.com/content/cloudera/en/documentation/cloudera-impala/latest/topics/impala_datatypes.html Also, see this error from a query to a parquet with map data types: ERROR: AnalysisException: Expr 'default.parquet_1.avreau_col_1' in select list of root statement returns a complex type 'MAP<STRING,STRING>'. Only scalar types are allowed in the select list of the root statement. - Sergio ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27404/#review59527 ----------------------------------------------------------- On Oct. 30, 2014, 11:43 p.m., Sergio Pena wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27404/ > ----------------------------------------------------------- > > (Updated Oct. 30, 2014, 11:43 p.m.) > > > Review request for hive. > > > Bugs: HIVE-8359 > https://issues.apache.org/jira/browse/HIVE-8359 > > > Repository: hive-git > > > Description > ------- > > The patch changes the way DataWritableWriter class writes an array of > elements to the Parquet record. > It wraps each array record into a new startGroup/endGroup block so that > Parquet can detect null values on those optional fields. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java > c7078efe27482df0a11dd68ac068da27dbcf51b3 > ql/src/test/queries/clientpositive/parquet_map_null.q PRE-CREATION > ql/src/test/results/clientpositive/parquet_map_null.q.out PRE-CREATION > > Diff: https://reviews.apache.org/r/27404/diff/ > > > Testing > ------- > > > Thanks, > > Sergio Pena > >
