> On June 5, 2014, 8:43 a.m., justin coffey wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java, > > line 165 > > <https://reviews.apache.org/r/22174/diff/3/?file=603954#file603954line165> > > > > A stupid question perhaps, but is INT96 reserved for timestamps in > > parquet? > > > > I dug this up, but not sure if it's definitive: > > https://github.com/Parquet/parquet-mr/issues/101
Yea I dont think its reserved, but there is missing an OriginalType annotation called 'Timestamp' in parquet for the application to recognize, which will require yet another parquet version-bump. Do you think we can go ahead with it now and then add it later in a follow-up JIRA? Or wait for that to be added first? - Szehon ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22174/#review44805 ----------------------------------------------------------- On June 5, 2014, 7:33 a.m., Szehon Ho wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/22174/ > ----------------------------------------------------------- > > (Updated June 5, 2014, 7:33 a.m.) > > > Review request for hive, Brock Noland, justin coffey, and Xuefu Zhang. > > > Bugs: HIVE-6394 > https://issues.apache.org/jira/browse/HIVE-6394 > > > Repository: hive-git > > > Description > ------- > > This uses the Jodd library to convert java.sql.Timestamp type used by Hive > into the {julian-day:nanos} format expected by parquet, and vice-versa. > > > Diffs > ----- > > data/files/parquet_types.txt 0be390b > pom.xml 4bb8880 > ql/pom.xml 13c477a > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java > 4da0d30 > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java > 29f7e11 > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java > 57161d8 > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java > fb2f5a8 > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/utils/NanoTimeUtils.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java > 3490061 > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestParquetTimestampUtils.java > PRE-CREATION > ql/src/test/queries/clientpositive/parquet_types.q 5d6333c > ql/src/test/results/clientpositive/parquet_types.q.out c23f7f1 > > Diff: https://reviews.apache.org/r/22174/diff/ > > > Testing > ------- > > Unit tests the new libraries, and also added timestamp data in the > "parquet_types" q-test. > > > Thanks, > > Szehon Ho > >
