----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23681/ -----------------------------------------------------------
(Updated July 24, 2014, 4:26 p.m.) Review request for drill, Jacques Nadeau and Parth Chandra. Changes ------- Removed a catch statement not needed in AbstractRecordBatch Repository: drill-git Description ------- Implemented support for fixed length repeated columns, including ones with individuals values that span across pages. Patch includes a pretty substantial refactoring of the reader as well as a fix for an outstanding issue with reading impala prduced files with dictionary encoding. Diffs (updated) ----- exec/java-exec/src/main/codegen/templates/NullableValueVectors.java f50aae8 exec/java-exec/src/main/codegen/templates/ParquetOutputRecordWriter.java 92267e7 exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java 7bf84f2 exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 7f01058 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/BitReader.java 7ae95cd exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ColumnDataReader.java 8c6f120 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ColumnReader.java 775fc73 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/FixedByteAlignedReader.java 26e1f09 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableBitReader.java 29ca30a exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableColumnReader.java 585fd66 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableFixedByteAlignedReader.java 17759d3 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableFixedByteAlignedReaders.java 76cc937 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/PageReadStatus.java 3ad1d6c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFixedWidthDictionaryReader.java c0720a9 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java 0189c9b exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRecordReader.java 703ad1f exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRecordWriter.java b26f688 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetScanBatchCreator.java df6581f exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/VarLenBinaryReader.java 813a799 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/VarLengthColumnReaders.java 56f687c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/BitReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableBitReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableVarLengthValuesColumn.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/PageReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetFixedWidthDictionaryReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetToDrillTypeConverter.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumn.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthValuesColumn.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedFixedWidthVector.java 9b0a6cd exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedVariableWidthVector.java bd03038 exec/java-exec/src/main/java/org/apache/drill/exec/vector/VariableWidthVector.java 2b07750 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java ef8aef8 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java f05ab1b exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriter.java aa2b66f exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetRecordReaderTest.java 3e679bb exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java 4a0efc9 exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java 2d2a2ec exec/java-exec/src/test/resources/parquet/alltypes_repeated.json PRE-CREATION exec/java-exec/src/test/resources/parquet/basic_repeated.json PRE-CREATION exec/java-exec/src/test/resources/parquet/null_test_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/parquet_repeated_performance_test_input_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/repeated_bool_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/repeated_double_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/repeated_integer_data.json PRE-CREATION Diff: https://reviews.apache.org/r/23681/diff/ Testing ------- smoke tests, unit tests Thanks, Jason Altekruse
