----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23681/ -----------------------------------------------------------
(Updated July 29, 2014, 3:59 a.m.) Review request for drill, Jacques Nadeau and Parth Chandra. Changes ------- Removed a line that was left over from before the reader refactoring causing a regression in larger queries. Also renamed the filenames used in some tests that were having issues when the files would not be deleted properly between tests so they would not collide. Repository: drill-git Description ------- Implemented support for fixed length repeated columns, including ones with individuals values that span across pages. Patch includes a pretty substantial refactoring of the reader as well as a fix for an outstanding issue with reading impala prduced files with dictionary encoding. Diffs (updated) ----- exec/java-exec/src/main/codegen/templates/NullableValueVectors.java f50aae8 exec/java-exec/src/main/codegen/templates/ParquetOutputRecordWriter.java 92267e7 exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java 7bf84f2 exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 7f01058 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/BitReader.java 7ae95cd exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ColumnDataReader.java 8c6f120 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ColumnReader.java 775fc73 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/FixedByteAlignedReader.java 26e1f09 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableBitReader.java 29ca30a exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableColumnReader.java 585fd66 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableFixedByteAlignedReader.java 17759d3 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/NullableFixedByteAlignedReaders.java 76cc937 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/PageReadStatus.java 3ad1d6c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFixedWidthDictionaryReader.java c0720a9 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java 0189c9b exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRecordReader.java 703ad1f exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRecordWriter.java b26f688 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetScanBatchCreator.java df6581f exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/VarLenBinaryReader.java 813a799 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/VarLengthColumnReaders.java 56f687c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/BitReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableBitReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableVarLengthValuesColumn.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/PageReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetFixedWidthDictionaryReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetToDrillTypeConverter.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumn.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthValuesColumn.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedFixedWidthVector.java 9b0a6cd exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedVariableWidthVector.java bd03038 exec/java-exec/src/main/java/org/apache/drill/exec/vector/VariableWidthVector.java 2b07750 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java ef8aef8 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java f05ab1b exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriter.java aa2b66f exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetRecordReaderTest.java 3e679bb exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java 4a0efc9 exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java 2d2a2ec exec/java-exec/src/test/resources/parquet/alltypes_repeated.json PRE-CREATION exec/java-exec/src/test/resources/parquet/basic_repeated.json PRE-CREATION exec/java-exec/src/test/resources/parquet/null_test_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/parquet_repeated_performance_test_input_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/repeated_bool_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/repeated_double_data.json PRE-CREATION exec/java-exec/src/test/resources/parquet/repeated_integer_data.json PRE-CREATION Diff: https://reviews.apache.org/r/23681/diff/ Testing ------- smoke tests, unit tests Thanks, Jason Altekruse
