----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29723/ -----------------------------------------------------------
(Updated Jan. 22, 2015, 9:40 p.m.) Review request for drill, Hanifi Gunes and Jacques Nadeau. Repository: drill-git Description ------- When reaching the end of a buffer, the current way drill handles this is for setSafe() to return false, and then whatever operator is doing the write will send the batch, and the redo the last record. This creates a lot of difficulty, because it sometimes requires being able to "rewind" the input stream to replay the last record. The proposal is to move the handling of buffer sizing and allocation into the value vectors themselves, making it transparent to the user of the value vector. The operators will now no longer have to worry about the possibility that writing into a vector may fail due to lack of space. Diffs (updated) ----- contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java 33bf376 contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java 82e038c contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java 625a7b2 contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveTextRecordReader.java 5406048 contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java 79abe60 exec/java-exec/src/main/codegen/templates/ComplexReaders.java 027f61d exec/java-exec/src/main/codegen/templates/ComplexWriters.java 5ba1c64 exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 58e6ccc exec/java-exec/src/main/codegen/templates/MapWriters.java b8bd73e exec/java-exec/src/main/codegen/templates/NullableValueVectors.java b222024 exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java d261050 exec/java-exec/src/main/codegen/templates/TypeHelper.java c83c301 exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java b8ffe5d exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java 5cf4a35 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillAggFuncHolder.java 6e0b282 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillComplexWriterFuncHolder.java df56174 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java 35faf22 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java 2fd5ce1 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggregator.java 0f7f394 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java ef85a36 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggTemplate.java 556b260 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/ChainedHashTable.java b5cfdca exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java 1ec74bf exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableTemplate.java ba980d7 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java e82dd29 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java f370dc7 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinProbe.java 7599f9e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinProbeTemplate.java c58f9a3 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/JoinTemplate.java c1dffc1 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java d0f9d7d exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingReceiverGeneratorBase.java 2885c52 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingReceiverTemplate.java c29ef75 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java acbb755 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionProjectorTemplate.java f5068b4 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java 20f6195 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java 539d028 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectorTemplate.java 49ad390 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/CopierTemplate2.java 5cc308a exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/CopierTemplate4.java c42332d exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameRecordBatch.java a3e7940 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameTemplate.java b4e3fed exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java c57ec28 exec/java-exec/src/main/java/org/apache/drill/exec/record/TransferPair.java 9645be9 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java 0c4437a exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableBitReader.java b9b808b exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 83f9bde exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthValuesColumn.java 7c4b33b exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoRecordReader.java 241fa95 exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoWriter.java 0ffa55c exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/Writers.java fee011a exec/java-exec/src/main/java/org/apache/drill/exec/store/text/DrillTextRecordReader.java 495e3e2 exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java 51726a3 exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java d2523c5 exec/java-exec/src/main/java/org/apache/drill/exec/vector/CopyUtil.java 1ed7f37 exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedFixedWidthVector.java aadc563 exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedMutator.java ad2ba1b exec/java-exec/src/main/java/org/apache/drill/exec/vector/VariableWidthVector.java ff3ee63 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java 62a5140 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java 58eb546 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java 43a3881 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/RepeatedListReaderImpl.java ae2f779 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/RepeatedMapReaderImpl.java 3171d8a exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/SingleMapReaderImpl.java 76f9e2f exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/window/TestWindowFrame.java 7c04477 exec/java-exec/src/test/java/org/apache/drill/exec/vector/TestAdaptiveAllocation.java ebc4df7 Diff: https://reviews.apache.org/r/29723/diff/ Testing ------- Thanks, Steven Phillips