----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29723/#review68625 -----------------------------------------------------------
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableTemplate.java <https://reviews.apache.org/r/29723/#comment112951> Same here, do we need this to ensure the hash is positive? exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java <https://reviews.apache.org/r/29723/#comment112954> Do we really want this to default this high? I'd be concerned about wide varlen values, will the automatic reallocation just keep increasing the buffer size to try to allow this many record go through? exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java <https://reviews.apache.org/r/29723/#comment112957> While this refactoring is happening it might be worth taking this out of the project operator and trying to look for other places with similar requirements to consolidate the code. I think it would be especially useful here because the code currently uses instanceof and we will likely want to add an interface to the vector eventually for "isFixedWidth". exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameRecordBatch.java <https://reviews.apache.org/r/29723/#comment112959> remove exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java <https://reviews.apache.org/r/29723/#comment112963> I think I marked them where they appeared, but there are places inthe patch that have this code duplicated. exec/java-exec/src/main/java/org/apache/drill/exec/vector/CopyUtil.java <https://reviews.apache.org/r/29723/#comment112964> As in the other cases, this should be encapsualted to allow later refactoring. - Jason Altekruse On Jan. 8, 2015, 8:35 p.m., Steven Phillips wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29723/ > ----------------------------------------------------------- > > (Updated Jan. 8, 2015, 8:35 p.m.) > > > Review request for drill and Jacques Nadeau. > > > Repository: drill-git > > > Description > ------- > > When reaching the end of a buffer, the current way drill handles this is for > setSafe() to return false, and then whatever operator is doing the write will > send the batch, and the redo the last record. This creates a lot of > difficulty, because it sometimes requires being able to "rewind" the input > stream to replay the last record. > The proposal is to move the handling of buffer sizing and allocation into the > value vectors themselves, making it transparent to the user of the value > vector. The operators will now no longer have to worry about the possibility > that writing into a vector may fail due to lack of space. > > > Diffs > ----- > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java > 33bf376 > > contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java > 82e038c > > contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java > 625a7b2 > > contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveTextRecordReader.java > 5406048 > > contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java > 79abe60 > exec/java-exec/src/main/codegen/templates/ComplexReaders.java 027f61d > exec/java-exec/src/main/codegen/templates/ComplexWriters.java 5ba1c64 > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 58e6ccc > exec/java-exec/src/main/codegen/templates/MapWriters.java b8bd73e > exec/java-exec/src/main/codegen/templates/NullableValueVectors.java b222024 > exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java d261050 > exec/java-exec/src/main/codegen/templates/TypeHelper.java c83c301 > exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java > b8ffe5d > > exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java > 5cf4a35 > > exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillAggFuncHolder.java > 6e0b282 > > exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillComplexWriterFuncHolder.java > df56174 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java > 113e883 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java > d7cf904 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java > ef85a36 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggTemplate.java > 556b260 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/ChainedHashTable.java > 0502f7e > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java > e8ccd62 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableTemplate.java > 5b56f8e > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java > e82dd29 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java > 5deb67f > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinProbe.java > 7599f9e > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinProbeTemplate.java > c58f9a3 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/JoinTemplate.java > c1dffc1 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java > d0f9d7d > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingReceiverGeneratorBase.java > 2885c52 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingReceiverTemplate.java > c29ef75 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java > acbb755 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionProjectorTemplate.java > f5068b4 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java > 20f6195 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java > fa983aa > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectorTemplate.java > 49ad390 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/CopierTemplate2.java > 5cc308a > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/CopierTemplate4.java > c42332d > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameRecordBatch.java > a3e7940 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameTemplate.java > b4e3fed > exec/java-exec/src/main/java/org/apache/drill/exec/record/TransferPair.java > 9645be9 > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java > 0c4437a > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableBitReader.java > b9b808b > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java > 83f9bde > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthValuesColumn.java > 7c4b33b > > exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoRecordReader.java > 241fa95 > > exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoWriter.java > 0ffa55c > exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/Writers.java > fee011a > > exec/java-exec/src/main/java/org/apache/drill/exec/store/text/DrillTextRecordReader.java > e0cce8b > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java > 51726a3 > exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java > d2523c5 > exec/java-exec/src/main/java/org/apache/drill/exec/vector/CopyUtil.java > 1ed7f37 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedFixedWidthVector.java > aadc563 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedMutator.java > ad2ba1b > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/VariableWidthVector.java > ff3ee63 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java > cc3d24c > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java > 362d806 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java > e140c8b > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/RepeatedListReaderImpl.java > c60730c > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/RepeatedMapReaderImpl.java > 15f8a2b > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/SingleMapReaderImpl.java > 3ec66ff > > exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/window/TestWindowFrame.java > 7c04477 > > exec/java-exec/src/test/java/org/apache/drill/exec/vector/TestAdaptiveAllocation.java > ebc4df7 > > Diff: https://reviews.apache.org/r/29723/diff/ > > > Testing > ------- > > > Thanks, > > Steven Phillips > >
