[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184483395 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -536,6 +556,11 @@ public ColumnSize getColumn(String name

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-25 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184258865 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -536,6 +556,11 @@ public ColumnSize getColumn(String name

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-25 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184236170 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -277,18 +286,29 @@ public boolean isRepeatedList

[GitHub] drill issue #1228: DRILL-6307: Handle empty batches in record batch sizer co...

2018-04-25 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1228 @paul-rogers Paul, I addressed code review comments. Can you take a look when you get a chance ? ---

[GitHub] drill issue #1227: DRILL-6236: batch sizing for hash join

2018-04-25 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1227 @Ben-Zvi I manually added the PR link to the JIRA. all code review comments are addressed. can you look at the latest changes ? ---

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-25 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184202508 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -536,6 +556,11 @@ public ColumnSize getColumn(String name

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-25 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184200281 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -277,18 +286,29 @@ public boolean isRepeatedList

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-25 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184192443 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -50,7 +50,7 @@ public class RecordBatchSizer

[GitHub] drill issue #1218: DRILL-6335: Refactor row set abstractions to prepare for ...

2018-04-23 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1218 LGTM. +1 ---

[GitHub] drill issue #1227: Drill-6236: batch sizing for hash join

2018-04-20 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1227 @Ben-Zvi my bad. I updated the title. but, it has not updated the JIRA. trying to figure this out. ---

[GitHub] drill pull request #1227: Drill 6236: batch sizing for hash join

2018-04-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1227#discussion_r183171726 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java --- @@ -560,6 +554,40 @@ public void close

[GitHub] drill issue #1227: Drill 6236: batch sizing for hash join

2018-04-20 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1227 @sohami thanks for the review. updated with review comments addressed. please take a look. ---

[GitHub] drill pull request #1227: Drill 6236: batch sizing for hash join

2018-04-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1227#discussion_r183112258 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java --- @@ -147,7 +150,19 @@ NUM_BUCKETS

[GitHub] drill issue #1227: Drill 6236: batch sizing for hash join

2018-04-20 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1227 @Ben-Zvi Thanks a lot for the review. updated PR with review comments taken care of. Please take a look. Regarding spill files, here are my thoughts. For build side, I am using aggregate

[GitHub] drill pull request #1227: Drill 6236: batch sizing for hash join

2018-04-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1227#discussion_r183108078 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java --- @@ -300,13 +322,14 @@ public void setupHashTable

[GitHub] drill pull request #1218: DRILL-6335: Refactor row set abstractions to prepa...

2018-04-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1218#discussion_r18296 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetWriterImpl.java --- @@ -158,4 +159,10 @@ public SingleRowSet done

[GitHub] drill pull request #1218: DRILL-6335: Refactor row set abstractions to prepa...

2018-04-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1218#discussion_r182921553 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/impl/RowSetLoaderImpl.java --- @@ -95,4 +96,10 @@ public void endBatch

[GitHub] drill pull request #1218: DRILL-6335: Refactor row set abstractions to prepa...

2018-04-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1218#discussion_r182729279 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/test/TestHyperVectorReaders.java --- @@ -362,4 +363,136 @@ public void testRepeatedMap

[GitHub] drill pull request #1218: DRILL-6335: Refactor row set abstractions to prepa...

2018-04-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1218#discussion_r182923416 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/test/TestIndirectReaders.java --- @@ -0,0 +1,151 @@ +/* + * Licensed to the

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-19 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1228 DRILL-6307: Handle empty batches in record batch sizer correctly When we get empty batch, record batch sizer calculates row width as zero. In that case, we do not do accounting and memory allocation

[GitHub] drill pull request #1227: Drill 6236: batch sizing for hash join

2018-04-19 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1227 Drill 6236: batch sizing for hash join You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppadma/drill DRILL-6236 Alternatively you can review and

[GitHub] drill issue #1161: DRILL-6230: Extend row set readers to handle hyper vector...

2018-03-31 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1161 @paul-rogers ran the tests. they all pass. ---

[GitHub] drill issue #1181: DRILL-6284: Add operator metrics for batch sizing for fla...

2018-03-31 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1181 @paul-rogers thanks a lot for the review. updated PR with review comments taken care of. ---

[GitHub] drill issue #1181: DRILL-6284: Add operator metrics for batch sizing for fla...

2018-03-29 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1181 @paul-rogers I added metrics for merge join also. I refactored AbstractRecordBatchMemoryManager to handle batches from multiple streams. Please review when you get a chance. ---

[GitHub] drill issue #1181: DRILL-6284: Add operator metrics for batch sizing for fla...

2018-03-28 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1181 @paul-rogers thanks for the review. Please take a look at updated changes. ---

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-28 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177810092 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-28 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r177564887 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/model/ReaderIndex.java --- @@ -28,26 +28,30 @@ public abstract class

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-28 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r177182331 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/test/TestHyperVectorReaders.java --- @@ -0,0 +1,365 @@ +/* + * Licensed to the

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-28 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r177175974 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/HyperRowSetImpl.java --- @@ -45,8 +50,67 @@ public RowSetReader buildReader(HyperRowSet

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-21 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1181 DRILL-6284: Add operator metrics for batch sizing for flatten @kkhatua please review. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppadma

[GitHub] drill pull request #1179: DRILL-6254: IllegalArgumentException: the requeste...

2018-03-21 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1179 DRILL-6254: IllegalArgumentException: the requested size must be non-… …negative We should limit memory allocation to number of records that are going to be in the next batch, not the

[GitHub] drill issue #1175: DRILL-6262: IndexOutOfBoundException in RecordBatchSize f...

2018-03-19 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1175 LGTM. +1. ---

[GitHub] drill issue #1171: DRILL-6231: Fix memory allocation for repeated list vecto...

2018-03-18 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1171 @paul-rogers I added the null check. Also, added a test case for 3D array i.e. repeated repeated list. ---

[GitHub] drill issue #1171: DRILL-6231: Fix memory allocation for repeated list vecto...

2018-03-15 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1171 @paul-rogers please review. ---

[GitHub] drill issue #1161: DRILL-6230: Extend row set readers to handle hyper vector...

2018-03-15 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1161 @paul-rogers I started the review. will get back soon. ---

[GitHub] drill pull request #1171: DRILL-6231: Fix memory allocation for repeated lis...

2018-03-15 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1171 DRILL-6231: Fix memory allocation for repeated list vector You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppadma/drill DRILL-6231

[GitHub] drill issue #1150: DRILL-6210: Enhanced test schema utilities

2018-03-06 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1150 @Ben-Zvi Boaz, can you do a committer review for this PR and include it in this week's batch commit ? ---

[GitHub] drill issue #1125: DRILL-6126: Allocate memory for value vectors upfront in ...

2018-03-06 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1125 @paul-rogers Thank you Paul. I will be incorporating some of your suggestions in future PRs. This change will allow us to build more things on top. ---

[GitHub] drill issue #1150: DRILL-6210: Enhanced test schema utilities

2018-03-05 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1150 This is very useful for testing complex types. LGTM. +1. ---

[GitHub] drill issue #1125: DRILL-6126: Allocate memory for value vectors upfront in ...

2018-03-03 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1125 @paul-rogers Updated the PR with latest changes. I have decided not to use vector initializer for allocation as it is subject to alias issues like you mentioned. Instead, I added allocate vector

[GitHub] drill pull request #1125: DRILL-6126: Allocate memory for value vectors upfr...

2018-03-02 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1125#discussion_r171999276 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -245,16 +251,30 @@ private void buildVectorInitializer

[GitHub] drill pull request #1125: DRILL-6126: Allocate memory for value vectors upfr...

2018-03-02 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1125#discussion_r171999148 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -418,11 +438,13 @@ private void measureColumn(ValueVector v

[GitHub] drill pull request #1147: DRILL-6205: Reduce memory consumption of testFlatt...

2018-03-02 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1147 DRILL-6205: Reduce memory consumption of testFlattenUpperLimit test You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppadma/drill DRILL-6205

[GitHub] drill issue #1129: DRILL-6180: Use System Option "output_batch_size" for Ext...

2018-02-28 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1129 @paul-rogers Made the change you suggested. Please take a look when you get a chance. ---

[GitHub] drill issue #1125: DRILL-6126: Allocate memory for value vectors upfront in ...

2018-02-28 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1125 @paul-rogers Paul, Thanks a lot for your review comments and bringing up some good issues. Just want to let you know. I am working on refactoring the batch sizer code, writing bunch of unit tests to

[GitHub] drill pull request #1129: DRILL-6180: Use System Option "output_batch_size" ...

2018-02-22 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1129 DRILL-6180: Use System Option "output_batch_size" for External Sort External Sort has boot time configuration for output batch size "drill.exec.sort.external.spill.merge_batch

[GitHub] drill issue #1125: DRILL-6126: Allocate memory for value vectors upfront in ...

2018-02-21 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1125 @paul-rogers Paul, will you be able to review this ? ---

[GitHub] drill pull request #1125: DRILL-6126: Allocate memory for value vectors upfr...

2018-02-21 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1125 DRILL-6126: Allocate memory for value vectors upfront in flatten operator Made changes to allocate memory upfront for flatten operator based on sizing calculations. Need to do allocation of

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r169155001 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,222 @@ +/* + * Licensed to

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r169159418 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,222 @@ +/* + * Licensed to

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r169155210 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,222 @@ +/* + * Licensed to

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-19 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r169162527 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/VectorContainerAccessor.java --- @@ -0,0 +1,132

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r168324742 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,183 @@ +/* + * Licensed to

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r168331779 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,183 @@ +/* + * Licensed to

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r168337843 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,183 @@ +/* + * Licensed to

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r168325261 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorExec.java --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r168325765 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorRecordBatch.java --- @@ -0,0 +1,167 @@ +/* + * Licensed

[GitHub] drill pull request #1121: DRILL-6153: Operator framework

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1121#discussion_r168317696 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/protocol/OperatorDriver.java --- @@ -0,0 +1,183 @@ +/* + * Licensed to

[GitHub] drill pull request #1112: DRILL-6114: Metadata revisions

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1112#discussion_r168289206 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/metadata/AbstractColumnMetadata.java --- @@ -0,0 +1,206 @@ +/* + * Licensed to

[GitHub] drill pull request #1112: DRILL-6114: Metadata revisions

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1112#discussion_r168291276 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/metadata/MetadataUtils.java --- @@ -0,0 +1,165 @@ +/* + * Licensed to the Apache

[GitHub] drill pull request #1112: DRILL-6114: Metadata revisions

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1112#discussion_r168301865 --- Diff: exec/vector/src/main/java/org/apache/drill/exec/record/metadata/ColumnMetadata.java --- @@ -15,36 +15,115 @@ * See the License for the

[GitHub] drill pull request #1112: DRILL-6114: Metadata revisions

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1112#discussion_r168295700 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/metadata/RepeatedListColumnMetadata.java --- @@ -0,0 +1,94 @@ +/* + * Licensed

[GitHub] drill pull request #1112: DRILL-6114: Metadata revisions

2018-02-14 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1112#discussion_r168288500 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/metadata/AbstractColumnMetadata.java --- @@ -0,0 +1,206 @@ +/* + * Licensed to

[GitHub] drill issue #1107: DRILL-6123: Limit batch size for Merge Join based on memo...

2018-02-09 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1107 @paul-rogers Thank you very much for the review. Updated the PR with review comments taken care of. Please take a look. ---

[GitHub] drill pull request #1107: DRILL-6123: Limit batch size for Merge Join based ...

2018-02-09 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1107#discussion_r167380917 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatchMemoryManager.java --- @@ -0,0 +1,63 @@ +/* + * Licensed to

[GitHub] drill pull request #1107: DRILL-6123: Limit batch size for Merge Join based ...

2018-02-09 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1107#discussion_r167380761 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/spill/RecordBatchSizer.java --- @@ -311,8 +311,8 @@ public static ColumnSize

[GitHub] drill pull request #1107: DRILL-6123: Limit batch size for Merge Join based ...

2018-02-09 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1107#discussion_r167380394 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/spill/RecordBatchSizer.java --- @@ -137,10 +137,10 @@ public ColumnSize

[GitHub] drill pull request #1115: DRILL-6138: Move RecordBatchSizer to org.apache.dr...

2018-02-06 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1115 DRILL-6138: Move RecordBatchSizer to org.apache.drill.exec.record pac… …kage Also, changed columnSizes in RecordBatchSizer from list to map so we can lookup using field names. @Ben

[GitHub] drill pull request #1107: DRILL-6123: Limit batch size for Merge Join based ...

2018-02-06 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1107#discussion_r166427892 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java --- @@ -102,20 +105,78 @@ private final List

[GitHub] drill issue #1112: DRILL-6114: Metadata revisions

2018-02-06 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1112 @paul-rogers I ran the pre commit tests. No issues. Everything passed. Will do one more time once code reviews are done. ---

[GitHub] drill issue #1107: DRILL-6123: Limit batch size for Merge Join based on memo...

2018-02-05 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1107 @sachouche @ilooner @paul-rogers Can one of you review this PR for me ? ---

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-02-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r166142178 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/spill/RecordBatchSizer.java --- @@ -65,6 +70,14 @@ public int

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-02-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r166098364 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -140,6 +131,9 @@ private

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-02-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r166141274 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -733,28 +780,32 @@ private void

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-02-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r166096630 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -215,6 +206,7 @@ public BatchHolder

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-02-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r166142507 --- Diff: exec/vector/src/main/codegen/templates/FixedValueVectors.java --- @@ -298,6 +298,11 @@ public int getPayloadByteCount(int valueCount

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-02-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r166136279 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -226,7 +221,7 @@ public BatchHolder

[GitHub] drill pull request #1107: DRILL-6123: Limit batch size for Merge Join based ...

2018-02-01 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1107 DRILL-6123: Limit batch size for Merge Join based on memory Merge join limits output batch size to 32K rows irrespective of row size. This can create large batches (in terms of memory), depending

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-01-31 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r165156589 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -215,6 +206,7 @@ public BatchHolder

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-01-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r164634551 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -215,6 +206,7 @@ public BatchHolder

[GitHub] drill pull request #1101: DRILL-6032: Made the batch sizing for HashAgg more...

2018-01-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1101#discussion_r164599681 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -215,6 +206,7 @@ public BatchHolder

[GitHub] drill pull request #1091: DRILL-6071: Limit batch size for flatten operator

2018-01-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1091#discussion_r164231963 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -94,8 +98,57 @@ private void clear

[GitHub] drill pull request #1091: DRILL-6071: Limit batch size for flatten operator

2018-01-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1091#discussion_r164233329 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/unit/TestOutputBatchSize.java --- @@ -0,0 +1,498 @@ +/* + * Licensed to the

[GitHub] drill issue #1091: DRILL-6071: Limit batch size for flatten operator

2018-01-26 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1091 @paul-rogers Thank you Paul. Ready for review. Please take a look when you get a chance. ---

[GitHub] drill issue #1091: DRILL-6071: Limit batch size for flatten operator

2018-01-19 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1091 @paul-rogers Addressed code review comments. Added a new system/session option to configure output batch size. Please review. ---

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161900034 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161895458 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161906070 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161899357 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161905865 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161895158 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2018-01-16 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r161899883 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,286 @@ import

[GitHub] drill issue #1091: DRILL-6071: Limit batch size for flatten operator

2018-01-16 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1091 @paul-rogers Paul, can you please review this PR ? ---

[GitHub] drill pull request #1091: DRILL-6071: Limit batch size for flatten operator

2018-01-14 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/1091 DRILL-6071: Limit batch size for flatten operator Please see DRILL-6071 for details. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppadma/drill

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2017-12-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r158033645 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,283 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2017-12-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r158029512 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,283 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2017-12-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r158025793 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,283 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2017-12-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r158032627 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,283 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2017-12-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r158034838 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,283 @@ import

[GitHub] drill pull request #1072: DRILL-5879: Improved SQL Pattern Contains Performa...

2017-12-20 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1072#discussion_r158030478 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SqlPatternContainsMatcher.java --- @@ -19,44 +19,283 @@ import

  1   2   3   4   >