[GitHub] drill pull request #1213: DRILL-6334: Minor code cleanup

2018-04-19 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1213#discussion_r182949848 --- Diff: exec/vector/src/test/java/org/apache/drill/exec/vector/VariableLengthVectorTest.java --- @@ -37,6 +37,7 @@ public void

[GitHub] drill pull request #1206: DRILL_6314: Add complex types to result set loader

2018-04-17 Thread paul-rogers
Github user paul-rogers closed the pull request at: https://github.com/apache/drill/pull/1206 ---

[GitHub] drill issue #1218: DRILL-6335: Refactor row set abstractions to prepare for ...

2018-04-17 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1218 @ppadma , this PR contains a subset of the changes from the previous big PR. Mostly refactoring. Can you take a look? Thanks. ---

[GitHub] drill issue #1206: DRILL_6314: Add complex types to result set loader

2018-04-17 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1206 Replacing this PR with a set of smaller ones. ---

[GitHub] drill pull request #1218: DRILL-6335: Refactor row set abstractions to prepa...

2018-04-17 Thread paul-rogers
GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/1218 DRILL-6335: Refactor row set abstractions to prepare for unions Refactors the column accessors to prepare for adding unions, lists and repeated lists. This is a subset of a PR done a

[GitHub] drill issue #1206: DRILL_6314: Add complex types to result set loader

2018-04-16 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1206 Going to close this one and try to split this into smaller chunks. ---

[jira] [Created] (DRILL-6335) Refactor row set abstractions to prepare for unions

2018-04-16 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6335: -- Summary: Refactor row set abstractions to prepare for unions Key: DRILL-6335 URL: https://issues.apache.org/jira/browse/DRILL-6335 Project: Apache Drill Issue

[GitHub] drill pull request #1213: Minor code cleanup

2018-04-16 Thread paul-rogers
GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/1213 Minor code cleanup Pulled the remaining code cleanup items out of the Result Set work into this simple PR. You can merge this pull request into a Git repository by running: $ git pull

[jira] [Created] (DRILL-6334) Code cleanup

2018-04-16 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6334: -- Summary: Code cleanup Key: DRILL-6334 URL: https://issues.apache.org/jira/browse/DRILL-6334 Project: Apache Drill Issue Type: Improvement Reporter

[GitHub] drill pull request #1210: DRILL-6270: Add debug startup option flag for dril...

2018-04-16 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1210#discussion_r181799954 --- Diff: distribution/src/resources/runbit --- @@ -65,6 +65,47 @@ drill_rotate_log () fi } +args=( $@ ) +RBARGS=() +for

Re: [DISCUSS] Regarding mutator interface

2018-04-16 Thread Paul Rogers
ache/drill/blob/master/exec/java- > exec/src/main/java/org/apache/drill/exec/expr/fn/impl/Mappify.java#L55 > [3] https://github.com/apache/drill/blob/master/exec/java- > exec/src/main/java/org/apache/drill/exec/expr/fn/ValueReference.java#L76 > > -Aman > > On Fri, Apr 13, 201

Re: [DISCUSS] Regarding mutator interface

2018-04-13 Thread Paul Rogers
on. As a result, only fixed-width types can be updated in random order. DRILL-6087 describes this issue. Thanks, - Paul [1] https://github.com/paul-rogers/drill/wiki/UDFs-Background-Information On Wednesday, April 11, 2018, 4:09:47 PM PDT, Aman Sinha wrote: Here's so

Re: [DISCUSS] Regarding mutator interface

2018-04-11 Thread Paul Rogers
        } > >            vv9 .getMutator().setSafe((outIndex), out8); //Don't have setSafe >for MapVector > >        } > >    } > > > Please let me know your thoughts. > > > Gautam > > > > > From: Paul Rogers

Re: [DISCUSS] Regarding mutator interface

2018-04-11 Thread Paul Rogers
Note that, for maps and lists, there is nothing to set. Maps are purely containers for other vectors. Lists (you didn't mention whether "repeated" or "non-repeated") are also containers. Non-repeated lists are containers for unions, repeated-lists are containers for arrays. Any setting should be

[GitHub] drill issue #1206: DRILL_6314: Add complex types to result set loader

2018-04-10 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1206 @ppadma can you look at this one? It's a big one. Suggestion: work through small groups of files, perhaps one commit at a time as the layers are separated into separate commits. Maybe

[GitHub] drill pull request #1206: DRILL_6314: Add complex types to result set loader

2018-04-10 Thread paul-rogers
GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/1206 DRILL_6314: Add complex types to result set loader This PR is a bit of a large one as it adds Union, (non repeated) List and Repeated List type support to the column accessors, row set

Re: Unit Testing Docs and [Discuss] Dev Docs location

2018-04-10 Thread Paul Rogers
ood information that people may be missing out on since they aren't aware of your wiki pages. Thanks, Tim ________ From: Paul Rogers Sent: Monday, April 9, 2018 11:30:52 PM To: dev@drill.apache.org Subject: Re: Unit Testing Docs and [Discuss] Dev Docs location

Re: Unit Testing Docs and [Discuss] Dev Docs location

2018-04-09 Thread Paul Rogers
e that new mechanism. Thanks, - Paul [1] https://github.com/paul-rogers/drill/wiki/Testing-Tips On Monday, April 9, 2018, 8:02:48 PM PDT, Timothy Farkas wrote: Hi All, I am wrapping up a unit testing doc here https://github.com/ilooner/drill/blob/DRILL-6249/docs/DevDocs.md which

[jira] [Created] (DRILL-6314) Add complex types to row set abstraction

2018-04-08 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6314: -- Summary: Add complex types to row set abstraction Key: DRILL-6314 URL: https://issues.apache.org/jira/browse/DRILL-6314 Project: Apache Drill Issue Type

Re: "Death of Schema-on-Read"

2018-04-08 Thread Paul Rogers
Hi Ted, All good points. In the Jira ticket, I mentioned that using casts to infer type is a special case of a general problem: inferring column type information in a downward traversal of the parse tree, then refining that type information with a bottom-up traversal of the DAG at run time. Tur

Re: [DISCUSS] DrillBuf

2018-04-07 Thread Paul Rogers
2018, 9:35:06 PM PDT, Vlad Rozov wrote: Hi Paul, My comments in-line. Thank you, Vlad On 4/5/18 20:50, Paul Rogers wrote: > Hi Vlad, >>  I'd suggest to keep focus on DrillBuf design and implementation as the only >>gate for accessing raw (direct) memory. > I was

Re: Non-column filters in Drill

2018-04-07 Thread Paul Rogers
Hi Ryan, There is an obscure, but very handy feature of Drill called table functions. [1] These allow you to set parameters of your format plugin as part of a query. You mentioned a storage plugin. I've not tried a table function with a storage plugin. I have tested table functions with a forma

Re: "Death of Schema-on-Read"

2018-04-07 Thread Paul Rogers
Hi Hanu, Thanks! After sleeping on the idea, I realized that it can be generalized for any kind of expression. But, I also realized that the cast mechanism, by itself, cannot be a complete solution. Details posted in the JIRA for anyone who is interested. Thanks, - Paul On Saturday, Apri

[jira] [Created] (DRILL-6313) ScanBatch.Mutator does not report new schema for empty first batch

2018-04-07 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6313: -- Summary: ScanBatch.Mutator does not report new schema for empty first batch Key: DRILL-6313 URL: https://issues.apache.org/jira/browse/DRILL-6313 Project: Apache Drill

Re: "Death of Schema-on-Read"

2018-04-06 Thread Paul Rogers
Hi Aman, As we get into details, I suggested to Hanu that we move the discussion into a JIRA ticket. >On the subject of CAST pushdown to Scans, there are potential drawbacks  >  - In general, the planner will see a Scan-Project where the Project has  CAST functions.  But the Project can have

Re: "Death of Schema-on-Read"

2018-04-06 Thread Paul Rogers
values to other scalar values (like int to double etc). But we should be also able to give option to the customer to enable rules such as scalar int to array[int] when creating the view itself. Thanks, -Hanu On Fri, Apr 6, 2018 at 3:10 PM, Paul Rogers wrote: > Ted, this is why your participa

Re: "Death of Schema-on-Read"

2018-04-06 Thread Paul Rogers
Ted, this is why your participation in Drill is such a gift: cast push-down is an elegant, simple solution that even works in views. Beautiful. Thanks, - Paul On Friday, April 6, 2018, 11:35:37 AM PDT, Ted Dunning wrote: On Thu, Apr 5, 2018 at 9:43 PM, Paul Rogers wrote: > Gr

Re: "Death of Schema-on-Read"

2018-04-06 Thread Paul Rogers
is bad" can you please share your view point on what all design issues it can cause. I know that it can be bottleneck but just want to know about other issues. Put in other terms if centralized metastore engineered in a good way to avoid most of the bottleneck, then do you think it can

Re: "Death of Schema-on-Read"

2018-04-05 Thread Paul Rogers
Great discussion. Really appreciate the insight from the Drill users! To Ted's points: the simplest possible solution is to allow a table function to express types. Just making stuff up: SELECT a FROM schema(myTable, (a: INT)) Or, a SQL extension: SELECT a FROM myTable(a: INT) Or, really ugly

Re: "Death of Schema-on-Read"

2018-04-05 Thread Paul Rogers
ema change. > > > > > > > > > > On Tue, Apr 3, 2018 at 10:01 PM, Ted Dunning > > wrote: > > > > > Well, the restart strategy still works for your examples. And you only > > pay > > > once. From them you look at the cached type in

Re: [DISCUSS] DrillBuf

2018-04-05 Thread Paul Rogers
sabled, always on, always off, any other suggestions) 2. Ref count checking (delegate to netty or have a separate mechanism to enable/disable, always on or off) 3. Usage of UDLE 4. Changing DrillBuf to follow Netty convention 5. Moving DrillBuf to a different package Thank you, Vlad On 4/4/18 11

Re: [DISCUSS] DrillBuf

2018-04-04 Thread Paul Rogers
reeing upon the right design abstractions, we can then make informed decisions about how best to improve our low-level abstractions, including DrillBuf. Thanks, - Paul [1] http://drill.apache.org/docs/value-vectors/ [2] https://github.com/paul-rogers/drill/wiki/Batch-Handling-Upgrades O

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179026313 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179027582 --- Diff: distribution/src/resources/drillbit.sh --- @@ -154,6 +192,7 @@ start_bit ( ) nohup nice -n $DRILL_NICENESS "$DRILL_HOME/bin/runbit&

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179025617 --- Diff: distribution/src/resources/drill-env.sh --- @@ -86,6 +86,12 @@ #export DRILL_PID_DIR=${DRILL_PID_DIR:-$DRILL_HOME

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028127 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028625 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179027761 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179026462 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179025777 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028377 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028078 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179025400 --- Diff: distribution/src/resources/drill-env.sh --- @@ -86,6 +86,12 @@ #export DRILL_PID_DIR=${DRILL_PID_DIR:-$DRILL_HOME

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179027910 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start

Re: "Death of Schema-on-Read"

2018-04-03 Thread Paul Rogers
n. Views are another way to communicate schema information. It is true that you can't, say, view comments on fields from the command line. But I don't understand saying "schema-forbidden". On Tue, Apr 3, 2018 at 10:01 AM, Paul Rogers wrote: > Here is another way to think

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179023466 --- Diff: src/main/resources/checkstyle-config.xml --- @@ -30,10 +30,15 @@ + --- End diff -- Do we want to do

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179022090 --- Diff: exec/memory/base/src/main/java/io/netty/buffer/DrillBuf.java --- @@ -777,23 +778,20 @@ public int getActualMemoryConsumed() { * @return

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179022533 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -534,15 +534,11 @@ public void setSafe(int index, byte[] bytes

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179023079 --- Diff: exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/writer/BaseScalarWriter.java --- @@ -211,7 +211,7 @@ protected boolean

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179023675 --- Diff: src/main/resources/checkstyle-suppressions.xml --- @@ -16,4 +16,13 @@ +

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179022338 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -534,15 +534,11 @@ public void setSafe(int index, byte[] bytes

[GitHub] drill issue #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsException ...

2018-04-03 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1144 My two cents... DrillBuf is the only memory-level abstraction that (low level) Drill code should reference. The UDLE and other bits should be fully encapsulated. This guideline lets us evolve

Re: "Death of Schema-on-Read"

2018-04-03 Thread Paul Rogers
that they use your software for in the first place. The goal of schema on read is to facilitate the task of interpreting the data that already exists, is mutating, and is undocumented (or documented badly). [1] https://en.wikipedia.org/wiki/Begging_the_question On Mon, Apr 2, 2018 at 11:16 AM, Paul

Re: "Death of Schema-on-Read"

2018-04-02 Thread Paul Rogers
ted set of rules that do type conversions. [1] https://www.marklogic.com/blog/schema-on-read-vs-schema-on-write/ -Aman On Sun, Apr 1, 2018 at 10:46 PM, Paul Rogers wrote: > ...is the name of a provocative blog post [1]. > Quote: "Once found, diverse data sets are very hard to integra

"Death of Schema-on-Read"

2018-04-01 Thread Paul Rogers
...is the name of a provocative blog post [1]. Quote: "Once found, diverse data sets are very hard to integrate, since the data typically contains no documentation on the semantics of its attributes. ... The rule of thumb is that data scientists spend 70% of their time finding, interpreting, and

[GitHub] drill issue #1161: DRILL-6230: Extend row set readers to handle hyper vector...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1161 Rebased on latest master. Ran Maven unit tests; they passed up to Mongo. @ppadma, please run against the pre-commit tests @parthchandra, can you give this one a committer review

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r178445472 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/test/TestHyperVectorReaders.java --- @@ -0,0 +1,365 @@ +/* + * Licensed to

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r178445424 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/HyperRowSetImpl.java --- @@ -45,8 +50,67 @@ public RowSetReader buildReader

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r177833383 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/model/ReaderIndex.java --- @@ -28,26 +28,30 @@ public abstract

[jira] [Created] (DRILL-6305) Add unit tests for repeated list vectors to hyper row set reader code

2018-03-31 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6305: -- Summary: Add unit tests for repeated list vectors to hyper row set reader code Key: DRILL-6305 URL: https://issues.apache.org/jira/browse/DRILL-6305 Project: Apache

[GitHub] drill pull request #1161: DRILL-6230: Extend row set readers to handle hyper...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1161#discussion_r177846274 --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/rowSet/HyperRowSetImpl.java --- @@ -45,8 +50,67 @@ public RowSetReader buildReader

[jira] [Created] (DRILL-6304) In hyper-batch row set reader, handle disjoint unions

2018-03-31 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6304: -- Summary: In hyper-batch row set reader, handle disjoint unions Key: DRILL-6304 URL: https://issues.apache.org/jira/browse/DRILL-6304 Project: Apache Drill Issue

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445213 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -463,4 +488,20 @@ protected boolean

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445205 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,7 +100,23 @@ private void clear

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445267 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java --- @@ -573,4 +610,21 @@ private LogicalExpression

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445230 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java --- @@ -109,12 +110,37 @@ private static

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445224 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -463,4 +489,21 @@ protected boolean

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445244 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java --- @@ -127,17 +153,20 @@ @Override

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-31 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r178445262 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java --- @@ -153,16 +182,22 @@ public void update(int

[GitHub] drill issue #1170: DRILL-6223: Fixed several Drillbit failures due to schema...

2018-03-30 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1170 Over the last year, we've tended to favor including unit tests with each PR. There don't seem to be any with this one, yet we are proposing to make a fairly complex change. Perhaps te

[GitHub] drill issue #1170: DRILL-6223: Fixed several Drillbit failures due to schema...

2018-03-30 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1170 BTW: thanks for tackling such a difficult, core issue in Drill. Drill claims to be a) schema free and b) SQL compliant. SQL is based on operations over relations with a fixed number of columns

[GitHub] drill pull request #1170: DRILL-6223: Fixed several Drillbit failures due to...

2018-03-29 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1170#discussion_r178222960 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java --- @@ -60,13 +60,7 @@ public LimitRecordBatch

[GitHub] drill pull request #1170: DRILL-6223: Fixed several Drillbit failures due to...

2018-03-29 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1170#discussion_r178225930 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java --- @@ -136,14 +138,28 @@ public void transferOut

[GitHub] drill pull request #1170: DRILL-6223: Fixed several Drillbit failures due to...

2018-03-29 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1170#discussion_r178225725 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java --- @@ -136,14 +138,28 @@ public void transferOut

[GitHub] drill issue #1170: DRILL-6223: Fixed several Drillbit failures due to schema...

2018-03-29 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1170 Sorry to say, I still disagree with this statement: "This pull request adds logic to detect and eliminate dangling columns". There was a prior discussion that `SELECT *` mea

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-28 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177825111 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill issue #1161: DRILL-6230: Extend row set readers to handle hyper vector...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1161 @ppadma , any partial comments that I can start to take a look at? There are still a number of PRs in this chain and it would be great if we could keep things ticking along... Thanks! ---

[GitHub] drill pull request #1179: DRILL-6254: IllegalArgumentException: the requeste...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1179#discussion_r177619465 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -237,7 +237,10 @@ protected

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617185 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617922 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatchMemoryManager.java --- @@ -29,6 +29,50 @@ private int

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617262 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617387 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -463,4 +488,20 @@ protected boolean

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617131 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617860 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatchMemoryManager.java --- @@ -29,6 +29,50 @@ private int

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177616893 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill pull request #1181: DRILL-6284: Add operator metrics for batch sizing ...

2018-03-27 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1181#discussion_r177617047 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/flatten/FlattenRecordBatch.java --- @@ -99,6 +100,22 @@ private void clear

[GitHub] drill issue #1175: DRILL-6262: IndexOutOfBoundException in RecordBatchSize f...

2018-03-19 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1175 Thanks. See how confusing it is? I wrote the darn thing originally and even I can't keep the names straight... :-) ---

[GitHub] drill pull request #1175: DRILL-6262: IndexOutOfBoundException in RecordBatc...

2018-03-19 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1175#discussion_r175619764 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -321,10 +321,8 @@ public ColumnSize(ValueVector v

Re: [VOTE] Apache Drill release 1.13.0 - RC0

2018-03-18 Thread Paul Rogers
be we should extend the voting period for the next release so as to allow voters more time. On Sun, Mar 18, 2018 at 5:21 AM, Paul Rogers wrote: > Downloaded the tar.gz files. Followed the USAGE.md [1] instructions for > running Drill-on-YARN. > * Launched Drill using DoY on a single

Re: Interesting in contributing to Drill

2018-03-17 Thread Paul Rogers
notes to help future travelers. The GitHub wiki is free and pretty handy for this task. Thanks, - Paul [1] https://github.com/paul-rogers/drill/wiki On Tuesday, March 6, 2018, 12:00:56 PM PST, Kunal Khatua wrote: The code base is huge so many contributions that I doubt anyone know

[GitHub] drill issue #1170: DRILL-6223: Fixed several Drillbit failures due to schema...

2018-03-17 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1170 The waters here run deep. Please see a detailed comment in [DRILL-6223](https://issues.apache.org/jira/browse/DRILL-6223). ---

Re: [VOTE] Apache Drill release 1.13.0 - RC0

2018-03-17 Thread Paul Rogers
Downloaded the tar.gz files. Followed the USAGE.md [1] instructions for running Drill-on-YARN. * Launched Drill using DoY on a single node YARN/HDFS cluster on my Mac.* Played with the DoY AM UI, which looks good.* Ran a few queries on the Drillbit which DoY launched. Also looks good. Found a nu

[jira] [Created] (DRILL-6269) Sort properties in the DoY UI Configuration tab

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6269: -- Summary: Sort properties in the DoY UI Configuration tab Key: DRILL-6269 URL: https://issues.apache.org/jira/browse/DRILL-6269 Project: Apache Drill Issue Type

[jira] [Created] (DRILL-6268) Drill-on-YARN client obtains HDFS URL incorrectly

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6268: -- Summary: Drill-on-YARN client obtains HDFS URL incorrectly Key: DRILL-6268 URL: https://issues.apache.org/jira/browse/DRILL-6268 Project: Apache Drill Issue

[jira] [Created] (DRILL-6267) Revise DoY DfsFacade class to use DrillFileSystem

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6267: -- Summary: Revise DoY DfsFacade class to use DrillFileSystem Key: DRILL-6267 URL: https://issues.apache.org/jira/browse/DRILL-6267 Project: Apache Drill Issue

[jira] [Created] (DRILL-6266) Incorrect log path when Drill launched with --site option

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6266: -- Summary: Incorrect log path when Drill launched with --site option Key: DRILL-6266 URL: https://issues.apache.org/jira/browse/DRILL-6266 Project: Apache Drill

[jira] [Created] (DRILL-6265) Drill pid dir not set to site directory when using --site option

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6265: -- Summary: Drill pid dir not set to site directory when using --site option Key: DRILL-6265 URL: https://issues.apache.org/jira/browse/DRILL-6265 Project: Apache Drill

[jira] [Created] (DRILL-6264) Incorrect log config path for Drill-on-YARN client

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6264: -- Summary: Incorrect log config path for Drill-on-YARN client Key: DRILL-6264 URL: https://issues.apache.org/jira/browse/DRILL-6264 Project: Apache Drill Issue

[jira] [Created] (DRILL-6263) Improvements to DoY initial experience

2018-03-17 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6263: -- Summary: Improvements to DoY initial experience Key: DRILL-6263 URL: https://issues.apache.org/jira/browse/DRILL-6263 Project: Apache Drill Issue Type

[GitHub] drill pull request #1114: Drill-6104: Added Logfile Reader

2018-03-16 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1114#discussion_r175244984 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/log/LogFormatPlugin.java --- @@ -0,0 +1,151 @@ +package

<    1   2   3   4   5   6   7   8   9   10   >