[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98288738 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -426,6 +459,7 @@ public void

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98288632 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -399,9 +430,11 @@ public void

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98289161 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/BufferedDirectBufInputStream.java --- @@ -179,10 +189,10 @@ private int

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98288871 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/BufferedDirectBufInputStream.java --- @@ -43,7 +45,7 @@

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98268266 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -41,26 +42,33 @@ import

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98288225 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -197,6 +213,21 @@ public

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98269966 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -109,17 +122,30 @@ private DrillBuf

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98271891 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -282,89 +356,119 @@ public

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98270983 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -176,12 +202,29 @@ private DrillBuf

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98269034 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -41,26 +42,33 @@ import

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98271038 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -192,45 +235,74 @@ private DrillBuf

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98268204 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -41,26 +42,33 @@ import

[GitHub] drill pull request #723: DRILL-5207: Improve Parquet Scan pipelining.

2017-01-27 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/723#discussion_r98289478 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/DirectBufInputStream.java --- @@ -81,7 +87,15 @@ public synchronized int

Re: Integration with Spark

2017-01-27 Thread Hanifi GUNES
I authored the initial Spark(DoS) plugin but it was never released due to priorities. The initial implementation allows full duplex data exchange between Drill and Spark. That is, we can use Drill to query your data lake and do further iterative ML on it via Spark or vice versa. I will need to

[jira] [Created] (DRILL-5230) Translation of millisecond duration into hours is incorrect

2017-01-27 Thread Kunal Khatua (JIRA)
Kunal Khatua created DRILL-5230: --- Summary: Translation of millisecond duration into hours is incorrect Key: DRILL-5230 URL: https://issues.apache.org/jira/browse/DRILL-5230 Project: Apache Drill

[GitHub] drill pull request #704: DRILL-5125: Provide option to use generic code for ...

2017-01-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/704#discussion_r98299585 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java --- @@ -34,4 +38,33 @@ public void

[GitHub] drill pull request #704: DRILL-5125: Provide option to use generic code for ...

2017-01-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/704#discussion_r97992610 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/GenericSV4Copier.java --- @@ -0,0 +1,67 @@ +/* + * Licensed to

Dynamic UDF Registration

2017-01-27 Thread Charles Givre
I’m having some trouble registering a UDF I wrote. The UDF was working in previous versions of Drill and now Drill doesn’t seem to recognize it. When I try to register it I get the following error: : jdbc:drill:zk=local> create function using jar 'drill-geoip-functions-1.0.jar';

Re: Data types

2017-01-27 Thread Charles Givre
I’m actually one of the contributors for the forthcoming O’Reilly book on Drill (along with Ted and Ellen), and this is a specific functionality I’m planning on writing a chapter about. (Not the buffers, but how to get Drill to ingest other file formats) > On Jan 27, 2017, at 11:50, Paul

Re: Data types

2017-01-27 Thread Paul Rogers
Hi Charles, Congrats! Unfortunately, no, there is no documentation. Drill seems to be of the “code speaks for itself” persuasion. I try to document the bits I’ve had to learn on my Github Wiki, but (until now) I’ve not looked at this particular area. IMHO, now that the plugins basically work,

Re: Data types

2017-01-27 Thread Charles Givre
Hi Paul, VICTORY!! I just set the buffer size to 4096 and it worked perfectly without truncating my data! Is this documented anywhere? I’ve been trying to really wrap my head around the mechanics of how Drill reads data and how the format plugins work and really haven’t found much. I’ve

[GitHub] drill pull request #580: DRILL-4824: JSON with complex nested data produces ...

2017-01-27 Thread Serhii-Harnyk
Github user Serhii-Harnyk commented on a diff in the pull request: https://github.com/apache/drill/pull/580#discussion_r98232198 --- Diff: exec/vector/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java --- @@ -317,6 +317,12 @@ public Object getObject(int index) {

[GitHub] drill pull request #731: DRILL-5224: CTTAS: fix errors connected with system...

2017-01-27 Thread arina-ielchiieva
GitHub user arina-ielchiieva opened a pull request: https://github.com/apache/drill/pull/731 DRILL-5224: CTTAS: fix errors connected with system path delimiters (… …Windows) Replaced `java.nio.file.Paths.get()` to `org.apache.hadoop.fs.Path` as the latest works the

[GitHub] drill pull request #685: Drill 5043: Function that returns a unique id per s...

2017-01-27 Thread arina-ielchiieva
Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/685#discussion_r98186438 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ContextFunctions.java --- @@ -64,17 +65,45 @@ public void eval() {