[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135979350 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java --- @@ -130,562 +145,248 @@ public IterOutcome in

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135977556 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java --- @@ -39,88 +35,107 @@ import org.apache.d

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135976739 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/unit/TestEmptyBatchMiniPlan.java --- @@ -0,0 +1,453 @@ +/* + * Licensed to th

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974928 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974916 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974905 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974302 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974310 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974133 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135974107 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

Re: Custom extension storage plugin development

Thank you very much Muhammad and Paul, I'll go through the pcap plugin and read the docs again. Regards, Charuta On Tue, Aug 29, 2017 at 11:35 PM, Paul Rogers wrote: > Also look at the recently-added pcap format plugin. Seems the developers > there found a minimum-effort path to doing all the

Re: MAX_FIELD_LENGTH Validation doesn't allow queries over large strings

(Reposting an answer sent earlier for those whose e-mail filters discard Github-related e-mails…) Drill is an analytic engine optimized for numbers and short strings. At present, Drill’s practical limit on string (i.e. Varchar) length is 256 characters or less. Drill is vectorized. Drill tries

MAX_FIELD_LENGTH Validation doesn't allow queries over large strings

Hi, We are working with a dataset with columns of large Strings on it, and we are having the error "UNSUPPORTED_OPERATION ERROR: Trying to write something big in a column columnIndex 0 Limit 65536 Failure while reading file ..." There is a validation on FieldVarCharOutput.java

[jira] [Resolved] (DRILL-5732) Unable to allocate sv2 for 9039 records, and not enough batchGroups to spill.

[ https://issues.apache.org/jira/browse/DRILL-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Hou resolved DRILL-5732. --- Resolution: Not A Problem > Unable to allocate sv2 for 9039 records, and not enough batchGroups to spi

[GitHub] drill pull request #919: DRILL-5721: Query with only root fragment and no no...

Github user sohami commented on a diff in the pull request: https://github.com/apache/drill/pull/919#discussion_r135949946 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -1109,7 +1102,7 @@ private void setupNonRootFragments(final Coll

Re: Drill 2.0 (design) hackathon

Hi Paul, certainly makes sense to have the API compatibility discussions during this hackathon. The 2.0 release may be a good checkpoint to introduce breaking changes necessitating changes to the ODBC/JDBC drivers and other external applications. As part of this exercise (not during the hackathon

[GitHub] drill pull request #919: DRILL-5721: Query with only root fragment and no no...

Github user sohami commented on a diff in the pull request: https://github.com/apache/drill/pull/919#discussion_r135948567 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -1073,26 +1070,22 @@ public QueryId getQueryId() { */

[GitHub] drill pull request #919: DRILL-5721: Query with only root fragment and no no...

Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/919#discussion_r135937433 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -1073,26 +1070,22 @@ public QueryId getQueryId() {

[GitHub] drill pull request #919: DRILL-5721: Query with only root fragment and no no...

Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/919#discussion_r135937508 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -1109,7 +1102,7 @@ private void setupNonRootFragments(final

[GitHub] drill pull request #926: DRILL-5269 Make DirectSubScan Jackson JSON deserial...

GitHub user vrozov opened a pull request: https://github.com/apache/drill/pull/926 DRILL-5269 Make DirectSubScan Jackson JSON deserializable @amansinha100 Please review You can merge this pull request into a Git repository by running: $ git pull https://github.com/vrozov/drill

[GitHub] drill pull request #924: DRILL-5740: Ensure spill directories are unique

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/924 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [GitHub] drill pull request #:

Drill is an analytic engine optimized for numbers and short strings. At present, Drill’s practical limit on string (i.e. Varchar) length is 256 characters or less. Drill is vectorized. Drill tries to create “batches” of data with up to 64K records. When individual columns are wider than 256 (on

[GitHub] drill pull request #:

Github user paul-rogers commented on the pull request: https://github.com/apache/drill/commit/745bcd1f378397d921860eeba382530483644cd3#commitcomment-23958825 Please move this discussion to the Apache Drill "dev" list. This PR is a bit old... --- If your project is set up for it, you

[GitHub] drill pull request #:

Github user jcesart20 commented on the pull request: https://github.com/apache/drill/commit/745bcd1f378397d921860eeba382530483644cd3#commitcomment-23958024 Hi, We are working with a dataset with columns of large Strings on it, and we are having the error "UNSUPPORTED_OPERATION ERR

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135925091 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135925006 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135924402 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135924386 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135924412 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -768,4 +765,73 @@ else if (exprHasPrefix

[GitHub] drill issue #924: DRILL-5740: Ensure spill directories are unique

Github user jinfengni commented on the issue: https://github.com/apache/drill/pull/924 +1 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[jira] [Created] (DRILL-5750) NPE on botched Drillbit startup

Paul Rogers created DRILL-5750: -- Summary: NPE on botched Drillbit startup Key: DRILL-5750 URL: https://issues.apache.org/jira/browse/DRILL-5750 Project: Apache Drill Issue Type: Bug Affects

Re: Drill 2.0 (design) hackathon

Thanks Aman for organizing the Hackathon! The list included many good ideas for Drill 2.0. Some of those require changes to Drill’s “public” interfaces (file format, client protocol, SQL behavior, etc.) At present, Drill has no good mechanism to handle backward/forward compatibility at the API

Re: Speeding Up Unit Tests

Thanks Paul, I see now the parent pom already configures surefire to fork 2 processes to run tests. I tried running multiple tests in parallel in each forked process, but there were errors with BaseTestQuery, probably because it uses static variables. One additional speedup I found was that the

Re: Drill 2.0 (design) hackathon

Hi Anil, yes, certainly talking about your work with the Kafka+Drill integration would be very welcome. Regarding the registration, there isn't anything formal yet. We would like to keep it lightweight but to keep track of the topics and attendees I will send out either a Google doc or some such w

Re: Drill 2.0 (design) hackathon

Hi Aman, To attend Drill Developer's Day event, is there any registration process? Me and Kamesh wanted to present Kafka integration with Drill ( https://github.com/akumarb2010/incubator-drill/tree/master/contrib/storage-kafka & https://issues.apache.org/jira/browse/DRILL-4779 ) Is it possible t

[GitHub] drill issue #913: - DRILL-5729 Fix Travis Build

Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/913 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r135872295 --- Diff: logical/src/main/java/org/apache/drill/common/expression/SchemaPath.java --- @@ -115,6 +112,33 @@ public static SchemaPath create(NamePart nameP

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r135868684 --- Diff: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/binary/CompareFunctionsProcessor.java --- @@ -82,466 +45,37 @@ public s

Re: Custom extension storage plugin development

Also look at the recently-added pcap format plugin. Seems the developers there found a minimum-effort path to doing all the necessary steps. The steps are more complex than we’d like, but pcap is as a good example as any for how to work your way through those steps. - Paul > On Aug 29, 2017, a

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/904#discussion_r135848539 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java --- @@ -22,14 +22,15 @@ import org.apache.drill.BaseTestQ

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/904#discussion_r135850298 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java --- @@ -73,16 +74,23 @@ public void testCastByConstantFolding(

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/904#discussion_r135849992 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java --- @@ -73,16 +74,23 @@ public void testCastByConstantFolding(

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/904#discussion_r135850825 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestDateFunctions.java --- @@ -20,7 +20,9 @@ import static org.junit.Assert.asse

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/904#discussion_r135851159 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestDateFunctions.java --- @@ -135,11 +139,16 @@ public void testIntervalArithmetic()

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/904#discussion_r135850258 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java --- @@ -73,16 +74,23 @@ public void testCastByConstantFolding(

[GitHub] drill pull request #925: DRILL-5749: solve foreman and netty threads deadloc...

GitHub user weijietong opened a pull request: https://github.com/apache/drill/pull/925 DRILL-5749: solve foreman and netty threads deadlock break the nest invocation of channelClosed method to avoid nested lock holding You can merge this pull request into a Git repository by runnin

[jira] [Created] (DRILL-5749) Foreman and Netty threads occure deadlock

weijie.tong created DRILL-5749: -- Summary: Foreman and Netty threads occure deadlock Key: DRILL-5749 URL: https://issues.apache.org/jira/browse/DRILL-5749 Project: Apache Drill Issue Type: Bug

Re: Custom extension storage plugin development

I assume this page can give you the overview you are looking for ? May be you can start with asking specific questions. Thanks, Gelbana On Tue, Aug 29, 2017 at 12:26 PM, Charuta Rajopadhye wrote: > Hi Team, > > I need

[GitHub] drill pull request #909: DRILL-4264: Allow field names to include dots

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/909#discussion_r135753493 --- Diff: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/binary/CompareFunctionsProcessor.java --- @@ -82,466 +45,37 @@ public st

Custom extension storage plugin development

Hi Team, I need to develop an extension storage plugin for my data source. I have been through this and Apache Drill's documentation but did not fin

Re: Test cases that require a UTC timezone.

Hi Muhammad, You can use UTC timezone to pass all unit tests. I replied in your jira ticket regarding two test failures for the local timezone. Kind regards Vitalii On Mon, Aug 28, 2017 at 5:44 PM, Muhammad Gelbana wrote: > ​​ > In the "Drill developer guide or code organization" thread > ​ y

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135710183 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -329,9 +326,11 @@ public TypedFieldId getValueVectorId(Sch

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135709915 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -252,14 +235,28 @@ public IterOutcome next() { }

[GitHub] drill pull request #906: DRILL-5546: Handle schema change exception failure ...

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/906#discussion_r135708467 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -152,97 +157,75 @@ public void kill(boolean sendUpstream)