[GitHub] drill issue #914: DRILL-5657: Size-aware vector writer structure

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/914 Thank you, @bitblender, for your in-person review. I've pushed a commit that reflects your comments. ---

[GitHub] drill issue #1014: DRILL-5771: Fix serDe errors for format plugins

Github user ilooner commented on the issue: https://github.com/apache/drill/pull/1014 Thanks I took a look. ---

Re: Hangout Topics for 11/14/2017

Here are the topics so far: Unit Testing - Tim 1.12 Release - Arina Metadata Management - Padma Thanks Padma On Nov 13, 2017, at 1:15 PM, Padma Penumarthy > wrote: Drill hangout tomorrow Nov 14th, at 10 AM PST. Please send email or bring them

[GitHub] drill pull request #978: DRILL-5842: Refactor and simplify the fragment, ope...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/978 ---

[jira] [Resolved] (DRILL-4708) connection closed unexpectedly

[ https://issues.apache.org/jira/browse/DRILL-4708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthikeyan Manivannan resolved DRILL-4708. --- Resolution: Works for Me > connection closed unexpectedly >

"Batch size control" and unit testing

Hi All, Here is the next installment in the “batch size control” project update. Drill has a great many operators. As we move forward, we must update them to use the new batch size control framework. Unit testing becomes a major concern. This note explains how we address that issue in this

[GitHub] drill issue #1026: DRILL-5919: Add non-numeric support for JSON processing

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1026 On the two functions... Maybe just have one function that handles the Nan/Infinity case. As noted earlier, no matter what we do, JSON without these symbols will work. So, we need only consider

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150688725 --- Diff: exec/java-exec/src/main/resources/drill-module.conf --- @@ -502,6 +502,8 @@ drill.exec.options: { store.format: "parquet",

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150688949 --- Diff: exec/java-exec/src/main/resources/drill-module.conf --- @@ -502,6 +502,8 @@ drill.exec.options: { store.format: "parquet",

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150689236 --- Diff: exec/java-exec/src/main/resources/drill-module.conf --- @@ -502,6 +502,8 @@ drill.exec.options: { store.format: "parquet",

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150689617 --- Diff: contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java --- @@ -73,6 +73,7 @@ private final

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150682963 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/SchemaTreeProvider.java --- @@ -20,12 +20,13 @@ import java.io.IOException;

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150686404 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java --- @@ -229,7 +229,7 @@ public DrillbitContext

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150685672 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -175,6 +193,21 @@ public WorkspaceSchema

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150685414 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -150,14 +152,30 @@ public

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150680095 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150684042 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemSchemaFactory.java --- @@ -52,9 +55,20 @@ private List

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150679941 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150686157 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -532,7 +572,10 @@ public boolean isMutable()

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150682252 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150680632 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150680951 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150685946 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -532,7 +572,10 @@ public boolean isMutable()

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150682787 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicSchema.java --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150679636 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150684114 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemSchemaFactory.java --- @@ -52,9 +55,20 @@ private List

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150682074 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150684557 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemSchemaFactory.java --- @@ -73,9 +87,10 @@ public void

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150683680 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/SchemaTreeProvider.java --- @@ -105,12 +106,36 @@ public SchemaPlus

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150682678 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicSchema.java --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150685113 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemSchemaFactory.java --- @@ -73,9 +87,10 @@ public void

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150680454 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150678803 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill pull request #1032: DRILL-5089: Dynamically load schema of storage plu...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1032#discussion_r150678680 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DynamicRootSchema.java --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] drill issue #1026: DRILL-5919: Add non-numeric support for JSON processing

Github user vladimirtkach commented on the issue: https://github.com/apache/drill/pull/1026 what do you think about having two functions instead: convertFromJSON and convertFromJSON+some suffix. Second will be able to convert NaN, Infinity ---

Hangout Topics for 11/14/2017

Drill hangout tomorrow Nov 14th, at 10 AM PST. Please send email or bring them up tomorrow, if you have topics to discuss. Hangout link: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks Padma

[GitHub] drill issue #1030: DRILL-5941: Skip header / footer improvements for Hive st...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1030 For FWIW, the native CSV reader does the following: * To read the header, it seeks to offset 0 in the file, regardless of the block being read, then reads the header, which may be a

[GitHub] drill issue #1030: DRILL-5941: Skip header / footer improvements for Hive st...

Github user ppadma commented on the issue: https://github.com/apache/drill/pull/1030 @arina-ielchiieva That will have performance impact. Better way would be to keep one reader per split and see if we can figure out a way to tell readers how many rows they should skip (for header or

[GitHub] drill issue #1031: DRILL-5917: Ban org.json:json library in Drill

Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1031 @arina-ielchiieva Please test with the latest commit. I changed mapr version back to 5.2.1, note that 2.7.0-mapr-1707 is compiled with 5.2.2 ---

[jira] [Created] (DRILL-5960) Add function STAsGeoJSON to extend GIS support

Chris Sandison created DRILL-5960: - Summary: Add function STAsGeoJSON to extend GIS support Key: DRILL-5960 URL: https://issues.apache.org/jira/browse/DRILL-5960 Project: Apache Drill Issue

[GitHub] drill issue #984: DRILL-5783 Made a unit test for generated Priority Queue. ...

Github user ilooner commented on the issue: https://github.com/apache/drill/pull/984 @arina-ielchiieva I can put **DRILL-5783** in a separate PR **DRILL-5841** and **DRILL-5894** are however tightly coupled and must be in the same PR. Unfortunately **DRILL-5841** and **DRILL-5894**

[GitHub] drill issue #1031: DRILL-5917: Ban org.json:json library in Drill

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1031 @vrozov I have posted all content of drillbit.out. drillbit.log just had information that could not create remote function registry directory. ---

[GitHub] drill issue #1031: DRILL-5917: Ban org.json:json library in Drill

Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1031 @arina-ielchiieva Please attach the full log to JIRA. ---

[GitHub] drill issue #1030: DRILL-5941: Skip header / footer improvements for Hive st...

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1030 @ppadma really good point! I did not realize that when creating readers input splits can be already distributed among several drillbits. So I guess we should make sure that when we have

[GitHub] drill pull request #970: DRILL-5832: Migrate OperatorFixture to use SystemOp...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/970 ---

[GitHub] drill issue #978: DRILL-5842: Refactor and simplify the fragment, operator c...

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/978 +1, LGTM. ---

[GitHub] drill issue #970: DRILL-5832: Migrate OperatorFixture to use SystemOptionMan...

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/970 +1, LGTM. ---

[GitHub] drill issue #984: DRILL-5783 Made a unit test for generated Priority Queue. ...

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/984 @ilooner changes in 361 files seems to be too much :) I suggest you split this PR at least at three according to the addressed issues. So we try to merge smaller chunks. ---

[GitHub] drill issue #1031: DRILL-5917: Ban org.json:json library in Drill

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1031 @vrozov when I have deployed Drill with your changes on test cluster, Drill start up failed with the following error: ``` 2017-11-13 01:30:12,3367 ERROR JniCommon

[GitHub] drill pull request #904: DRILL-5717: change some date time test cases with s...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/904 ---

[GitHub] drill pull request #774: DRILL-5337: OpenTSDB storage plugin

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/774 ---

[GitHub] drill pull request #987: DRILL-5863: Sortable table incorrectly sorts fragme...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/987 ---

[GitHub] drill pull request #1005: DRILL-5896: Handle HBase columns vector creation i...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1005 ---

[GitHub] drill pull request #1021: DRILL-5923: Display name for query state

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1021 ---

[GitHub] drill pull request #1015: DRILL-5899: Simple pattern matchers can work with ...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1015 ---

[GitHub] drill pull request #1019: DRILL-5909: Added new Counter metrics

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1019 ---

[GitHub] drill pull request #949: DRILL-5795: Parquet Filter push down at rowgroup le...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/949 ---

[GitHub] drill pull request #1017: DRILL-5822: The query with "SELECT *" with "ORDER ...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1017 ---

[GitHub] drill pull request #1020: DRILL-5921: Display counter metrics in table

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1020 ---

[GitHub] drill pull request #1033: DRILL-5952: Implement "CREATE TABLE IF NOT EXISTS"

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1033#discussion_r150512392 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestCTAS.java --- @@ -313,6 +313,59 @@ public void createTableWithCustomUmask()

[GitHub] drill pull request #1033: DRILL-5952: Implement "CREATE TABLE IF NOT EXISTS"

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1033#discussion_r150513171 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/CreateTableHandler.java --- @@ -83,7 +84,20 @@ public

[GitHub] drill pull request #1033: DRILL-5952: Implement "CREATE TABLE IF NOT EXISTS"

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1033#discussion_r150512944 --- Diff: exec/java-exec/src/main/codegen/includes/parserImpls.ftl --- @@ -241,8 +243,8 @@ SqlNode SqlCreateTable() : query =

[GitHub] drill pull request #1033: DRILL-5952: Implement "CREATE TABLE IF NOT EXISTS"

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1033#discussion_r150512516 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestCTAS.java --- @@ -313,6 +313,59 @@ public void createTableWithCustomUmask()

[GitHub] drill pull request #1033: DRILL-5952: Implement "CREATE TABLE IF NOT EXISTS"

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1033#discussion_r150512175 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestCTAS.java --- @@ -313,6 +313,59 @@ public void createTableWithCustomUmask()

[jira] [Created] (DRILL-5959) Unable to apply Union on multiple select query expressions

Rajeshwar Rao created DRILL-5959: Summary: Unable to apply Union on multiple select query expressions Key: DRILL-5959 URL: https://issues.apache.org/jira/browse/DRILL-5959 Project: Apache Drill

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user vladimirtkach commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150500913 --- Diff: exec/java-exec/src/main/resources/drill-module.conf --- @@ -502,6 +502,8 @@ drill.exec.options: { store.format: "parquet",

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user vladimirtkach commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150500204 --- Diff: exec/java-exec/src/main/resources/drill-module.conf --- @@ -502,6 +502,8 @@ drill.exec.options: { store.format: "parquet",

[GitHub] drill pull request #1026: DRILL-5919: Add non-numeric support for JSON proce...

Github user vladimirtkach commented on a diff in the pull request: https://github.com/apache/drill/pull/1026#discussion_r150500059 --- Diff: exec/java-exec/src/main/resources/drill-module.conf --- @@ -502,6 +502,8 @@ drill.exec.options: { store.format: "parquet",

[GitHub] drill issue #1014: DRILL-5771: Fix serDe errors for format plugins

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1014 @ilooner Paul has correctly described race condition issue. At planning time when we have defined plugin configuration we want to make sure it would stay the same during query execution.

[GitHub] drill issue #904: DRILL-5717: change some date time test cases with specific...

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/904 +1, LGTM. ---

[GitHub] drill issue #774: DRILL-5337: OpenTSDB storage plugin

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/774 +1, LGTM. ---