Re: Handling schema change in blocking operators

Hi Boaz, As noted earlier, it would be wonderful if Drill could handle schema changes on the fly, using only the information in the files as they are read, and with only a few code changes. Alas, such is not the case. Question: is the goal to have schema changes somewhat less often (but they

[GitHub] lushuifeng commented on issue #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use

lushuifeng commented on issue #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use URL: https://github.com/apache/drill/pull/1524#issuecomment-436478375 @vvysotskyi There are some type mismatch errors in the tests since RelBuilder.simplify is not always false so

Re: [DISCUSS] 1.15.0 release

Hi Vitalii, Thanks for volunteering to be the release manager! I am currently working on DRILL-6770( https://issues.apache.org/jira/browse/DRILL-6770) which is upgrading the MapR/OJAI versions. We are running into some wrong results issues and I am actively working on it. I think we should

Re: Handling schema change in blocking operators

 Hi Paul, (_a_)  Having a "schema file" sounds like contradiction to calling Drill "schema free"; maybe we could "sweep it under the mat" by creating a new convention for scanners, such that if a scanner has multiple files to read (e.g. f1.csv, f2,csv, ...), then is there's some file named

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231341908 ## File path: contrib/format-msgpack/src/main/java/org/apache/drill/exec/store/msgpack/MsgpackSchema.java ## @@

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231341649 ## File path: contrib/format-msgpack/src/main/java/org/apache/drill/exec/store/msgpack/MsgpackSchema.java ## @@

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231340882 ## File path: contrib/format-msgpack/src/main/java/org/apache/drill/exec/store/msgpack/MsgpackSchema.java ## @@

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231339350 ## File path: contrib/format-msgpack/src/main/java/org/apache/drill/exec/store/msgpack/MsgpackSchema.java ## @@

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231338575 ## File path: contrib/format-msgpack/src/main/java/org/apache/drill/exec/store/msgpack/MsgpackReader.java ## @@

[GitHub] bbevens closed pull request #1526: Md 4946: Added Developer Day and User Meet-up links

bbevens closed pull request #1526: Md 4946: Added Developer Day and User Meet-up links URL: https://github.com/apache/drill/pull/1526 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a

[jira] [Created] (DRILL-6833) MapRDB queries with Run Time Filters with row_key/Secondary Index Should Support Pushdown

Gautam Parai created DRILL-6833: --- Summary: MapRDB queries with Run Time Filters with row_key/Secondary Index Should Support Pushdown Key: DRILL-6833 URL: https://issues.apache.org/jira/browse/DRILL-6833

[jira] [Created] (DRILL-6832) Remove old "unmanaged" sort implementation

Paul Rogers created DRILL-6832: -- Summary: Remove old "unmanaged" sort implementation Key: DRILL-6832 URL: https://issues.apache.org/jira/browse/DRILL-6832 Project: Apache Drill Issue Type:

[GitHub] jcmcote commented on issue #1500: DRILL-6820: Msgpack format reader

jcmcote commented on issue #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#issuecomment-436417739 > @jcmcote, in [HADOOP-13578](https://github.com/apache/hadoop/commit/a0a276162147e843a5a4e028abdca5b66f5118da#diff-8983b1157165c3e54c37877bc01da2ec)

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231300996 ## File path: contrib/format-msgpack/src/main/java/org/apache/drill/exec/store/msgpack/MsgpackReader.java ## @@

Re: [DISCUSS] 1.15.0 release

We should try to get "DRILL-5671: Set secure ACLs (Access Control List) for Drill ZK nodes in a secure cluster" into 1.15. Already +1ed by a committer but waiting for a +1 from another committer who had also participated in the review. On Tue, Nov 6, 2018 at 9:46 AM Vitalii Diravka wrote: > Hi

Re: Handling schema change in blocking operators

HI Aman, I would completely agree with the analysis -- except for the fact that we can't create a general solution, only a patchwork of incomplete ad-hoc solutions. The question is not whether it would be useful to have a general solution (it would), rather whether it is technically possible

Re: [DISCUSS] Resurrect support for Table Statistics in Drill

Hi All, Stats would be a great addition. Here are a couple of issues that came up in the earlier code review, revisited in light of recent proposed work. First, the code to gather the stats is rather complex; it is the evolution of some work an intern did way back when. We'd be advised to find

Re: [DISCUSS] 1.15.0 release

Hi Vitalii We should investigate and fix this issue. https://issues.apache.org/jira/browse/DRILL-6816 Thanks, Khurram On Tue, Nov 6, 2018 at 9:46 AM Vitalii Diravka wrote: > Hi Drillers, > > It's been 3 months since the last release and it is time to do the next > one. > > I'll volunteer to

[GitHub] dvjyothsna opened a new pull request #1526: Md 4946: Added Developer Day and User Meet-up links

dvjyothsna opened a new pull request #1526: Md 4946: Added Developer Day and User Meet-up links URL: https://github.com/apache/drill/pull/1526 @aashreya , @bbevens Please review This is an automated message from the Apache

[GitHub] jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader

jcmcote commented on a change in pull request #1500: DRILL-6820: Msgpack format reader URL: https://github.com/apache/drill/pull/1500#discussion_r231264607 ## File path: contrib/codec-zstd/src/main/java/org/apache/hadoop/io/compress/zstd/ZstdDecompressor.java ## @@ -0,0

Re: [DISCUSS] 1.15.0 release

Hi Vitalii, I have a JIRA which I’ve been working on for a format plugin for Syslog formatted data. It’s basically done and if I can get the PR submitted in the nexf few days or so, could we try to get that into 1.15? > On Nov 6, 2018, at 12:45, Vitalii Diravka wrote: > > Hi Drillers, > >

[DISCUSS] 1.15.0 release

Hi Drillers, It's been 3 months since the last release and it is time to do the next one. I'll volunteer to manage the release :) There are 32 open tickets that are still intended to be included in 1.15.0 release [1]. What do you guys think which tickets do we want to include and what time will

Re: Handling schema change in blocking operators

Hi Paul, Thanks for the feedback ! I am in complete favor of doing the schema discovery and schema hinting. But even on this list in the past we have discussed other use cases such as IoT devices where the schema-on-read is needed (I think it was in the context of the 'death of schema-on-read'

[GitHub] gfilipiak commented on issue #1525: DRILL-6831: Adding information about possible authentication settings in connection URL for mongodb

gfilipiak commented on issue #1525: DRILL-6831: Adding information about possible authentication settings in connection URL for mongodb URL: https://github.com/apache/drill/pull/1525#issuecomment-436320863 @vdiravka done

[jira] [Created] (DRILL-6831) Adding information about possible authentication settings in connection URL for mongodb

Gabriel Filipiak created DRILL-6831: --- Summary: Adding information about possible authentication settings in connection URL for mongodb Key: DRILL-6831 URL: https://issues.apache.org/jira/browse/DRILL-6831

[GitHub] vdiravka commented on issue #1525: Update 090-mongodb-storage-plugin.md

vdiravka commented on issue #1525: Update 090-mongodb-storage-plugin.md URL: https://github.com/apache/drill/pull/1525#issuecomment-436317464 Thanks for the contribution, @gfilipiak Could you also create a JIRA and add it to the title of the PR similar to this one #1469

[GitHub] gfilipiak opened a new pull request #1525: Update 090-mongodb-storage-plugin.md

gfilipiak opened a new pull request #1525: Update 090-mongodb-storage-plugin.md URL: https://github.com/apache/drill/pull/1525 Adding information about possible authentication settings in connection URL for mongodb This is

Re: [DISCUSS] Resurrect support for Table Statistics in Drill

+1 It will help to rely on that code in the process of implementing Drill Metastore, DRILL-6552. @Gautam Please address all current commits and rebase onto latest master, then Vova and me will do additional review for it. Just for clarification, am I right, the changes state is the same as in

[GitHub] oleg-zinovev edited a comment on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

oleg-zinovev edited a comment on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-436205818 @vvysotskyi review, please. Also, can you validate the `sqlline.bat` on Windows? I

[GitHub] oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-436205818 @vvysotskyi review, please. Also, can you validate the sqlline.bat on Windows? I have no

[GitHub] oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

oleg-zinovev commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-436203549 @vvysotskyi > Let's set false for maven-surefire-plugin Look's like this problem

[GitHub] lushuifeng commented on a change in pull request #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use

lushuifeng commented on a change in pull request #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use URL: https://github.com/apache/drill/pull/1524#discussion_r231061174 ## File path:

[GitHub] vvysotskyi commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

vvysotskyi commented on issue #1446: DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM URL: https://github.com/apache/drill/pull/1446#issuecomment-436195360 @oleg-zinovev, now `javassist 3.24.0-GA` available from the maven central repo:

[GitHub] vvysotskyi commented on a change in pull request #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use

vvysotskyi commented on a change in pull request #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use URL: https://github.com/apache/drill/pull/1524#discussion_r231056622 ## File path:

[GitHub] lushuifeng edited a comment on issue #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use

lushuifeng edited a comment on issue #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use URL: https://github.com/apache/drill/pull/1524#issuecomment-436191071 @vvysotskyi Could you please review this? This

[GitHub] lushuifeng commented on issue #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use

lushuifeng commented on issue #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use URL: https://github.com/apache/drill/pull/1524#issuecomment-436191071 @vvysotskyi Could please review this? This is an

[GitHub] lushuifeng opened a new pull request #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use

lushuifeng opened a new pull request #1524: DRILL-6830: Remove Hook.REL_BUILDER_SIMPLIFY handler after use URL: https://github.com/apache/drill/pull/1524 Please see

[jira] [Created] (DRILL-6830) Hook.REL_BUILDER_SIMPLIFY handler didn't removed cause performance degression

shuifeng lu created DRILL-6830: -- Summary: Hook.REL_BUILDER_SIMPLIFY handler didn't removed cause performance degression Key: DRILL-6830 URL: https://issues.apache.org/jira/browse/DRILL-6830 Project:

[GitHub] oleg-zinovev commented on a change in pull request #1450: DRILL-6717: lower and upper functions not works with national characters

oleg-zinovev commented on a change in pull request #1450: DRILL-6717: lower and upper functions not works with national characters URL: https://github.com/apache/drill/pull/1450#discussion_r231030533 ## File path: