Re: Partition pruning inconsistency

Yes we would need to enhance the protocol a bit. Depending on whether the warning is issued before results are sent (as in planning/optimization) or whether the warning is issued after the results are completed (for example we can issue a warning if, in future, Drill decides to drop bad rows and co

Re: Partition pruning inconsistency

I've been thinking that we need to add support for returning warnings. Have you looked how to add to JDBC or ODBC? We'll need to update the RPC protocol since I believe we don't currently have an accommodation for warnings. Maybe add along with DDL queries? -- Jacques Nadeau CTO and Co-Founder, D

Re: Partition pruning inconsistency

A better idea would be to return a warning in the results and let jdbc/odbc show a warning with the result data. On Wed, Aug 26, 2015 at 8:31 AM, Aman Sinha wrote: > We have had some issues where the same query run at different times > (possibly with other queries running concurrently...not sure

Re: Source code for forked parquet library.

If I recall, part of the initial patch for Parquet pushdown was more focused on rowgroup pruning during planning. I believe it was based on the old partition pruning code (could be wrong). Furthermore, it conflicted with the behavior of the metadata caching since the caching didn't (at the time) r

Re: Potential resource for large scale testing

I spent a bunch of time looking at the Phi coprocessors and forgot to get back to the thread. I'd love it if someone spent some time looking at leveraging them (since Drill is frequently processor bound). Any takers? -- Jacques Nadeau CTO and Co-Founder, Dremio On Mon, Aug 31, 2015 at 10:24 PM

Re: Dev tool

Just remember to do a clean build from the command line before importing the maven project in Eclipse/IntelliJ. On Mon, Aug 31, 2015 at 10:10 PM, Jacques Nadeau wrote: > I think it is about 1/3 Eclipse, 2/3 Intellij. > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Mon, Aug 31, 2015 a

Re: Cross Origin REST API calls in embedded mode (for testing)

Drill uses Jetty internally. You can insert a ServletFilter that provides the additional headers required to allow a cross origin policy in the code by updating the Jetty initialization. The code for the server is here: https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org

Re: Potential resource for large scale testing

Hi Edmon, Sorry no one seems to have got back to you on this. We are in the process of publishing a test suite for regression testing Drill and the cluster you have (even a few nodes ) would be a great resource for folks to run the test suite. Rahul, et al are working on this and I would sugges

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386470 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -321,8 +327,101 @@ public DrillTable create(String

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386449 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -321,8 +327,101 @@ public DrillTable create(String

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386365 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -104,6 +107,9 @@ public WorkspaceSchemaFactory(Dri

Re: Cross Origin REST API calls in embedded mode (for testing)

Your error means that you cannot invoke a script from a location that is different from the location of your original script, i.e. the scripts must be from the same 'origin'. 'Origin' is different even if the ports the scripts are served from are different, which is probably the case here. Unfortun

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386336 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/BasicFormatMatcher.java --- @@ -72,7 +72,7 @@ public boolean supportDirectoryReads()

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386305 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -321,8 +327,101 @@ public DrillTable create(String

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386284 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -321,8 +327,101 @@ public DrillTable create(String

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386206 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/BasicFormatMatcher.java --- @@ -72,7 +72,7 @@ public boolean supportDirectoryReads()

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/140#discussion_r38386177 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/DrillPathFilter.java --- @@ -23,7 +23,7 @@ public class DrillPathFilter extends

Re: Dev tool

I think it is about 1/3 Eclipse, 2/3 Intellij. -- Jacques Nadeau CTO and Co-Founder, Dremio On Mon, Aug 31, 2015 at 10:08 PM, Edmon Begoli wrote: > What are you all programming Drill with? > > IDEA, eclipse, vim, emacs, sublime, atom, ...? > > I started with Sublime with vim bindings, but I am

Re: Dev tool

I use IDEA On Tue, Sep 1, 2015 at 7:08 AM, Edmon Begoli wrote: > What are you all programming Drill with? > > IDEA, eclipse, vim, emacs, sublime, atom, ...? > > I started with Sublime with vim bindings, but I am switching to IDEA 14.1. > Easier to catch errors before running manual compiles ...

Dev tool

What are you all programming Drill with? IDEA, eclipse, vim, emacs, sublime, atom, ...? I started with Sublime with vim bindings, but I am switching to IDEA 14.1. Easier to catch errors before running manual compiles ...

Re: Source code for forked parquet library.

I heard that there are some issues between filter push-down and parquet metadata caching thing. But I'm not clear what exactly the problem is, and whether we have a plan to resolve that. Can you elaborate what the open questions are and the conflicts with metadata caching? The reason I'm trying to

[GitHub] drill pull request: DRILL-3724 - added javadoc for core classes fo...

GitHub user ebegoli reopened a pull request: https://github.com/apache/drill/pull/139 DRILL-3724 - added javadoc for core classes for storage plugin development. Changes are only to the javadoc. Code is not affected. You can merge this pull request into a Git repository by running:

[GitHub] drill pull request: DRILL-3724 - added javadoc for core classes fo...

Github user ebegoli closed the pull request at: https://github.com/apache/drill/pull/139 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: Source code for forked parquet library.

Given that Julien and Jason are working heavily on a merge into Parquet, I strongly suggest waiting on merging other patches around that code (or at least working on top of the changes they are doing. I thought there were a number of open questions around the filter pushdown and how it related to

[GitHub] drill pull request: DRILL-3684: CTAS : Memory Leak when using CTAS...

GitHub user adeneche opened a pull request: https://github.com/apache/drill/pull/141 DRILL-3684: CTAS : Memory Leak when using CTAS with tpch sf100 @parthchandra can you please review ? thx You can merge this pull request into a Git repository by running: $ git pull ht

[jira] [Created] (DRILL-3729) Remove info message about invalid argument during query planning where filter is a directory

Victoria Markman created DRILL-3729: --- Summary: Remove info message about invalid argument during query planning where filter is a directory Key: DRILL-3729 URL: https://issues.apache.org/jira/browse/DRILL-3729

[GitHub] drill pull request: DRILL-3661: Edit JDBC doc. for clarity, more c...

Github user adeneche commented on a diff in the pull request: https://github.com/apache/drill/pull/119#discussion_r38373686 --- Diff: exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionConfig.java --- @@ -22,6 +22,24 @@ import net.hydromatic.avatica.ConnectionC

[GitHub] drill pull request: DRILL-3535: Add support for Drop Table

GitHub user mehant opened a pull request: https://github.com/apache/drill/pull/140 DRILL-3535: Add support for Drop Table You can merge this pull request into a Git repository by running: $ git pull https://github.com/mehant/drill DRILL-3535 Alternatively you can review and a

[jira] [Created] (DRILL-3728) millisecond portion of time value missing from query results

Khurram Faraaz created DRILL-3728: - Summary: millisecond portion of time value missing from query results Key: DRILL-3728 URL: https://issues.apache.org/jira/browse/DRILL-3728 Project: Apache Drill

Re: Unforking Calcite...

The idea of Hackathon sounds pretty helpful. I would love to help too. On Mon, Aug 31, 2015 at 3:41 PM, Ted Dunning wrote: > We have a place to do it, in case that helps. > > > > On Mon, Aug 31, 2015 at 9:58 AM, Jinfeng Ni wrote: > > > Hackathon seems to be a brilliant idea. Definitely, it wi

Re: Unforking Calcite...

We have a place to do it, in case that helps. On Mon, Aug 31, 2015 at 9:58 AM, Jinfeng Ni wrote: > Hackathon seems to be a brilliant idea. Definitely, it will help a lot if > we > could get Julian to join and help to review / work through these issues. > > > > On Mon, Aug 31, 2015 at 9:33 AM,

Re: Source code for forked parquet library.

btw, why do not we public the source code jar in the maven nexus repo? That way, the source code could be downloaded with just one click in IDE. On Mon, Aug 31, 2015 at 3:25 PM, Jinfeng Ni wrote: > I'm actually trying Adam's parquet filter pushdown patch (DRILL-1950). > That's > why I happened

Re: Source code for forked parquet library.

I'm actually trying Adam's parquet filter pushdown patch (DRILL-1950). That's why I happened to click one parquet class and hit the above "source code not found" error. Thanks! On Mon, Aug 31, 2015 at 3:20 PM, Jason Altekruse wrote: > https://github.com/mapr/incubator-parquet-mr/tree/1.6.0rc3

Re: Source code for forked parquet library.

https://github.com/mapr/incubator-parquet-mr/tree/1.6.0rc3-drill-r0.3 I am working with Julien Le Dem on getting us off of the fork, but for now the source code is accessible here. Let me know if you need any help looking through the parquet code. Is there a particular JIRA you are trying to addre

Re: Source code for forked parquet library.

The source code is here: https://github.com/mapr/incubator-parquet-mr the branch is 1.6.0rc3-drill-r0.3 Thanks On Mon, Aug 31, 2015 at 3:15 PM, Jinfeng Ni wrote: > It seems we are using a forked parquet library. Can someone point me > to the source code for the forked parquet ? > > I tried to

Source code for forked parquet library.

It seems we are using a forked parquet library. Can someone point me to the source code for the forked parquet ? I tried to download the source code within IDE, and it complains the following: "*Cannot download sources* Sources not found for: com.twitter:parquet-column:1.6.0rc3-drill-r0.3 " So

[jira] [Created] (DRILL-3727) Drill should return NULL instead of failure if cast column is empty

Hao Zhu created DRILL-3727: -- Summary: Drill should return NULL instead of failure if cast column is empty Key: DRILL-3727 URL: https://issues.apache.org/jira/browse/DRILL-3727 Project: Apache Drill

[jira] [Created] (DRILL-3726) Drill is not properly interpreting CRLF (0d0a). CR gets read as content.

Edmon Begoli created DRILL-3726: --- Summary: Drill is not properly interpreting CRLF (0d0a). CR gets read as content. Key: DRILL-3726 URL: https://issues.apache.org/jira/browse/DRILL-3726 Project: Apache

[GitHub] drill pull request: DRILL-3715 Support all encoded vector types fo...

Github user amithadke commented on the pull request: https://github.com/apache/drill/pull/136#issuecomment-136476945 @mehant @jaltekruse Could you take a look at recent patch? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

Re: The meaning of the methods in StoragePlugin and EasyFormatPlugin

Thanks, Daniel. Definitely helpful. On Mon, Aug 31, 2015 at 3:00 PM, Daniel Barclay wrote: > Edmon, > > I see you're working on adding documentation about creating storage > plug-ins. > > I was looking into that myself a little while ago, but wasn't able to > continue. > > Below are some notes o

Re: Unforking Calcite...

Just to clarify, what time frame are we looking for the Hackathon? Given the fact that 1.2 release is quite close, it probably makes more sense to do Hackathon during the 1.3 release time frame. Thoughts? On Mon, Aug 31, 2015 at 9:58 AM, Jinfeng Ni wrote: > Hackathon seems to be a brilliant

Re: The meaning of the methods in StoragePlugin and EasyFormatPlugin

Edmon, I see you're working on adding documentation about creating storage plug-ins. I was looking into that myself a little while ago, but wasn't able to continue. Below are some notes on the detailed requirements I had extracted from the code. Hopefully they'll be helpful in filling in your

Re: Is this known issue?

Hey Edmon, this is definitely a different issue than DRILL-1428. You should open a JIRA and attach the text file to the issue. Thanks On Mon, Aug 31, 2015 at 11:15 AM, Edmon Begoli wrote: > We might have seen an issue with Drill, so I want to see if this is already > reported. > > I don't thin

Is this known issue?

We might have seen an issue with Drill, so I want to see if this is already reported. I don't think that this one addresses it: https://issues.apache.org/jira/browse/DRILL-1428 When we query the last attribute of a text file, we get missing characters. Looking at the row through Drill, a \r is i

Re: Announcing new committer: Abdel Hakim Deneche

Thanks everyone :) On Mon, Aug 31, 2015 at 10:52 AM, Ellen Friedman wrote: > Hakim, > > Congratulations! And thanks for all your efforts on behalf of Apache Drill! > > Ellen F. > > On Sun, Aug 30, 2015 at 9:06 PM, Jacques Nadeau > wrote: > > > The Apache Drill PMC is very proud to announce Haki

Re: Announcing new committer: Abdel Hakim Deneche

Hakim, Congratulations! And thanks for all your efforts on behalf of Apache Drill! Ellen F. On Sun, Aug 30, 2015 at 9:06 PM, Jacques Nadeau wrote: > The Apache Drill PMC is very proud to announce Hakim Deneche as a new > committer. > > Since he arrived, he has been a pleasure to work with. No

[jira] [Created] (DRILL-3725) Add HTTPS support for Drill web interface

Venki Korukanti created DRILL-3725: -- Summary: Add HTTPS support for Drill web interface Key: DRILL-3725 URL: https://issues.apache.org/jira/browse/DRILL-3725 Project: Apache Drill Issue Type

DRILL-3548 - plug-in-exploration--related doc.

Jacques, Since you seem to be addressing plug-in--related documentation now, could you please review my patch for DRILL-3548 in Pull Request #118 , or at least assimilate the plug-in--related parts of

Re: Unforking Calcite...

Hackathon seems to be a brilliant idea. Definitely, it will help a lot if we could get Julian to join and help to review / work through these issues. On Mon, Aug 31, 2015 at 9:33 AM, Jacques Nadeau wrote: > I'd like to propose that we do a Hackathon to try to close up the gap. > Hopefully, we

Re: Unforking Calcite...

I'd like to propose that we do a Hackathon to try to close up the gap. Hopefully, we could get Julian to join and work through these issues with him. I'm sure that there is a way that we can either merge our patches (with different options) or change them to use new extension points. It is bad for

Re: Announcing new committer: Abdel Hakim Deneche

Congratulations :D > On Aug 31, 2015, at 8:18 AM, Abdel Hakim Deneche > wrote: > > Thanks everyone =D > > On Mon, Aug 31, 2015 at 5:29 AM, Edmon Begoli wrote: > >> Congrats Hakim! >> Hoping to follow your path. This is a brilliant idea and a project to be >> working on. >> >> On Mon, Aug 31

Re: Announcing new committer: Abdel Hakim Deneche

Thanks everyone =D On Mon, Aug 31, 2015 at 5:29 AM, Edmon Begoli wrote: > Congrats Hakim! > Hoping to follow your path. This is a brilliant idea and a project to be > working on. > > On Mon, Aug 31, 2015 at 12:54 AM, Mehant Baid > wrote: > > > Congrats Hakim. > > > > > > On 8/30/15 9:29 PM, Hsu

Re: Announcing new committer: Abdel Hakim Deneche

Congrats Hakim! Hoping to follow your path. This is a brilliant idea and a project to be working on. On Mon, Aug 31, 2015 at 12:54 AM, Mehant Baid wrote: > Congrats Hakim. > > > On 8/30/15 9:29 PM, Hsuan Yi Chu wrote: > >> Congrats! >> >> On Sun, Aug 30, 2015 at 9:20 PM, Abhishek Girish < >> abh