Re: Continued Avro Frustration

2016-04-01 Thread Stefán Baxter
Hi Jason, Thank you for writing this up, it's appreciated. First things first. We would be more than happy to help on these Avro related issues but the Drill code base is quite complex, with a fairly steep learning curve, and lately a lot of my time has been spent on dealing with the repercussion

Re: Continued Avro Frustration

2016-04-01 Thread Stefán Baxter
Julian, you can give this speech to someone that has not been trying to contribute. Contributions come in more forms than fixing bugs or building features, especially when dealing with complex project like Drill. My contribution have been: - UDF samples (MIT licenesed and available to all) - Lu

[GitHub] drill pull request: DRILL-4573: Zero copy LIKE, REGEXP_MATCHES, SU...

2016-04-01 Thread jcmcote
GitHub user jcmcote opened a pull request: https://github.com/apache/drill/pull/458 DRILL-4573: Zero copy LIKE, REGEXP_MATCHES, SUBSTR All the functions using the java.util.regex.Matcher are currently creating Java string objects to pass into the matcher.reset(). However this cr

[GitHub] drill pull request: Drill 4573: Zero copy LIKE, REGEXP_MATCHES, SU...

2016-04-01 Thread jcmcote
Github user jcmcote closed the pull request at: https://github.com/apache/drill/pull/457 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] drill pull request: Drill 4573: Zero copy LIKE, REGEXP_MATCHES, SU...

2016-04-01 Thread jcmcote
GitHub user jcmcote opened a pull request: https://github.com/apache/drill/pull/457 Drill 4573: Zero copy LIKE, REGEXP_MATCHES, SUBSTR All the functions using the java.util.regex.Matcher are currently creating Java string objects to pass into the matcher.reset(). However this cr

[jira] [Created] (DRILL-4573) Zero copy LIKE, REGEXP_MATCHES, SUBSTR

2016-04-01 Thread jean-claude (JIRA)
jean-claude created DRILL-4573: -- Summary: Zero copy LIKE, REGEXP_MATCHES, SUBSTR Key: DRILL-4573 URL: https://issues.apache.org/jira/browse/DRILL-4573 Project: Apache Drill Issue Type: Improveme

[GitHub] drill pull request: DRILL-3714: Query runs out of memory and remai...

2016-04-01 Thread jacques-n
Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/442#discussion_r58286360 --- Diff: exec/rpc/src/main/java/org/apache/drill/exec/rpc/RpcBus.java --- @@ -159,19 +159,15 @@ public ChannelClosedHandler(C clientConnection, Channel chan

[GitHub] drill pull request: DRILL-3714: Query runs out of memory and remai...

2016-04-01 Thread sudheeshkatkam
Github user sudheeshkatkam commented on a diff in the pull request: https://github.com/apache/drill/pull/442#discussion_r58285085 --- Diff: exec/rpc/src/main/java/org/apache/drill/exec/rpc/RpcBus.java --- @@ -159,19 +159,15 @@ public ChannelClosedHandler(C clientConnection, Channel

[jira] [Created] (DRILL-4572) Column Alias Not Available in Subquery

2016-04-01 Thread John Omernik (JIRA)
John Omernik created DRILL-4572: --- Summary: Column Alias Not Available in Subquery Key: DRILL-4572 URL: https://issues.apache.org/jira/browse/DRILL-4572 Project: Apache Drill Issue Type: Bug

[GitHub] drill pull request: DRILL-4546: Only generate one zip archive when...

2016-04-01 Thread jacques-n
Github user jacques-n commented on the pull request: https://github.com/apache/drill/pull/449#issuecomment-204617555 Can you confirm that the release artifacts do not change between applying this and not applying it? --- If your project is set up for it, you can reply to this email a

Re: Continued Avro Frustration

2016-04-01 Thread Julian Hyde
Stefan, I wanted to chime in. I don’t think Jacques was out of line. I understand your frustration, but the project does not owe you anything. The only surefire way to get a feature into the project is to contribute it yourself AND to work through the process of getting it accepted. If you co

Re: adding int96 datatype in Drill

2016-04-01 Thread Ted Dunning
Or add a cast on the Hive side. On Fri, Apr 1, 2016 at 10:22 AM, Jacques Nadeau wrote: > It sounds like Hive doesn't work with the full Parquet spec. Why don't we > just fix that? > > Let's add Hive handling so that an annotated INT64 date works. > > -- > Jacques Nadeau > CTO and Co-Founder, D

Re: [DISCUSS] Nonblocking RPC

2016-04-01 Thread Jacques Nadeau
I think you're going to really have to break the encapsulation model to accomplish this in the RPC layer. What about updating the serialized executor for those situations to do a resubmission rather than blocking operation? Basically, it seems like we want a two phase termination: request terminat

[DISCUSS] Nonblocking RPC

2016-04-01 Thread Sudheesh Katkam
Hey y’all, There are some blocking requests that could make an event loop *await uninterruptibly*. At this point, the Drillbit might seem unresponsive. This is worsened if the the event loop is not unblocked (due to a bug), which requires a Drillbit restart. Although Drill supports *offloading fro

[GitHub] drill pull request: DRILL-4551: Implement new functions (cot, rege...

2016-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/452 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: Continued Avro Frustration

2016-04-01 Thread Parth Chandra
+1 on marking Avro experimental. @Stefan, we have been trying to help you as much as our time permits. I know that I held up the 1.6 release while Jason fixed the issues that you brought up. As was said earlier, this is personal time we are spending to help users in the community, so providing an

Re: Continued Avro Frustration

2016-04-01 Thread Stefán Baxter
To give credit where it is due. MapR seems to be willing to provide us with paid support even though we are not using their platform and we welcome that but we are still waiting for a followup email from the people Ted sent an introduction to. If that is a misunderstanding on my part, and they ar

Re: Continued Avro Frustration

2016-04-01 Thread Stefán Baxter
Anybody wishing to verify this is welcome to search this email archives for my email address/name and find the countless of help request regarding contribution. The Lucene plugin is an example where I tried to contribute but pointer/documentation requests were unanswered and pull requests were ign

Re: Continued Avro Frustration

2016-04-01 Thread Stefán Baxter
Jacques, You are way out of line here! Let me remind you that I have volunteered resources, mine and other, on some of these and asked for pointers several times, to NO AVAIL! It's not as if the Drill project is easy to wrap ones head around or test (this latter has improved a bit). I have also

Re: Continued Avro Frustration

2016-04-01 Thread Jacques Nadeau
Stefan, It makes sense to me to mark the Avro plugin experimental. Clearly, there are bugs. I also want to note your requirements and expectations haven't always been in alignment with what the Avro plugin developers built/envisioned (especially around schemas). As part of trying to address these

Re: adding int96 datatype in Drill

2016-04-01 Thread Jacques Nadeau
It sounds like Hive doesn't work with the full Parquet spec. Why don't we just fix that? Let's add Hive handling so that an annotated INT64 date works. -- Jacques Nadeau CTO and Co-Founder, Dremio On Fri, Apr 1, 2016 at 4:33 AM, Arina Yelchiyeva wrote: > Hi all! > > There is business requireme

[GitHub] drill pull request: DRILL-4459: SchemaChangeException while queryi...

2016-04-01 Thread vkorukanti
Github user vkorukanti commented on the pull request: https://github.com/apache/drill/pull/431#issuecomment-204471963 LGTM, +1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] drill pull request: DRILL-4523: Disallow using loopback address in...

2016-04-01 Thread sudheeshkatkam
Github user sudheeshkatkam commented on a diff in the pull request: https://github.com/apache/drill/pull/445#discussion_r58234800 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/service/ServiceEngine.java --- @@ -177,6 +182,18 @@ public void run() { });

[GitHub] drill pull request: DRILL-4459: SchemaChangeException while queryi...

2016-04-01 Thread vdiravka
Github user vdiravka commented on the pull request: https://github.com/apache/drill/pull/431#issuecomment-204466782 @vkorukanti Could you please review this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] drill pull request: DRILL-2100: Added deleting temporary spill dir...

2016-04-01 Thread vdiravka
Github user vdiravka commented on the pull request: https://github.com/apache/drill/pull/454#issuecomment-204436770 I made changes due to your suggestions. Also I deleted spaces in one level directory. Is it better? `tmp/drill/spill/29014cb9-7eda-f4d1-f216-ce1baf83f58c_majorfr

Re: Continued Avro Frustration

2016-04-01 Thread John Omernik
Stefán - I don't think you are being unreasonable. I think the topics you've brought up are valid, and have seriously tempered my willingness to use avro more in my organization. Since we haven't committed to it yet, I think that makes it less of a priority, thus I am less vocal, but I would ima

Continued Avro Frustration

2016-04-01 Thread Stefán Baxter
Hi, Is it at all possible that we are the only company trying to use Avro with Drill to some serious extent? We continue to coma across all sorts of embarrassing shortcomings like the one we are dealing with now where a schema change exception is thrown even when working with a single Avro file (

[GitHub] drill pull request: DRILL-4523: Disallow using loopback address in...

2016-04-01 Thread arina-ielchiieva
Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/445#discussion_r58196652 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/service/ServiceEngine.java --- @@ -177,6 +180,16 @@ public void run() { });

adding int96 datatype in Drill

2016-04-01 Thread Arina Yelchiyeva
Hi all! There is business requirement to read parquet files generated by Drill using Hive. Hive reads timestamps only if they are int96, while Drill writes timestamps as binary. I just wanted to know community opinion on adding int96 datatype. For example, if there will be session property that w

[jira] [Created] (DRILL-4571) Add link to the Drill log from the web UI

2016-04-01 Thread Arina Ielchiieva (JIRA)
Arina Ielchiieva created DRILL-4571: --- Summary: Add link to the Drill log from the web UI Key: DRILL-4571 URL: https://issues.apache.org/jira/browse/DRILL-4571 Project: Apache Drill Issue Ty

[GitHub] drill pull request: DRILL-4544: Improve error messages for REFRESH...

2016-04-01 Thread arina-ielchiieva
Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/448#discussion_r58179906 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/RefreshMetadataHandler.java --- @@ -64,6 +65,11 @@ public PhysicalP