[jira] [Created] (DRILL-7039) logRegex plugin doesn't implicitly handle .gz files

John Omernik created DRILL-7039: --- Summary: logRegex plugin doesn't implicitly handle .gz files Key: DRILL-7039 URL: https://issues.apache.org/jira/browse/DRILL-7039 Project: Apache Drill Issue

Re: [DISCUSS] Add Hadoop Credentials API support for Drill S3 storage plugin

I think in general Drill should consider the issue of credentials across storage plugins. Credentials exist for S3, but also Mongo, JDBC, and others as they get added. This can be a pain to manage and leads to insecurely setup Drill clusters. One option may be to allow a generic integration with

Re: [DISCUSS] sqlline upgrade

Since we are "shipping" sqlline, can we include a sqline shell script that puts the command line arguments we need, and then include another that people can use if they want to use without our arguments? This way anyone who is used to just "sqlline" doesn't have to change, it's the "new"

[jira] [Created] (DRILL-6607) Index Out of Bounds Error in string_binary function

John Omernik created DRILL-6607: --- Summary: Index Out of Bounds Error in string_binary function Key: DRILL-6607 URL: https://issues.apache.org/jira/browse/DRILL-6607 Project: Apache Drill Issue

Re: Odd Error on Login with 1.13-SNAPSHOT

t; > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues. > apache.org_jira_browse_DRILL-2D5425=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg= > gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg= > eWbxbVf7ysp5Kkl8FpO63Em5a65AnSOP5kYU4KIwQTo= > WfamtfbUdrG9eS2v6VLz0FI80E-2a_YjfrjkuztQzME= > > Kind regards

Odd Error on Login with 1.13-SNAPSHOT

I am probably missing something minor here, but I am working with Ted Dunning on some PCAP plugin stuff, so I built his 1.13 SNAPSHOT, and when I try to login I see { "errorMessage" : "No configuration setting found for key 'drill.exec.http.auth'" } I am guessing that something was added

Re: PCAP Issues

I've been following this and and also now playing with them more. A couple of tidbits that I think are worth re-mentioning: 1. pcapng the ability to read this too will be handy. 2. The flags are really cool, and the work here is great. The other thing I'd like to consider for this is with

Re: Proposed Slack Channel for Drill Users & Devs

Not all of us have a required email address? :) If you have an *@apache.org , @dremio.com , @mapr.com , @maprtech.com , @utk.edu , @simba.com , or @sfu.ca *email address,

Re: Proposed Slack Channel for Drill Users & Devs

Yes Please! On Thu, Jan 4, 2018 at 11:36 AM, Robert Wu wrote: > Hi, > > I think someone created one a while back (under "drillers.slack.com"). > > Best regards, > > Rob > > -Original Message- > From: Charles Givre [mailto:cgi...@gmail.com] > Sent: Thursday, January

[jira] [Created] (DRILL-5692) Umbrella Issue to Improve Error Messages to Users in Apache Drill

John Omernik created DRILL-5692: --- Summary: Umbrella Issue to Improve Error Messages to Users in Apache Drill Key: DRILL-5692 URL: https://issues.apache.org/jira/browse/DRILL-5692 Project: Apache Drill

[jira] [Created] (DRILL-5471) Provide better documentation around Parquet, Options and Integration with Arrow

John Omernik created DRILL-5471: --- Summary: Provide better documentation around Parquet, Options and Integration with Arrow Key: DRILL-5471 URL: https://issues.apache.org/jira/browse/DRILL-5471 Project

[jira] [Created] (DRILL-5461) Provide Ability to add Comments to Views, and Columns within Views

John Omernik created DRILL-5461: --- Summary: Provide Ability to add Comments to Views, and Columns within Views Key: DRILL-5461 URL: https://issues.apache.org/jira/browse/DRILL-5461 Project: Apache Drill

Parquet, Arrow, and Drill Roadmap

Hey all - I posted this to both dev and user as I could mentally make the argument for both, Sorry if this is answered somewhere already. I know in the past, there have been discussions around using two different readers for Parquet, and performance gains/losses, issues. etc. Right now, the

[jira] [Created] (DRILL-5030) Drill SSL Docs have Bad Link to Oracle Website

John Omernik created DRILL-5030: --- Summary: Drill SSL Docs have Bad Link to Oracle Website Key: DRILL-5030 URL: https://issues.apache.org/jira/browse/DRILL-5030 Project: Apache Drill Issue Type

[jira] [Created] (DRILL-4758) Option for Lazy/Late Materialization of columns during query with Parquet

John Omernik created DRILL-4758: --- Summary: Option for Lazy/Late Materialization of columns during query with Parquet Key: DRILL-4758 URL: https://issues.apache.org/jira/browse/DRILL-4758 Project

Re: Implement "DROP TABLE IIF EXISTS" statement

I agree with Julian. If we can backtick quote Hive's if and have an option for Hive users, it would be nice. But Hive made a mess, and there is precedent for IF. This makes from a cluster administration perspective, and even being a Hive user, as long as I had an option (with backticks) to allow

[jira] [Created] (DRILL-4753) dir0 labels in views are inconsistent

John Omernik created DRILL-4753: --- Summary: dir0 labels in views are inconsistent Key: DRILL-4753 URL: https://issues.apache.org/jira/browse/DRILL-4753 Project: Apache Drill Issue Type: Bug

[jira] [Created] (DRILL-4744) Fully Qualified JDBC Plugin Tables return Table not Found via Rest API

John Omernik created DRILL-4744: --- Summary: Fully Qualified JDBC Plugin Tables return Table not Found via Rest API Key: DRILL-4744 URL: https://issues.apache.org/jira/browse/DRILL-4744 Project: Apache

Re: Dynamic UDFs support

Charles - I think would be pretty cool. It would add in some functionality we "lost" when we moved on from Hive (you could use transform with a python script to do some basic in and out). This would be a neat feature, but I think it would benefit from a design document outlining what it would

[jira] [Created] (DRILL-4723) GC Memory Allocation Issues

John Omernik created DRILL-4723: --- Summary: GC Memory Allocation Issues Key: DRILL-4723 URL: https://issues.apache.org/jira/browse/DRILL-4723 Project: Apache Drill Issue Type: Bug

Administrative Draining of Drill bits

Hey all, Paul and I have been in discussion on the draining of drill bits (quiescent mode) https://issues.apache.org/jira/browse/DRILL-4286 I'd really like draw folks attention to this issue. As an administrator, having this ability is key to high availability while using proper

Re: median, quantile

Julian, great point. With a proper design, uses could session variables or use the select with options so that one query wouldn't change the session wide settings. That seems promising as an idea. John On Mon, Jun 6, 2016 at 8:12 PM, Julian Hyde wrote: > I’ve thought for

[jira] [Created] (DRILL-4699) Add Description Column in sys.options

John Omernik created DRILL-4699: --- Summary: Add Description Column in sys.options Key: DRILL-4699 URL: https://issues.apache.org/jira/browse/DRILL-4699 Project: Apache Drill Issue Type

Re: [ANNOUNCE] New PMC Chair of Apache Drill

Congratz Parth, and thank you Jacques! On Wed, May 25, 2016 at 1:25 PM, Xiao Meng wrote: > Big congratulations, Parth! > > And thank you, Jacques, for the leadership and the tremendous contributions > to the community. > > Best, > > Xiao > > On Wed, May 25, 2016 at 8:35 AM,

[jira] [Created] (DRILL-4692) Column named user unresolvable

John Omernik created DRILL-4692: --- Summary: Column named user unresolvable Key: DRILL-4692 URL: https://issues.apache.org/jira/browse/DRILL-4692 Project: Apache Drill Issue Type: Bug

[jira] [Created] (DRILL-4619) Provide bootstrap-cluster-options.json functionality similar to bootstrap-storage-plugins

John Omernik created DRILL-4619: --- Summary: Provide bootstrap-cluster-options.json functionality similar to bootstrap-storage-plugins Key: DRILL-4619 URL: https://issues.apache.org/jira/browse/DRILL-4619

[jira] [Created] (DRILL-4598) Update Website to Mark Avro Support as Experimental

John Omernik created DRILL-4598: --- Summary: Update Website to Mark Avro Support as Experimental Key: DRILL-4598 URL: https://issues.apache.org/jira/browse/DRILL-4598 Project: Apache Drill Issue

Re: Can this scenario cause a query to hang ?

Abdel - I think I've seen this on a MapR cluster I run, especially on CTAS. For me, I have not brought it up because the cluster I am running on has some serious personal issues (like being hardware that's near 7 years old, its a test cluster) and given the "hard to reproduce" nature of the

Re: Hangout starting in 5 minutes

Sorry I missed this, anything exciting happen? On Tue, Apr 5, 2016 at 11:57 AM, Jason Altekruse wrote: > Anyone with an interest in Drill is welcome to attend to hear what is > happening in the Drill community. Feel free to ask questions or just listen > in. > >

[jira] [Created] (DRILL-4572) Column Alias Not Available in Subquery

John Omernik created DRILL-4572: --- Summary: Column Alias Not Available in Subquery Key: DRILL-4572 URL: https://issues.apache.org/jira/browse/DRILL-4572 Project: Apache Drill Issue Type: Bug

Re: Failure Behavior

this would add a > lot of complication and overhead to Drill. > > On Sat, Mar 26, 2016 at 10:03 AM, John Omernik <j...@omernik.com> wrote: > > > Thanks for the responses.. So, even if the drillbit that died wasn't the > > foreman the query would fail? Interesting...

Re: Failure Behavior

t...@maprtech.com> wrote: > > > As far as I know, there is no failure handling in Drill. The query dies. > > > > On Sat, Mar 26, 2016 at 7:52 AM, John Omernik <j...@omernik.com> wrote: > > > > > With distributed Drill, what is the expected/desired

Re: Drill on YARN

Paul - Great write-up. Your description of Llama and Yarn is both informative and troubling for a potential cluster administrator. Looking at this solution, it would appear that to use Yarn with Llama, the "citizen" in this case Drill would have to be an extremely good citizen and honor all

Re: REST API with Basic Auth

using Python as well. So I'll do what you suggested. > > Thanks! > > Jean-Claude > > > > > > On Wed, Mar 23, 2016 at 8:30 PM, John Omernik <j...@omernik.com> wrote: > > > >> Yes, to do authenticated requests, I create a session object with Python > >> requ

Re: Drill on YARN

us verify if that hunch is valid. > > Thanks, > > - Paul > > > > On Mar 23, 2016, at 5:49 AM, John Omernik <j...@omernik.com > <javascript:;>> wrote: > > > > Hey Paul and Jacques, Great discussion here. Paul, I believe we met a > week > &

Re: REST API with Basic Auth

Yes, to do authenticated requests, I create a session object with Python requests, post creds to the login page, get the session cookies, and them do the queries. I do not believe Drill supports just using basic auth. This as far as I can tell is actually a good thing because forces you to manage

Re: Drill on YARN

Hey Paul and Jacques, Great discussion here. Paul, I believe we met a week or two ago on a call. I have been running Drill successfully and powerfully (Multi-tenant etc) using Apache Mesos and Marathon. While I didn't write a framework for Drill in Mesos, Marathon does give some very nice

Re: Help in Drill

When you say you have your own in memory database, is it a distributed in memory database? Is it something you wrote? Is it a frame work running on something like Alluxio? If it's distributed, you could write a storage plugin for it, and allow Drill to query it in a distributed fashion. If it's

Re: Aggregation OutOfMemoryException

I've had some luck disabling multi-phase aggregations on some queries where memory was an issue. https://drill.apache.org/docs/guidelines-for-optimizing-aggregation/ After I try that, than I typically look at the hash aggregation as you have done:

Re: Apache Drill Send Query Through Rest API

For a simple "use" query, what I have done is enable authentication in Drill via https://drill.apache.org/docs/configuring-user-authentication/. With that, you can now create sessions in the REST API. Since you mention HTTP Requests, basically, you create a requests.session() object, authenticate,

Re: On improving project maintainence

I've been meaning to respond to this (Even have the spreadsheet still open) I want to volunteer to do more list responding. Not sure what I have for time commitment, but in general, I want to be more helpful to user questions etc. We could also start to evaluate some of the response we get and

Re: Time for the 1.6 Release

> Hi, > > Can you please (pretty please with sugar on top) add the tickets involved > in proper (Yes, please excuse the language) Avro support > > - Stefán > > On Wed, Mar 2, 2016 at 12:23 PM, John Omernik <j...@omernik.com> wrote: > > > I'd lik

Re: Time for the 1.6 Release

I'd like to request drill-2517 be added as a bandaid for the planning issues when there are lots of directories of parquet files. This issue is really hurting drill adoption for me in my org. Thanks, John On Tuesday, March 1, 2016, Edmon Begoli wrote: > May I please ask

[jira] [Created] (DRILL-4460) Provide feature that allows fall back to sort aggregation

John Omernik created DRILL-4460: --- Summary: Provide feature that allows fall back to sort aggregation Key: DRILL-4460 URL: https://issues.apache.org/jira/browse/DRILL-4460 Project: Apache Drill

[jira] [Created] (DRILL-4379) Unexpected Table Behavior with only one subdirectory vs. Many

John Omernik created DRILL-4379: --- Summary: Unexpected Table Behavior with only one subdirectory vs. Many Key: DRILL-4379 URL: https://issues.apache.org/jira/browse/DRILL-4379 Project: Apache Drill

[jira] [Created] (DRILL-4378) CONVERT_FROM in View results in table scan of MapR-DB and perhaps HBASE

John Omernik created DRILL-4378: --- Summary: CONVERT_FROM in View results in table scan of MapR-DB and perhaps HBASE Key: DRILL-4378 URL: https://issues.apache.org/jira/browse/DRILL-4378 Project: Apache

[jira] [Created] (DRILL-4284) Complex Data Causing Index Out of Bounds with UNION Type

John Omernik created DRILL-4284: --- Summary: Complex Data Causing Index Out of Bounds with UNION Type Key: DRILL-4284 URL: https://issues.apache.org/jira/browse/DRILL-4284 Project: Apache Drill

[jira] [Created] (DRILL-4143) REFRESH TABLE METADATA - Permission Issues with metadata files

John Omernik created DRILL-4143: --- Summary: REFRESH TABLE METADATA - Permission Issues with metadata files Key: DRILL-4143 URL: https://issues.apache.org/jira/browse/DRILL-4143 Project: Apache Drill

[jira] [Created] (DRILL-4130) Ability to set settings at Table or View level rather than SESSION or SYSTEM

John Omernik created DRILL-4130: --- Summary: Ability to set settings at Table or View level rather than SESSION or SYSTEM Key: DRILL-4130 URL: https://issues.apache.org/jira/browse/DRILL-4130 Project

[jira] [Created] (DRILL-4129) Ability to Secure Storage plugins

John Omernik created DRILL-4129: --- Summary: Ability to Secure Storage plugins Key: DRILL-4129 URL: https://issues.apache.org/jira/browse/DRILL-4129 Project: Apache Drill Issue Type: Improvement

[jira] [Created] (DRILL-3922) select * on json document with dot in key name results in failure

John Omernik created DRILL-3922: --- Summary: select * on json document with dot in key name results in failure Key: DRILL-3922 URL: https://issues.apache.org/jira/browse/DRILL-3922 Project: Apache Drill

[jira] [Created] (DRILL-3880) sqlline does not allow for a password prompt - security issue

John Omernik created DRILL-3880: --- Summary: sqlline does not allow for a password prompt - security issue Key: DRILL-3880 URL: https://issues.apache.org/jira/browse/DRILL-3880 Project: Apache Drill

[jira] [Created] (DRILL-3879) sqlline user authentication provides poor feedback on authentication

John Omernik created DRILL-3879: --- Summary: sqlline user authentication provides poor feedback on authentication Key: DRILL-3879 URL: https://issues.apache.org/jira/browse/DRILL-3879 Project: Apache

[jira] [Created] (DRILL-3698) Expose Show Files Command As SQL for sorting/filtering

John Omernik created DRILL-3698: --- Summary: Expose Show Files Command As SQL for sorting/filtering Key: DRILL-3698 URL: https://issues.apache.org/jira/browse/DRILL-3698 Project: Apache Drill