COUNT DISTINCT with HAVING causing a failure

2014-12-10 Thread Christopher Matta
Tableau is using the following query to get the distinct count of a measure: SELECT COUNT(DISTINCT `custview`.`age`) AS `ctd_age_ok` FROM `mfs.views`.`nestedclickview` `nestedclickview` INNER JOIN `mfs.views`.`custview` `custview` ON (`nestedclickview`.`cust_id` = `custview`.`cust_id`) HAVING (COU

Re: COUNT DISTINCT with HAVING causing a failure

2014-12-11 Thread Christopher Matta
ING COUNT(*) > 0 > 2. SELECT COUNT(distinct column2) FROM HAVING COUNT(distinct > column2) > 0 > > Aman > > On Wed, Dec 10, 2014 at 12:29 PM, Christopher Matta > wrote: > > > Tableau is using the following query to get the distinct count of a > &g

Re: Validate your JSON files

2014-12-11 Thread Christopher Matta
215-701-3146 On Tue, Nov 25, 2014 at 7:31 AM, Christopher Matta wrote: > Ted, I'll take a look! Thanks. > > Chris Matta > cma...@mapr.com > 215-701-3146 > > On Tue, Nov 25, 2014 at 6:27 AM, Ted Dunning > wrote: > >> Chris, >> >> Your tool could

Query failed: You tried to do a batch data read operation when you were in a state of STOP.

2015-01-02 Thread Christopher Matta
I got this error while trying to load a drill view of twitter data in Tableau. This is the query Tableau is running (run in sqlline to get this error): 1/1 SELECT SUM(1) AS `sum_Number_of_Records_ok` FROM `mfs.views`.`twitter_tweets` `twitter_tweets` HAVING (COUNT(1) > 0); Query failed:

Re: Filter out empty arrays in JSON

2015-01-21 Thread Christopher Matta
Trying that locally did not work for me (drill 0.7.0): 0: jdbc:drill:zk=local> select `id`, `name`, `members` from `Downloads/test.json` where repeated_count(`members`) > 0; Query failed: Query stopped., Failure while trying to materialize incoming schema. Errors: Error in expression at index -1

Re: Filter out empty arrays in JSON

2015-02-05 Thread Christopher Matta
>>>>> select t.entities.hashtags from `twitter.json` t limit 10; > > >>>>>>>> > > >>>>>>>> ++ > > >>>>>>>> | EXPR$0 | > > >>>>>>>> ++ > &

Re: Filter out empty arrays in JSON

2015-02-05 Thread Christopher Matta
;23","27"]} | > | {"text":"Cowboys","indices":["7","15"]} | > | {"text":"Cowboys","indices":["7","15"]} | > | {"text":"Ebay","indices&q

Re: Filter out empty arrays in JSON

2015-02-05 Thread Christopher Matta
;48","61"]} | > > | {"text":"vegas","indices":["107","113"]} | > > | {"text":"nfl","indices":["23","27"]} | > > | {"text":"Cowboys","indices&q

Memory Settings

2015-02-13 Thread Christopher Matta
I’m attempting to see if increased available memory to Drill has a positive effect on certain queries, but I’m having trouble determining if changed memory settings are being respected. After setting DRILL_MAX_DIRECT_MEMORY="8G" and DRILL_MAX_HEAP="4G" I restarted drill. Checking the *metrics* pag

Re: Drill & Adjunct Data Warehouse

2015-02-13 Thread Christopher Matta
The potential for a JDBC storage plugin has come up in discussions a lot lately and would be a very positive addition to the project. I would love to know if there's been any work on this, or if not how something like this could get bootstrapped. Chris Matta cma...@mapr.com 215-701-3146 On Fri, F

Re: Connecting to multiple S3 locations

2015-03-26 Thread Christopher Matta
You can specify multiple “file” type storage plugins. One for each S3 instance, then join them. My storage plugins have the auth keys in the “file” specification configuration (not super secure, I know): s3test1: { "type": "file", "enabled": true, "connection": "s3n://:@cmatta", "workspa

Trying to wrap my head around using WHERE with flatten

2015-03-27 Thread Christopher Matta
I’m using the yelp academic data set for my testing purposes. Say I want to list all businesses in the “Pets” category, this query doesn’t work: 0: jdbc:drill:zk=sen11:5181,sen12:5181> select `name`, flatten(`categories`) from maprfs.cmatta.`yelp/yelp_acade

Convert UTC to specific timezone?

2015-03-30 Thread Christopher Matta
I have a column that’s UTC, how can I convert this column to a specific timezone? > select to_timestamp(columns[13], '-MM-dd HH:mm:ss') as `datetime_utc` > from `weather/2012` limit 10; +--+ | datetime_utc | +--+ | 2012-02-01 05:54:00.0 | | 2012-02-01 06:54:00.0 | | 2

Re: Convert UTC to specific timezone?

2015-03-31 Thread Christopher Matta
p/date information is stored in UTC and no > > timezone information is maintained, so AFAIK there isn't really a way to > > convert dates/timestamp to a specific timezone. However if your input > data > > contains timezone information Drill should be able to parse the input

Querying OpenTDSB data stored in HBase

2015-04-21 Thread Christopher Matta
I’m trying to use Drill to query time-series data stored in OpenTSDB . The row keys are supposed to be byte array encoded according to this schema: http://opentsdb.net/docs/build/html/user_guide/backends/hbase.html When trying to do a simple CONVERT_FROM I get the following r

Re: Querying OpenTDSB data stored in HBase

2015-04-21 Thread Christopher Matta
unction would take in a VarBinary and return a > ComplexWriter. The code would then map the bytes to a logical structure > like this: > > { > metric: xxx, > time: xxx, > tags: [ > {key: xxx, value xxx}, > {key: xxx, value xxx} > ] > } > > On T

Re: Drill and Python

2015-04-21 Thread Christopher Matta
Charles, I have an iPyghon notebook using PyODBC: https://github.com/cjmatta/drill_ipython_notebook On Tuesday, April 21, 2015, Charles Givre wrote: > All, > Forgive the n00b question, but I've been trying to write a Python > script to interact with Apache Drill. I've installed the JayDeBeAPI,

Drill and Spark integration

2015-05-26 Thread Christopher Matta
Spark integration with Drill is mentioned in this blog post, however I can’t find a JIRA for this feature on either the Drill or Spark trackers. What’s the status on this? Is there a timeframe? Is anyone working on it? --Chris ​

Error on Drill documentation page

2015-05-29 Thread Christopher Matta
Seems like the link for "apache- drill-1.0.0.tar.gz" on this page: https://drill.apache.org/docs/using-jdbc-with-squirrel-on-windows/ should be pointing to the drill binary package: http://apache.osuosl.org/drill/drill-1.0.0/apache-drill-1.0.0.tar.gz, not the source which it's pointing to now. Lat

convert unix timestamp

2015-05-29 Thread Christopher Matta
I need to be able to convert a unix timestamp to a drill timestamp. I can use the from_unixtime function return what seems like a timestamp, or a string formatted correctly, however it seems like it doesn’t return it as a string or as a timestamp type: 0: jdbc:drill:zk=sen11:5181,sen12:5181> sele

convert unix timestamp

2015-05-29 Thread Christopher Matta
29, 2015 at 4:22 PM, Kristine Hahn > wrote: > I think TO_TIMESTAMP is what you're looking for. Please see > http://drill.apache.org/docs/data-type-conversion/#to_timestamp. > > Kristine Hahn > Sr. Technical Writer > 415-497-8107 @krishahn > > > On Fri, May

Drill authentication with pyodbc

2015-06-08 Thread Christopher Matta
Does anyone know what the expected key names are for userid and password for an ODBC connection? I was using pyodbc to connect to Drill pre-1.0 but now with authentication enabled I haven’t figured out how to do it. Relevant errors: conn = pyodbc.connect('Driver=/opt/mapr/drillodbc/lib/universal

Re: Drill authentication with pyodbc

2015-06-08 Thread Christopher Matta
this notebook help? > > > http://nbviewer.ipython.org/github/cjmatta/drill_ipython_notebook/blob/master/Twitter%20Drill%20Pandas.ipynb > > https://github.com/cjmatta/drill_ipython_notebook > > > > On 8 Jun 2015, at 22:20, Christopher Matta wrote: > > Does anyone know what

Re: from_unixtime in drill explorer/ODBC

2015-06-09 Thread Christopher Matta
The function from_unixtime() is actually a Hive UDF and drill doesn’t yet know how to deal with the 2 extra bytes returned from it. It’s better to use the Drill function to_timestamp(): select to_timestamp(1432912733) from `sys`.`version` Chris mattacma...@mapr.com 215-701-3146 On Tue, Jun 9, 20

Re: Drill authentication with pyodbc

2015-06-09 Thread Christopher Matta
Mon, Jun 8, 2015 at 9:29 PM, Christopher Matta wrote: > > > That's actually my notebook, which I'm trying to update to use with Drill > > authentication, yes I'm using the DSN (the first argument in the connect > > function). > > > > Chris Matta > > cma

Adding an interval column to a date

2015-06-12 Thread Christopher Matta
Just to illustrate, I have this data: 0: jdbc:drill:zk=sen11:5181,sen12:5181> select * from `datemath.json`; +--+-+ | datetime | offset | +--+-+ | 2015-06-11 17:12:20 | 5 | +--+-+ 1 row selected

timestamp string to epoch time

2015-06-15 Thread Christopher Matta
Is there a way to convert a timestamp string to unix time? Chris Matta cma...@mapr.com 215-701-3146

Re: timestamp string to epoch time

2015-06-15 Thread Christopher Matta
on that can handle millisec and not require the string > function. > > > select to_timestamp(1432912733), unix_timestamp(to_timestamp(1432912733)) > from `sys`.`version`; > > Error: SYSTEM ERROR: java.lang.IllegalArgumentException: Invalid format: > "2015-05-29 15:1

Re: Query Failed: An Error Occurred

2015-06-17 Thread Christopher Matta
Yea, leave the default string out when querying the hive default configuration. ​ Chris Matta cma...@mapr.com 215-701-3146 On Wed, Jun 17, 2015 at 9:33 AM, Leon Clayton wrote: > Just done this on the sandbox and it works fine. > > 0: jdbc:drill:> select count(1) from hive.orders; > +---

Re: Exception thrown when float field has integer precedent

2015-06-18 Thread Christopher Matta
Wait, what’s wrong with doing it this way? 0: jdbc:drill:zk=sen11:5181,sen12:5181> select cast(`float` as float), cast (`int` as int) from `floatint.json`; +-+-+ | EXPR$0 | EXPR$1 | +-+-+ | 1.0 | 1 | | 0.1 | 1 | +-+-+ ​ Chris

Re: JSON/Join/Dynamic schema : java.lang.IllegalStateException: Failure while reading vector.

2015-06-22 Thread Christopher Matta
Just to clarify, you run the *exact same query* once and it works, then you remove say the “cool” field from orders/222.json and put it in orders/111.json and the next time the same query returns that error? ​ Chris Matta cma...@mapr.com 215-701-3146 On Mon, Jun 22, 2015 at 9:59 AM, Tugdual Grall

Re: JSON/Join/Dynamic schema : java.lang.IllegalStateException: Failure while reading vector.

2015-06-22 Thread Christopher Matta
ds.java:852) at sqlline.Commands.sql(Commands.java:751) at sqlline.SqlLine.dispatch(SqlLine.java:738) at sqlline.SqlLine.begin(SqlLine.java:612) at sqlline.SqlLine.start(SqlLine.java:366) at sqlline.SqlLine.main(SqlLine.java:259) ​ Chris Matta cma...@mapr.com 215

Re: Querying Drill profiles (JSON)

2015-06-30 Thread Christopher Matta
It would seem that user is a protected word that Drill uses to return your username. Try specifying user without backticks on any dataset: 0: jdbc:drill:zk=sen11:5181,sen12:5181> select user from sys.version . . . . . . . . . . . . . . . . . . . > ; +-+ | user | +-+ | cmatta |

Re: Querying parquet files

2015-07-07 Thread Christopher Matta
You might also want to check out the new partitioned Parquet creation that was launched with 1.1.0: https://drill.apache.org/docs/partition-by-clause/ This would increase your read speed if your queries tend to use predicates. Chris Matta cma...@mapr.com 215-701-3146 On Tue, Jul 7, 2015 at 9:3

Re: Rrounding timestamps to nearest period interval

2015-07-15 Thread Christopher Matta
I did this last week with a query on the Drill profiles, this query will break down the number of queries in 5 minute increments: select x.`timestamp`, x.`user`, count(1) from ( select t.`user`, to_timestamp(((cast(t.`start` as bigint)/1000) - MOD((cast(t.`start` as bigint)/1000), 300)

Support for Hive CHAR type

2015-09-04 Thread Christopher Matta
It doesn’t look like Drill 1.1.0 supports the Hive CHAR type? In Hive: create table development.foo ( bad CHAR(10) ); And then in sqlline: > use `hive.development`; > select * from foo; Error: PARSE ERROR: Unsupported Hive data type CHAR. Following Hive data types are supported in Drill INFO

Re: Support for Hive CHAR type

2015-09-04 Thread Christopher Matta
don't > jump out at you. We need to spell it out using a Hive example, right? > > Kristine Hahn > Sr. Technical Writer > 415-497-8107 @krishahn skype:krishahn > > > On Fri, Sep 4, 2015 at 7:20 AM, Christopher Matta > wrote: > > > It doesn’t look like Drill

Re: Support for Hive CHAR type

2015-09-04 Thread Christopher Matta
According to this page: http://kristinehahn.github.io/drill/docs/hive-to-drill-data-type-mapping/ it looks like Drill should convert CHAR to VARCHAR, but it's not. On Friday, September 4, 2015, Christopher Matta wrote: > Does this mean there aren't plans to support this data typ

Re: xml files with Drill

2015-09-04 Thread Christopher Matta
People have been asking for this functionality since Drill existed, if it were as straightforward as everyone says I'd think we'd have it working by now. On Friday, September 4, 2015, Jim Scott wrote: > Drill does not support XML. > > I have talked with folks who have tremendous experience with

Re: NullPointers in type conversions

2015-09-10 Thread Christopher Matta
Does changing the query to something like this work?: SELECT CAST(CASE WHEN table.cf.myColumn IS NOT NULL THEN table.cf.myColumn ELSE NULL END as INTEGER) ... ​ Chris Matta cma...@mapr.com 215-701-3146 On Thu, Sep 10, 2015 at 12:49 PM, Nathaniel Auvil wrote: > i have a MaprDB table which has

Re: NullPointers in type conversions

2015-09-15 Thread Christopher Matta
ful the dev who > picks up the JIRA. > > On Thu, Sep 10, 2015 at 12:49 PM, Christopher Matta > wrote: > > > Does changing the query to something like this work?: > > > > SELECT CAST(CASE WHEN table.cf.myColumn IS NOT NULL THEN > > table.cf.myColumn ELSE NULL END as

Re: NullPointers in type conversions

2015-09-17 Thread Christopher Matta
pr.com 215-701-3146 On Tue, Sep 15, 2015 at 3:09 PM, Christopher Matta wrote: > Did a Jira get opened for this? If not, Nathaniel, can you provide me with > your query and a sample data set so I can open one? > > Chris Matta > cma...@mapr.com > 215-701-3146 > > On Thu

Re: NullPointers in type conversions

2015-09-21 Thread Christopher Matta
n you give > it a try? > > That being said, it seems like the original issue was a NPE not a NFE. That > definitely seems like something else. > > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Sep 17, 2015 at 10:53 AM, Christopher Matta > wrote:

JSON Using "keys" as values

2015-09-21 Thread Christopher Matta
This has come up a few times in the past, maybe this is an opportunity to figure out the best way to handle poorly created JSON records at scale? Without having to transform it? Chris Matta cma...@mapr.com 215-701-3146 On Mon, Sep 21, 2015 at 3:53 PM, John Omernik > wrote: > Sorry about that, p

Re: NullPointers in type conversions

2015-09-22 Thread Christopher Matta
d > (setting at session level) works. > > Thank you, > Sudheesh > > > > > > > Sent from my iPhone > > > >> On Sep 21, 2015, at 7:18 AM, Christopher Matta wrote: > >> > >> I’m not sure if it worked, the result look

Re: NullPointers in type conversions

2015-09-23 Thread Christopher Matta
-3146 On Tue, Sep 22, 2015 at 2:33 PM, Christopher Matta wrote: > Was the result I saw expected? Seems like I got back an empty string and > not a NULL. > > Chris Matta > cma...@mapr.com > 215-701-3146 > > On Tue, Sep 22, 2015 at 1:48 PM, Sudheesh Katkam > wrote: > &

Re: NullPointers in type conversions

2015-09-23 Thread Christopher Matta
proving the experience right now > to at least tell users when they set it at the session level that it is not > allowed. > > > > On Wed, Sep 23, 2015 at 6:01 AM, Christopher Matta > wrote: > > > Just bumping this, I’m sure other users are going to be running acro

Re: NullPointers in type conversions

2015-09-24 Thread Christopher Matta
h varchar. We were primarily focused on issues with lots of case > statements when querying text files when it was added. > > - Jason > > On Wed, Sep 23, 2015 at 1:49 PM, Christopher Matta > wrote: > > > This system setting was set, yet I still experience the &g

Re: Querying local filesystem directory: How do I access the filename?

2015-09-24 Thread Christopher Matta
Seems like this was brought up in DRILL-3425 and identified as not something that was in the design spec. Chris Matta cma...@mapr.com 215-701-3146 On Thu, Sep 24, 2015 at 2:57 PM, Tim Harper wrote: > So, if the files are in subdirectories, I DO

CSV with windows carriage return causes issues

2015-09-30 Thread Christopher Matta
I’ve created a very simple reproduction of an issue I’ve observed with files that have a carriage return (\r) instead of a line feed (\n) ending. My CSV file was created using notepad on Windows and looks like this when queried directly from drill: 0: jdbc:drill:zk=local> select * from dfs.`Users

Re: FLATTEN doesn't quite give me what I need

2015-09-30 Thread Christopher Matta
Have you tried using the flatten query as a sub-query? Like: SELECT x.`household`, x.`minutes`, x.`LDItem`.`upc`, x.`LDItem`.`item_flags` from (SELECT household, minutes, FLATTEN(T.lditem_array.LDItem) as `LDItem` FROM `dfs`.`root`.`./S/Beddo/spark/pig-xpdb` T limit 8) as x; ​ Chris Matta cma..

Re: Drill to S3 set up and core-site.xml

2015-10-01 Thread Christopher Matta
I believe that’s referring to the Hadoop core-site.xml file, if that’s the case then the documentation should be updated to specify it. ​ Chris Matta cma...@mapr.com 215-701-3146 On Wed, Sep 30, 2015 at 5:45 PM, Dan Blondowski < dan.blondow...@dhigroupinc.com> wrote: > Hi. > Trying to set up dri

Re: Stop Drill querying .tmp files

2015-10-13 Thread Christopher Matta
Drill respects a file *inclusion* pattern, so you could build a view sort of like: select * from dfs.workspace.`dirname/*.csv`; ​ Chris Matta cma...@mapr.com 215-701-3146 On Tue, Oct 13, 2015 at 5:09 AM, wrote: > FYI - by real time I mean data files which Flume has finished writing > to...so

ODBC Installers should have the versions in the filenames

2015-10-13 Thread Christopher Matta
I end up with multiple ODBC installers in my downloads directory and it's hard to figure out which version each is. Why isn't the release version in the installer file name? Chris Matta cma...@mapr.com 215-701-3146

Re: Stop Drill querying .tmp files

2015-10-13 Thread Christopher Matta
ace so that you > don't have to code each query with this in mind. > > > > -Original Message- > From: Christopher Matta [mailto:cma...@mapr.com ] > Sent: 13 October 2015 14:21 > To: user@drill.apache.org > Subject: Re: Stop Drill querying .tmp files > > D

REST API Documentation?

2015-10-15 Thread Christopher Matta
I can't seem to find any reference documentation around the REST API on https://drill.apache.org/docs/, am I missing where it is? I'd like to know how you can pass a user along with the query when impersonation is enabled, also how would you authenticate when authentication is enabled? Chris Matt

Re: REST API Documentation?

2015-10-15 Thread Christopher Matta
skype:krishahn > > > On Thu, Oct 15, 2015 at 2:28 PM, Christopher Matta > wrote: > > > I can't seem to find any reference documentation around the REST API on > > https://drill.apache.org/docs/, am I missing where it is? > > > > I'd like to kn

Re: MapR Drill 1.2 Package

2015-10-21 Thread Christopher Matta
Good question, I'd like to know as well. I know that there were some JDBC fixes added to the official Drill 1.2 release, are those going to get back ported into the MapR release? Chris Matta cma...@mapr.com 215-701-3146 On Tue, Oct 20, 2015 at 2:36 PM, John Omernik wrote: > Hey quick questions

Re: CTAS over empty file throws NPE

2015-10-22 Thread Christopher Matta
I've run across this issue as well, in my opinion the CTAS should complete without complaint as you can create views on empty tables in RDBMSs. Chris Matta cma...@mapr.com 215-701-3146 On Thu, Oct 22, 2015 at 1:25 PM, Neeraja Rentachintala < nrentachint...@maprtech.com> wrote: > Hsuan > Is there

Re: Querying k/v pairs in a CSV column

2015-11-21 Thread Christopher Matta
Along the same lines I have a data set that's delimited by pipe with the last column further delimited by commas. It would be great to be able to flatten that last column. On Wednesday, October 7, 2015, Tugdual Grall wrote: > Hi, > > Can you clarify how to use our own format in CONVERT_FROM? > >

Re: Querying k/v pairs in a CSV column

2015-11-23 Thread Christopher Matta
---+---+ 10 rows selected (0.876 seconds) 0: jdbc:drill:> If you could come up with a regexp and string formatting to turn your var1=test var2=test var3=test column into valid JSON you could use CONVERT_FROM as Tomer suggested. ​ Chris Matta cma...@mapr.com 215

Re: Querying k/v pairs in a CSV column

2015-11-23 Thread Christopher Matta
ot;}', 'JSON') `values` from `kv.csv`) t; +-++ | id| EXPR$1 | +-+----+ | abc123 | {"key":"var1","value":"test"} | | abc123 | {"key"

Query elasticsearch data via Hive?

2015-11-30 Thread Christopher Matta
Hi, I can query data stored in Elasticsearch using Hive and the elasticsearch-hadoop code, but when trying to query that data using drill (via the hive storage plugin) I get a NullPointerException. Is there any way to use the work the ES team did t

Re: A field reference identifier must not have the form of a qualified name

2015-12-23 Thread Christopher Matta
Seems like Drill is explicitly checking for a period in the key and failing (from FieldReference.java ): private void checkSimpleString(CharSequence value) { if (value.to

Re: Sqlline Options

2015-12-28 Thread Christopher Matta
Hi Yogesh, You could achieve this by writing a wrapper script: #!/bin/env bash VARIABLE="testvalue" SQL="SELECT '${VARIABLE}' from sys.version;" sqlline -u jdbc:drill: -n cmatta -p <<< $SQL ​ Chris Matta cma...@mapr.com 215-701-3146 On Sat, Dec 26, 2015 at 10:38 PM, Yogesh Keshetty < yog

Re: SQL Lookup table - how to mimic?

2016-01-06 Thread Christopher Matta
Are you asking about a simple JOIN? Below I have a simple list of country_data.csv that I’m joining to an iso dataset: 0: jdbc:drill:> select * from maprfs.cmatta.`iso_3166_2_countries.csv` limit 10; +-+ | columns | +-+ | ["Sort Order","Common Name","Formal Name","Type","Sub Type"

Re: JSON File, Total numbers Record: 1

2016-01-13 Thread Christopher Matta
The apache mailing lists don't support image attachments, it would be better to attach text anyway since it's searchable. Chris Matta cma...@mapr.com 215-701-3146 On Wed, Jan 13, 2016 at 9:55 AM, Paolo Spanevello wrote: > Hi, > > thanks again Jason for your support. > > I'm trying as u suggest,

Changing Map schema in an embeded list

2016-01-20 Thread Christopher Matta
I have some POS log data that I’m converting from XML which has a nested list of items/actions added during the transaction. The issue I’m having is that when I try and flatten the list Drill complains: 0: jdbc:drill:> select flatten(t.RetailTransaction.POSLog.`ARTS:Transaction`.`ARTS:RetailTransa

Convert ISO 8601 string to timestamp

2016-02-03 Thread Christopher Matta
What’s the best way to convert this ISO 8601 timestamp to a drill timestamp type? Tried this: 0: jdbc:drill:> select to_timestamp('2016-01-08T15:00:15-05:00') from sys.version; Error: SYSTEM ERROR: DrillRuntimeException: Failure while materializing expression in constant expression evaluator [TO_

Re: Sqlline Tricks

2016-02-03 Thread Christopher Matta
The only way I know of getting sqlline to ask for a password is to run the !connect command after starting the shell: $ /opt/mapr/drill/drill-1.4.0/bin/sqlline apache drill 1.4.0 "a drill in the hand is better than two in the bush" sqlline> !connect jdbc:drill: scan complete in 427ms Enter usernam

Re: Sqlline Tricks

2016-02-03 Thread Christopher Matta
I passed the > > url > > > with -u, but without the username and password provided on the command > > > line, I get an auth error. If there truly is no way to ask for > > > username/password when providing auth string, I may open a JIRA on > that, > > I >

Re: Sqlline Tricks

2016-02-03 Thread Christopher Matta
<http://www.mapr.com/> <http://www.mapr.com/ < > > http://www.mapr.com/>> > > >>> On Feb 3, 2016, at 5:14 PM, John Omernik wrote: > > >>> > > >>> Ya, Andries, that's effectively what I did with my script, I passed > the &

Re: Sqlline Tricks

2016-02-04 Thread Christopher Matta
user and password. > > > > An attacker could see the file name on the [ps ax] output, but they > > wouldn't be able to read the props file. If that file is deleted shortly > > after starting sqlline, even better. > > > > So you can write the user and password

Re: Sqlline Tricks

2016-02-04 Thread Christopher Matta
/questions/3980668/how-to-get-a-password-from-a-shell-script-without-echoing > > Thus, I went that route for more portability. > > On Thu, Feb 4, 2016 at 8:54 AM, Christopher Matta wrote: > > > Looks good. You can streamline the no echo of the password by passing > read >

extractHeader in session variable?

2016-02-26 Thread Christopher Matta
Is it possible to set the extractHeader option for CSV/TSV in a session variable? Doing it on the format type is just too broad sometimes and I'd like to be able to set it based on the files I'm querying. Chris Matta cma...@mapr.com 215-701-3146

Re: extractHeader in session variable?

2016-02-29 Thread Christopher Matta
* FROM TABLE(dfs.`/path/to/CO.dat`(type => 'text', extractHeader => > true)); > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Fri, Feb 26, 2016 at 12:38 PM, Christopher Matta > wrote: > > > Is it possible to set the extractHeader option f

Re: extractHeader in session variable?

2016-02-29 Thread Christopher Matta
"","Clear","230","2012-01-01 08:54:00"] | | ["4:54 AM","37.9","34.0","86","30.11","10.0","SW","4.6","-","N/A","","Clear","230

Re: extractHeader in session variable?

2016-02-29 Thread Christopher Matta
46 On Mon, Feb 29, 2016 at 2:51 PM, Jason Altekruse wrote: > You need to specify the delimiter, it doesn't seem to default to comma as > the field delimiter. > > On Mon, Feb 29, 2016 at 11:46 AM, Christopher Matta > wrote: > > > Actually I spoke too soon. > >

Re: Strange error with hive null data

2016-03-18 Thread Christopher Matta
e a type of int? In that case what might be > failing here is your string literal 'null' failing to be implicitly cast to > match the column. > > Jason Altekruse > Software Engineer at Dremio > Apache Drill Committer > > On Fri, Mar 18, 2016 at 6:22 PM, Christopher M

Strange error with hive null data

2016-03-19 Thread Christopher Matta
I have some Hive data loaded with sqoop that looks like this: 0: jdbc:drill:> select curr_fiscal_week, load_dts from hive.dim.`YEAR_DATE_DIM`; +---++ | curr_fiscal_week |load_dts| +---++ | 12.0

Case-sensitive view names?

2016-03-21 Thread Christopher Matta
I’m experiencing case-sensitive view names, the drill documentation here: https://drill.apache.org/docs/lexical-structure/#case-sensitivity doesn't explicitly state weather view names are case-insensitive: 0: jdbc:drill:> select state, count(1) from `yelp/yelp_academic_dataset_business.json` group

Re: JDBC storage plugin fails

2016-03-24 Thread Christopher Matta
Scott, Could you paste the JSON you're using to define the MySQL storage plugin here? I've experienced that you get that error when either the JSON is invalid, or Drill can't find the class you defined. I've created a JIRA to improve this functionality: https://issues.apache.org/jira/browse/DRILL-

Support for UDTFs?

2016-04-29 Thread Christopher Matta
Are there plans to support UDTFs in Drill? I recall some discussions on a hangout last year about re-implementing the flatten function within a UDTF framework so that this framework could be made available to the community. Also Jacques mentions here that he'd look into putting together a proposal

Drill impersonation not respecting secondary group on view

2016-05-03 Thread Christopher Matta
Drill doesn’t seem to be respecting secondary group ownership with regards to a view, this view is owned by the analysts group: [mapr@ip-172-16-1-219 data]$ ls -l users_ssns.view.drill -rwxr-x--- 1 mapr analysts 1020 May 3 13:54 users_ssns.view.drill My user is a member of the analysts group (se

Re: Drill & Caravel

2016-05-13 Thread Christopher Matta
John, I've got some sample working code in this notebook for connecting to Drill via pyodbc: https://github.com/cjmatta/DrillPandasReddit I'd be interested in helping integrate this as well. Let us know if you push out a repo. -- Chris Matta 215-701-3146 ch...@mapr.com On Fri, May 13, 2016 at 2:

Re: Performance tuning

2016-05-17 Thread Christopher Matta
Leandro, I ran into a similar situation while building this demo: https://github.com/cjmatta/DrillPandasReddit/blob/master/Reddit%20Drill%20Pandas.ipynb I don't think Drill splits single JSON files the way it does for delimited one-record-per-line files, so that would explain why you're seeing sin