Re: How is dir0 inferred from the directory path

2015-10-19 Thread Jacques Nadeau
The first variable directory gets treated as a dirX starting point I believe. Doesn't seem like a bug to me. On Oct 19, 2015 9:56 AM, "rahul challapalli" wrote: > Drillers, > > The below result suggests that 'dir0' is inferred treating > '/drill/testdata/audits' as the root in the below query. I

Re: Apache Drill connecting to OpenTSDB hbase

2015-10-19 Thread Rajkumar Singh
Did you see any exception in drillbit logs and also what is the underlying Hbase version? > On 20-Oct-2015, at 10:30 AM, Tugdual Grall wrote: > > Hello, > > Have you tried to run a simple query from SqlLine or Web UI? (with a > limit just to be sure) > Any error? > > Tug > @tgrall > > On Mo

Re: Apache Drill connecting to OpenTSDB hbase

2015-10-19 Thread Tugdual Grall
Hello, Have you tried to run a simple query from SqlLine or Web UI? (with a limit just to be sure) Any error? Tug @tgrall On Mon, Oct 19, 2015 at 10:05 PM, De La Fuente, Pedro wrote: > Hi, > > > I am currently trying to connect to an OpenTSDB database with Apache Drill. > The test works succ

Apache Drill connecting to OpenTSDB hbase

2015-10-19 Thread De La Fuente, Pedro
Hi, I am currently trying to connect to an OpenTSDB database with Apache Drill. The test works successfully in the MAPR Drill ODBC Driver DSN Setup (Test button) after entering the connection type of Direct to DrillBit (host:port) and no authentication type. Next, when I open drill explore

Re: Empty Array in JSON

2015-10-19 Thread Ted Dunning
Cool. thanks for the nice comment. On Mon, Oct 19, 2015 at 4:07 PM, Kristine Hahn wrote: > Thanks for explaining what's going on, Ted! I'll use the info you've > provided to clarify things in the docs. > > Kristine Hahn > Sr. Technical Writer > 415-497-8107 @krishahn skype:krishahn > > > On Mon

Re: Empty Array in JSON

2015-10-19 Thread Kristine Hahn
Thanks for explaining what's going on, Ted! I'll use the info you've provided to clarify things in the docs. Kristine Hahn Sr. Technical Writer 415-497-8107 @krishahn skype:krishahn On Mon, Oct 19, 2015 at 4:03 PM, Ted Dunning wrote: > Kristine, > > I just tried working with that data file (sf

Re: Empty Array in JSON

2015-10-19 Thread Ted Dunning
Kristine, I just tried working with that data file (sf-city-logs-json/citylogs.json). First, I tried munging the file slightly. I deleted the outer wrapping of the file and then deleted the commas between records. Drill handles the resulting file wonderfully: 0: jdbc:drill:> select lots.geomet

Re: Empty Array in JSON

2015-10-19 Thread Kristine Hahn
Thanks for flagging the problem in the docs, guys. In some cases, removing an empty array can be used as a workaround to query a JSON file that otherwise fails, as shown in this example: http://drill.apache.org/docs/json-data-model/#example:-access-a-map-field-in-an-array Maybe this example, assu

Re: Empty Array in JSON

2015-10-19 Thread Andries Engelbrecht
Word of caution that Flatten may be better as only the first may be null. —Andries > On Oct 19, 2015, at 12:59 PM, John Omernik wrote: > > Awesome that worked. > > *The documentation should probably be updated on the array stuff, it's not > accurate as it pertains to empty arrays. > > > >

Re: Empty Array in JSON

2015-10-19 Thread John Omernik
Awesome that worked. *The documentation should probably be updated on the array stuff, it's not accurate as it pertains to empty arrays. On Mon, Oct 19, 2015 at 2:52 PM, Andries Engelbrecht < aengelbre...@maprtech.com> wrote: > Use where a[0] is not null > > 0: jdbc:drill:> select * from `./ar

Re: Empty Array in JSON

2015-10-19 Thread Andries Engelbrecht
Use where a[0] is not null 0: jdbc:drill:> select * from `./array.json`; +++ | b | a| +++ | 1 | [] | | 3 | [1,2] | +++ 2 rows selected (0.13 seconds) 0: jdbc:drill:> select * from `./array.json` where a[0] is not null; +++ | b | a

Re: Empty Array in JSON

2015-10-19 Thread John Omernik
Well you are in a sense confirming my suspicions that an empty array, as specified in the Docs as "error causing" doesn't actually cause an error, and that is expected. That is, empty arrays are not the big meanies that the docs make them out to be (my results are the same as your, that is, no erro

Re: Empty Array in JSON

2015-10-19 Thread Ted Dunning
John, I don't understand what you are seeing. Here is what I am seeing (and hopefully you can tell what I am missing). First the input is: $ cat x.json {"b":1, "a":[] } {"a":[1,2], "b":3} And then with this input, I get this: 0: jdbc:drill:> select * from dfs.tdunning.`x.json`; ++

Re: How is dir0 inferred from the directory path

2015-10-19 Thread Ted Dunning
I think that looks like a bug. On Mon, Oct 19, 2015 at 9:55 AM, rahul challapalli < challapallira...@gmail.com> wrote: > Drillers, > > The below result suggests that 'dir0' is inferred treating > '/drill/testdata/audits' as the root in the below query. Is this by design > that the first '*' get

How is dir0 inferred from the directory path

2015-10-19 Thread rahul challapalli
Drillers, The below result suggests that 'dir0' is inferred treating '/drill/testdata/audits' as the root in the below query. Is this by design that the first '*' gets treated as dir0? select * from dfs.`/drill/testdata/audits/*/audit/*.json` limit 1; +++--

Re: directory structure containing multiple file types

2015-10-19 Thread Aman Sinha
With regard to the last comment on directory based pruning, please watch DRILL-3759 (https://issues.apache.org/jira/browse/DRILL-3759). I don't have a timeline for it yet but hopefully in the next Drill release. Aman On Mon, Oct 19, 2015 at 3:50 AM, Dhruv Gohil wrote: > "What's needed in Dril

Empty Array in JSON

2015-10-19 Thread John Omernik
In https://drill.apache.org/docs/json-data-model/ there is a section that goes as laid out below. This is actually not occurring for me. I have a json dump from Mongo that has a field called tags where many records have "tags":[] and it's outputting that without error. (It just shows [] as the o

RE: version of running drillbit

2015-10-19 Thread Geercken, Uwe
Thanks Tugdual. -Original Message- From: Tugdual Grall [mailto:tugd...@gmail.com] Sent: Montag, 19. Oktober 2015 14:39 To: user Subject: Re: version of running drillbit Hello Not today, you have to look at the SqlLine on the node where the drillbit is running, or simply look at the pac

Re: version of running drillbit

2015-10-19 Thread Tugdual Grall
Hello Not today, you have to look at the SqlLine on the node where the drillbit is running, or simply look at the package name. We should soon have a fix for the following enhancement request https://issues.apache.org/jira/browse/DRILL-2726 This will allow you to give the version number of the d

version of running drillbit

2015-10-19 Thread Geercken, Uwe
Hello, Is there a way to find the information of the running drillbit(s) - in the logs and in the Web GUI? Tks, Uwe Uwe Geercken Application- & Projectmanager [http://galaxy.swissport.com/customtext/Swissport%20International%20Ltd.%20Station%20Zurich_sign.gif]

Re: directory structure containing multiple file types

2015-10-19 Thread Dhruv Gohil
"What's needed in Drill to truly eliminate ETL" +1 but in another thread ;-) few 'hacks' we want to share there of our 'work rounds' related to various drill limitations on multi directory queries (99% of our workload) , e.g. avoiding empty directory failures, building queries with direc

RE: JDBC storage

2015-10-19 Thread Geercken, Uwe
Thank you Rajkumar - It did work. One point though: I called my storage "mysql". Querying: select * from mysql.dim_carrier does not work, although I have specified the dbname in the storage config. But when I include the dbname in the query it does work: select * from mysql.mydbname.dim_carrie

Re: configuring the jdbc storage plugin

2015-10-19 Thread Rajkumar Singh
Kristine- I tried by providing username/passwd in plugin configuration and its worked. curl --request POST --header "Content-Type: application/json" -data '{"name": "myplugin", "config": { "type": "jdbc", "driver": "com.mysql.jdbc.Driver", "url": "jdbc:mysql://10.10.71.14:3306/

Re: JDBC storage

2015-10-19 Thread Rajkumar Singh
Please try with the following configuration and make sure that you have mysql connector jar into the DRILL_HOME/jars/3rdparty { "type": "jdbc", "driver": "com.mysql.jdbc.Driver", "url": "jdbc:mysql://hostname:3306/dbname", "username": "root", "password": "password", "enabled": true }

Re: directory structure containing multiple file types

2015-10-19 Thread Stefán Baxter
Hi Ted, Your approach only works for a single directory, not a directory structure. I will create an improvement request later today. I would welcome a session on "What's needed in Drill to truly eliminate ETL" (Just an idea) Regards, -Stefan On Sun, Oct 18, 2015 at 10:30 PM, Stefán Baxter w