Re: Drill query on empty directory fails

Hey Nikunj, This is a known issue and there are JIRAs filed for variants of it (empty file, empty directory, bad records, ...). To query specific file types, you could try using regex, but i don't think it works across nested directories. -Abhishek On Mon, Oct 12, 2015 at 7:44 AM, Nikunj

JSON Data with dot in keyname

I am getting some data that seems well formed, except for there are dots in some of the key names. Basically, something like ip.src seems to be making it fail with a "Field References must be singular names" I am using the MapR Packaged Drill 1.2. I can prove this out with {"hello":"goodbye",

Re: Drill and AWS Quicksight?

I did not try it yet, but if JDBC is supported, as it appears, I think it should work. We'll see about the performance. Edmon On Mon, Oct 12, 2015 at 8:06 AM, Jeroen van Dijk wrote: > Hi all, > > Maybe too early, but did someone already try out Drill together with

Re: Drill Logging

A follow-up, since I am using "runbit" The line below exists. It sources drill-config.sh which then sources drill-env.sh. In neither of these files do I see the variables DRILLBIT_LOG_PATH and DRILLBIT_QUERY_LOG_PATH so that may be the "file not found" error I am getting? If I set those via

Drill query on empty directory fails

I'm dynamically generating sql queries based on time range provided. My query module doesn't know if data exist on particular path or not. In case drill encounters empty directory it fails the scan showing parse error. Is it expected behavior? Shouldn't it result into no records found? Also Can I

Re: JSON Data with dot in keyname

Using the donuts.json file, I changed the first topping to top.ping and was able to query successfully using back ticks like this: select `top.ping`[3].type from >

Re: JSON Data with dot in keyname

Well of course, however, the json may not be easily read by a human, (pretty printed vs not) and might have lots of fields with nesting making it even harder to successfully read.. To explore the data, if we had it so we could select * from the file, then hone in on there, unless you are

Re: JSON Data with dot in keyname

That is correct, if this is failing in a select *, that is a bug. Can you please file a JIRA? On Mon, Oct 12, 2015 at 10:27 AM, Paul Ilechko wrote: > Well, if it's json then you do know the field names, they are right there > in the document > > On Mon, Oct 12, 2015 at

Re: JSON Data with dot in keyname

Interesting, so we can't use select * on those items. I think that should be a bug of some sort in that it's hard to explore the json data if you don't know the field names ahead of time. This is one of the situations where having the select * works really well for learning the data. (Am I

Re: JSON Data with dot in keyname

Kristine, if you have that handy, with the first topping to top.ping, can you try select * just to validate that it's failing like the data I am seeing fail? Once I get confirmation, I'll file a JIRA. On Mon, Oct 12, 2015 at 12:12 PM, Kristine Hahn wrote: > Using the

Re: JSON Data with dot in keyname

I've created https://issues.apache.org/jira/browse/DRILL-3922 for this issue. I put in notes to reproduce, and why it's important (data exploration). Could someone please take a quick look at it to ensure that something I may have done in creating the JIRA doesn't affect the chances of it getting

Re: JSON Data with dot in keyname

lgtm Kristine Hahn Sr. Technical Writer 415-497-8107 @krishahn skype:krishahn On Mon, Oct 12, 2015 at 10:45 AM, John Omernik wrote: > I've created https://issues.apache.org/jira/browse/DRILL-3922 for this > issue. I put in notes to reproduce, and why it's important (data >

Re: JSON Data with dot in keyname

I tried out using the example you shared. Saw the same failures. However, this worked for me: > select `hello.yoko`, yousayyes from dfs.`/Users/agirish/abc.json`; +-++ | hello.yoko | yousayyes | +-++ | goodbye | isayno |

Re: Convert from Array to String

John Omernik wrote: Is there an easy way in drill to convert from an ARRAY or MAP to a string representation of the same? I know this is an odd question, but I realized I just was spinning my wheels because I thought I was doing something wrong with JSON and KVGEN and FLATTEN but in reality, it

Re: Convert from Array to String

That seems to be what I am looking for, however, when I do convert_from(flatten(kvgen(allias.`column`)), 'json') I get an error "Missing function implementation (convert_fromjson, "map required") when I do "convert_to(flatten(kvgen()), 'json') then I get what appears to be hex encoded data...

Convert from Array to String

Is there an easy way in drill to convert from an ARRAY or MAP to a string representation of the same? I know this is an odd question, but I realized I just was spinning my wheels because I thought I was doing something wrong with JSON and KVGEN and FLATTEN but in reality, it was working fine, I

Re: Convert from Array to String

We don't implement casts on array or map, but we do have a convert function that will convert a complex structure to json. You can invoke it like this: convert_from( map_or_list_column_name, 'JSON') This will return the data serialized into JSON in a varchar column. On Mon, Oct 12, 2015 at

Re: Convert from Array to String

Yes Daniel - 2002 is Struct and 2003 is Array (I'm seeing both) This is not a Drill problem but a squierrel problem, and an annoying one at that, I've tried using the option to display unknown data types as strings, now I just rather than the "Error Unknown Type (2003 or 2002)" On Mon, Oct

Re: approximate count distinct?

This is something that has been talked about multiple times but no one has started work on it yet (as far as I know). Do you want to open a JIRA and maybe we can collaborate on getting something put together. There are probably a couple of dependent jiras that will need to be resolved but having