Re: NullPointers in type conversions

Does this system option not work: ALTER SESSION SET `drill.exec.functions.cast_empty_string_to_null` = true; The reason the bug was marked INVALID is that SQL engines (not sure about the spec) don't allow casting from empty string to number. The system option above is supposed to allow changing t

Re: null values while querying json files

thank you, it's greatly appreciated. On Fri, Sep 18, 2015 at 10:24 PM, Julien Le Dem wrote: > I'm going to look into this next week. > Here is a JIRA to follow the issue, comment, suggest solutions: > https://issues.apache.org/jira/browse/DRILL-3806 > > On Fri, Sep 18, 2015 at 3:10 PM, Stefán Ba

Re: null values while querying json files

I'm going to look into this next week. Here is a JIRA to follow the issue, comment, suggest solutions: https://issues.apache.org/jira/browse/DRILL-3806 On Fri, Sep 18, 2015 at 3:10 PM, Stefán Baxter wrote: > Hi, > > I have nothing meaningful to add but I had share that this BigInt > assumption h

Re: null values while querying json files

Hi, I have nothing meaningful to add but I had share that this BigInt assumption has caused more grief than any other single feature in Drill. I would go so far as to say that the "type intolerance" and lack of "sensible conversion" is the biggest hurdle on the way to fulfilling the "eliminate E

Re: null values while querying json files

This is a limitation of really late appearing fields. Right now, depending on the situation, if a value doesn't show up in the first ~4k, we assume that the value is BigInt. I think a developer is working on improving this behavior right now. I'll ping them to see when we might have a fix. -- Jacq

Re: CTAS exception

This could be a bug. Please open a new Jira and add as much information as possible ? thanks On Fri, Sep 18, 2015 at 10:07 AM, Stefán Baxter wrote: > Hi, > > There are no schema changes. > > I can select * from the table just fine! > > Regards, > -Stefán > > On Fri, Sep 18, 2015 at 4:55 PM, Abh

Re: CTAS exception

Hi, There are no schema changes. I can select * from the table just fine! Regards, -Stefán On Fri, Sep 18, 2015 at 4:55 PM, Abhishek Girish wrote: > Are you sure there is no schema change occurring between records for a > column other than *occurred_at?**. *In your second query, since the on

Re: CTAS exception

Are you sure there is no schema change occurring between records for a column other than *occurred_at?**. *In your second query, since the only column being read is *occurred_at, *you may not be hitting the issue. First query being a select * would read all columns and may hit this schema change er

Re: CTAS exception

This fails: - select * from dfs.asa.* where occurred_at < '2015-09-18' order by occurred_at; This, oddly enough, does not fail: - select occurred_at from dfs.* where occurred_at < '2015-09-18' order by occurred_at; -ste On Fri, Sep 18, 2015 at 4:08 PM, Stefán Baxter wrote: > > Th

Re: CTAS exception

The failing select query: select * from dfs.* where occurred_at < '2015-09-18' order by occurred_at; -ste On Fri, Sep 18, 2015 at 4:02 PM, Stefán Baxter wrote: > Hi, > > Both statements select everything but the CTAS statement included a date > filter + date order. > > The date field is always

Re: CTAS exception

Hi, Both statements select everything but the CTAS statement included a date filter + date order. The date field is always the same (extended json format date ISO) and is always present so I can say, with 100% certainty, that there is no schema change involved. The select statement fails as well

Re: CTAS exception

This kind of errors usually happens when there is an unsupported schema change in the json files, but you should be able to reproduce the error with just a select statement. Can you share both queries you tried (the failing CTAS and the successful SELECT *) ? Thanks On Fri, Sep 18, 2015 at 5:38 A

CTAS exception

Hi, I have some json files that I want to transform to parquet. We have been doing this without any issues but this time around I get this exception: Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector. Expected vector class of org.apache.drill.exec.vector.NullableVarCharVec

Re: directory pruning and UDFs

Hi, The short back story is this: - We are serving multiple tenants with vastly different data volume and needs - there no such thing as fixed period segment sizes (to get to approx. volume per segment) - We do queries that combined information from historical and fresh (stream

Re: null values while querying json files

Hi Andries, I've already tried writing where and case statements but none of them worked unfortunately. It's still: DATA_READ ERROR: Error parsing JSON - You tried to write a VarChar type when you are using a ValueWriter of type NullableIntWriterImpl. When I try this on sqlline, I can confirm th