Re: Avro - Schema is good - Schema validation is bad

Sigh of relief is premature. Nobody has committed to carrying this interpretation forward. On Mon, Dec 14, 2015 at 11:44 AM, Stefán Baxter wrote: > /me sighs of relief > > On Mon, Dec 14, 2015 at 7:28 PM, Ted Dunning > wrote: > > > Actually,

Avro - Schema is good - Schema validation is bad

Hi, I'm getting the following error when querying Avro files: Error: VALIDATION ERROR: From line 1, column 48 to line 1, column 57: Column 'some_col' not found in any table It's true that the field is in none of the tables I'm targeting, in that particular query, but that does not mean that it

Local File Storage

Hi, I am trying to create table from a log(local file system). table is getting created and accessible with low row count, same query runs successfully with higher row count but than table is not accessible. create table t296 as select DateFunc(serverTime) serverTime, sessionid, logininfoid,

rand() is not random by row

Hi, I found that rand() function does not reply random value. When I used it, all rows are same values through all rows. You can see this result with bellow query: select *, rand() from sys.boot limit 10 ver: 1.3.0 client: Web( drillhost:8047 ) environment: distributed on EMR Thanks. --

Re: Avro - Schema is good - Schema validation is bad

Well, at least I'm not alone here. I think it must be time to set some ground rules for these things and what it means to support evolving schema and what is needed to eliminate ETL. I trust that enforcing a strict schema "just because we think we can" must go against the principles of such

Re: Avro - Schema is good - Schema validation is bad

Hi, This simply can not be the desired behavior! This prevents from using a field from a changing schema with dir0 sub-selection (directory pruning) as the altered/full schema is never part of the query and it subsequently fails. Drill should, IMOP, never have rules that are dependent on the

Re: Avro - Schema is good - Schema validation is bad

Actually, even without multiple storage types, this could be radically confusing. If I have many avro files that are partitioned into directories, then queries that use the partitioning to limit the files that I see could include or exclude more recent files that have added a new field. That

Re: Avro - Schema is good - Schema validation is bad

/me sighs of relief On Mon, Dec 14, 2015 at 7:28 PM, Ted Dunning wrote: > Actually, even without multiple storage types, this could be radically > confusing. > > If I have many avro files that are partitioned into directories, then > queries that use the partitioning to

Drill Azure Blob Storage Plugin

Is there an Azure Blob Storage Plugin for Apache Drill? I'm looking for a solution that can be done without configuring Hadoop to access Azure blob (http://hadoop.apache.org/docs/r2.7.0/hadoop-azure/index.html). Thanks Kumiko