[
https://issues.apache.org/jira/browse/DRILL-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefán Baxter updated DRILL-4548:
---------------------------------
Summary: Avro fails on schema changes even when selecting from a single
file (Union related) (was: Avro fails on schema changes even when selecting
from a single file)
> Avro fails on schema changes even when selecting from a single file (Union
> related)
> -----------------------------------------------------------------------------------
>
> Key: DRILL-4548
> URL: https://issues.apache.org/jira/browse/DRILL-4548
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Avro
> Affects Versions: 1.6.0, 1.7.0
> Reporter: Stefán Baxter
>
> Hi,
> I have reworked/refactored our Avro based logging system trying to make the
> whole Drill + Avro->Parquet experience a bit more agreeable.
> Long story short I'm getting this error when selecting form multiple Avro
> files even though these files share the EXCACT same schema:
> Error: UNSUPPORTED_OPERATION ERROR: Hash aggregate does not support schema
> changes
> Fragment 0:0
> [Error Id: 00d49aa2-5564-497e-a330-e852d5889beb on swift:31010]
> (state=,code=0)
> We are using union types but only to allow for null values as seems to be
> supported by drill as per this comment in the Drill code:
> // currently supporting only nullable union (optional fields) like ["null",
> "some-type"].
> This happens for a very simple group_by + count(*) query that only uses two
> fields in Avro and neither one of them uses a Union construct so and both of
> them contain string values in every case.
> I now think this has nothing to do with the union types since the query uses
> only simple string, unless there is a full schema validation done on the
> content of the files rather then the identical Avro schema embedded in both
> files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
