Hi Owen,
Just to clarify. So in my parser as I am mapping fields from Avro to ORC
equivalent, any time I encounter a union type on the Avro side I map to
struct type on the ORC side and the N-1 one out of the N fields is to rid
of the null?
Best,
Ryan
On Fri, Jul 31, 2020 at 10:52 AM Owen O'Malle
Ryan,
I did just look at that code in Spark last week. The problem as you
correctly surmised is that Spark
doesn't have a uniontype. I think we probably need a fix that converts the
uniontype into a struct for
Spark. In such a translation, you would have fields for each variant of the
union and
I am writing ORC binaries in Java and they deserialize perfectly with the
Apache ORC jar on the docs that I've used to validate the data. The schemas
looks good, etc.
When reading this information via Spark, we are encountering failures - in
particular
mismatched input '<' expecting '>'(line 1, p
