Hello everyone, I am using Avro to load and validate JSON documents. Mostly this works very well and it is straightforward to express the structure of my document using Avro schema. However, I have a few fields which can have "any" content. It is impossible to declare all possible structures in advance, and I can't use a union type of primitives because the fields may also contain complex types (nested lists/maps) and Avro doesn't allow named unions.
So far as I have been able to determine, this is impossible with standard Avro schema, so I am curious if anyone else has dealt with this problem and can suggest any workarounds. Currently my best (least bad) idea is to preprocess the JSON to pull out the "any" fields and store them on the side before handing the document to Avro for loading. This is awkward so I would love to hear if anyone has any other ideas. Thanks, Peter