Re: Question on Representing DFDL/XSD choice data for Drill (Unions required?)

2023-10-08 Thread Mike Beckerle
Nevermind. I figured this out. Was due to 'properties' being a reserved keyword. I created a PR to fix the JSON doc on the drill site. On Sat, Oct 7, 2023 at 1:46 PM Mike Beckerle wrote: > Ok, after weeks of delay > > That helps a great deal. You flatten the array of maps into a table of >

Re: Question on Representing DFDL/XSD choice data for Drill (Unions required?)

2023-10-07 Thread Mike Beckerle
Ok, after weeks of delay That helps a great deal. You flatten the array of maps into a table of maps. I am confused still about when I must do square brackets versus dot notation: data['a'] vs. data.a The JSON documentation for Drill uses dot notation to reach into fields of a map. Ex: from

Re: Question on Representing DFDL/XSD choice data for Drill (Unions required?)

2023-09-18 Thread Charles Givre
Hi Mike, Let me answer your question with some queries: >>> select * from dfs.test.`record.json`; +--+ | record |

Re: Question on Representing DFDL/XSD choice data for Drill (Unions required?)

2023-09-13 Thread Paul Rogers
Hi Mike, Looks like you are wrestling with two separate issues. The first is how to read the encoded data that you showed. In Drill, each data format generally needs its own reader. Drill's reader operator provides all the plumbing needed to handle multiple format readers, pack data into vectors,

Question on Representing DFDL/XSD choice data for Drill (Unions required?)

2023-09-13 Thread Mike Beckerle
I'm thinking whether a first prototype of DFDL integration to Drill should just use JSON. But please consider this JSON: { "record": [ { "a": { "a1":5, "a2":6 } }, { "b": { "b1":55, "b2":66, "b3":77 } } { "a": { "a1":7, "a2":8 } }, { "b": { "b1":77, "b2":88, "b3":99 } } ] } It