Re: Discuss: JSON and XML and Daffodil - same Infoset, same Query should create same rowset?

2023-10-08 Thread Mike Beckerle
Or this single query: with pcapDoc as (select PCAP from `infoset.json`), packets as (select flatten(pcapDoc.PCAP.Packet) as packet from pcapDoc), ipv4Headers as (select packets.packet.LinkLayer.Ethernet.NetworkLayer.IPv4.IPv4Header as hdr from packets), ipsrcs as (select

Re: Discuss: JSON and XML and Daffodil - same Infoset, same Query should create same rowset?

2023-10-08 Thread Mike Beckerle
Ok. It took some time but putting the infoset.json attachment into /tmp this SQL pulls out all the IP addresses from the PCAP data: use dfs.tmp; create or replace view pcapDoc as select PCAP from `infoset.json`; create or replace view packets as select flatten(pcapDoc.PCAP.Packet) as packet

Re: Discuss: JSON and XML and Daffodil - same Infoset, same Query should create same rowset?

2023-09-13 Thread Paul Rogers
Hi Mike, I believe I sent a detailed response to this. Did it get through? If not, I'll try sending it again... - Paul On Wed, Sep 13, 2023 at 6:44 AM Mike Beckerle wrote: > ... sound of crickets on a summer night . > > It would really help me if I could get a response to this inquiry, to

Re: Discuss: JSON and XML and Daffodil - same Infoset, same Query should create same rowset?

2023-09-13 Thread Mike Beckerle
... sound of crickets on a summer night . It would really help me if I could get a response to this inquiry, to help me better understand Drill. I do realize people are busy, and also this was originally sent Aug 25, which is the wrong time of year to get timely response to anything. Hence,

Re: Discuss: JSON and XML and Daffodil - same Infoset, same Query should create same rowset?

2023-08-25 Thread Paul Rogers
Hi Mike, You asked about how to work with nested data items. As noted in a previous email, this can be a bit tricky. Drill uses SQL, and SQL does not have good native support for structured data: it was designed in the 1970's for record oriented data (tuples). Several attempts were made to extend

Re: Discuss: JSON and XML and Daffodil - same Infoset, same Query should create same rowset?

2023-08-25 Thread Paul Rogers
Great progress, Mike! First, let's address the schema issue. As you've probably noticed, Drill's original notion was that data needed no schema: the data itself provides sufficient syntactic structure to let Drill infer schema. Also as you've noticed, this assumption turned out to be more