Hi Roger, I really appreciate these success stories and they way you narrate them. The images help to tell the story also.
The original format appears to be a fixed-length fielded text format. I'd be curious to see a a sample record or two of the original data. Does the unparse generate identical data (byte for byte) compared to the original data? Can you give us details on how the unparse works in this same narrative style? Thanks for sharing these! -Davin On 2023/04/14 17:09:42 Roger L Costello wrote: > Hi Folks, > I created a parser to parse air navigation data (air nav data is the data > that is loaded into the aircraft’s computer to enable it to fly the > aircraft). The parser is completely declarative, i.e., it has no code. The > parser is specified using the DFDL language, which is a simple extension of > XML Schema. Here is an excerpt of the DFDL schema that I wrote: > <xs:element name="GLS_Channel" type="validString" dfdl:lengthKind="explicit" > dfdl:length="5" /> > <xs:element name="Runway_Identifier" type="validString" > dfdl:lengthKind="explicit" dfdl:length="5" /> > <xs:element name="Station_Latitude" type="validString" > dfdl:lengthKind="explicit" dfdl:length="9" /> > <xs:element name="Station_Longitude" type="validString" > dfdl:lengthKind="explicit" dfdl:length="10" /> > Notice how declarative it is. It specifies “what” data fields are in the air > nav document, not “how” to parse its data fields. That, in my opinion, is a > huge benefit. > I fed the air nav file into the Apache open-source DFDL processor (Daffodil), > along with my DFDL schema. I instructed the DFDL processor to serialize the > parsed air nav data to XML: > [cid:[email protected]] > I could have instructed the DFDL processor to serialize the parsed air nav > data to JSON or to EXI (binary XML, i.e., super-compact XML) or to a number > of other formats. > The XML document that parsing produced is highly readable. Here is an excerpt > of the XML output: > <GLS_Channel>20abc</GLS_Channel> > <Runway_Identifier>RW25R</Runway_Identifier> > <Station_Latitude>N82091570</Station_Latitude> > <Station_Longitude>E156450340</Station_Longitude> > Notice how readable it is. And, of course, the XML document is highly > processable. The entire XML suite of technologies may be used to process it. > In addition to the advantages just listed, DFDL also has the advantage of > unparsing. Thus, I was also able to unparse the XML: > [cid:[email protected]] > Here are some interesting statistics: > > 1. Size of the AirNav.dat file is 375 MB > 2. Size of the XML file that DFDL parsing produced is 4.67 GB > 3. Recall I said that the DFDL processor can output many different > formats, one being EXI (binary XML). I did that and the resulting output is > nearly as small as the original air nav text file. The size of the EXI file > is 384 MB > /Roger > >
