Daffodil won't convert XML to JSON or JSON to XML. I believe you can find
tools for doing that. E.g., google search for 'Convert JSON to XML "open
source" java'
Using Daffodil you can go from some native format to XML or JSON, and back
from that to native format.
E.g, to output in JSON, just add the "-I json" to the command line:
daffodil -t parse -I json -s bsc.dfdl.xsd bsc5_one_record.dat
Similar for unparsing.
On Tue, Apr 5, 2022 at 4:09 PM Sethuraman, Venkataraman <
[email protected]> wrote:
> Will do.
>
> Thanks a lot.
>
> Meanwhile Mike, will I still be able to use Apache Daffodil for the
> following conversions as well?
>
> - Convert XML or JSON back to fixed length text file.
> - Convert XML file to JSON file.
> - Convert JSON file to XML file.
>
>
>
>
>
> *From:* Mike Beckerle <[email protected]>
> *Sent:* Tuesday, April 5, 2022 3:05 PM
> *To:* [email protected]
> *Subject:* [**EXTERNAL**] Re: Request help with creating annotations for
> fixed length to XML and vice versa and JSON to XML and vice versa
>
>
>
> Be aware this external email contains an attachment and/or link.
>
> Ensure the email and contents are expected. If there are concerns, please
> submit suspicious messages to the Cyber Intelligence Center using the
> Report Phishing button.
>
> Venkataraman,
>
>
>
> I have made a few changes to your schema, and it doesn't parse the data
> fully yet, but I think I got it to where you can more easily make progress.
>
>
>
> Suggestions are these:
>
>
>
> 1) factor out redundant properties into reusable named formats so that you
> don't have so many properties cluttering the schema
>
> 2) remove NL separator from records since the data doesn't seem to have
> them.
>
> 3) nilled elements - these are somewhat awkward in DFDL. You can't put the
> nil-oriented properties like dfdl:nilKind or dfdl:nilValue on a simpleType,
> only on an element. This limits the ability to hide them nicely when
> reusing.
>
> 4) the nillable="true" is an XML Schema thing, not DFDL property, so that
> also has to appear only on elements.
>
> 5) make a file with just one record in it and test that first.
>
>
>
> I can now get a trace of the parse via:
>
>
>
> daffodil -t parse -s bsc.dfdl.xsd bsc5_one_record.dat
>
>
>
> It fails, but you can see it is making its way through the fields one by
> one. So you should be able to figure out which field(s) length is incorrect
> and make progress.
>
>
>
> Good luck.
>
>
>
> parser: <Element name='file'><Sequence>...</Sequence></Element>
> bitPosition: 0
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 2033 3920 3838 4761 6d20 5065 6742 442b ␣39␣88Gam␣PegBD+
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <ex:file xmlns:ex="http://example.com"; xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance";></ex:file>
> diff:
> (no differences)
> ----------------------------------------------------------------- 1
> parser: <Element
> name='record'><Sequence>...............................................................................................................................................................</Sequence></Element>
> bitPosition: 0
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 2033 3920 3838 4761 6d20 5065 6742 442b ␣39␣88Gam␣PegBD+
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <ex:file xmlns:ex="http://example.com"; xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance";>
> <record></record>
> </ex:file>
> diff:
> (no differences)
> ----------------------------------------------------------------- 2
> parser: <Element
> name='HR'><ConvertTextCombinatorParser>......</ConvertTextCombinatorParser></Element>
> bitPosition: 0
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 2033 3920 3838 4761 6d20 5065 6742 442b ␣39␣88Gam␣PegBD+
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR></HR>
> </record>
> diff:
> (no differences)
> ----------------------------------------------------------------- 3
> parser: <HR parser='StringOfSpecifiedLengthParser' />
> bitPosition: 32
> data:
> ├───────┤ ├──┤
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 3838 4761 6d20 5065 6742 442b 3134 2020 88Gam␣PegBD+14␣␣
> 00000010: 2031 3420 2020 2038 3836 2039 3137 3831 ␣14␣␣␣␣886␣91781
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR></HR>
> </record>
> diff:
> bitLimit: (no value) -> 32
> bitPosition: 0 -> 32
> ----------------------------------------------------------------- 4
> parser: <ConvertTextNumberParser/>
> bitPosition: 32
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 2033 3920 3838 4761 6d20 5065 6742 442b ␣39␣88Gam␣PegBD+
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> </record>
> diff:
> bitLimit: 32 -> (no value)
> ----------------------------------------------------------------- 5
> parser: <Element name='Name'><SpecifiedLengthExplicitParser><Name
> parser='StringOfSpecifiedLengthParser'
> /></SpecifiedLengthExplicitParser></Element>
> bitPosition: 32
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 2033 3920 3838 4761 6d20 5065 6742 442b ␣39␣88Gam␣PegBD+
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name></Name>
> </record>
> diff:
> childIndex: 1 -> 2
> groupIndex: 1 -> 2
> ----------------------------------------------------------------- 6
> parser: <Name parser='StringOfSpecifiedLengthParser' />
> bitPosition: 112
> data:
> ├──────────────────────┤ ├────────┤
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 5065 6742 442b 3134 2020 2031 3420 2020 PegBD+14␣␣␣14␣␣␣
> 00000010: 2038 3836 2039 3137 3831 2020 2037 4927 ␣886␣91781␣␣␣7I'
> 00000020: 5720 2020 2020 2020 4761 6d20 5065 6720 W␣␣␣␣␣␣␣Gam␣Peg␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> </record>
> diff:
> bitLimit: (no value) -> 112
> bitPosition: 32 -> 112
> ----------------------------------------------------------------- 7
> parser: <Element name='DM'><SpecifiedLengthExplicitParser><DM
> parser='StringOfSpecifiedLengthParser'
> /></SpecifiedLengthExplicitParser></Element>
> bitPosition: 112
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 2033 3920 3838 4761 6d20 5065 6742 442b ␣39␣88Gam␣PegBD+
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM></DM>
> </record>
> diff:
> bitLimit: 112 -> (no value)
> childIndex: 2 -> 3
> groupIndex: 2 -> 3
> ----------------------------------------------------------------- 8
> parser: <DM parser='StringOfSpecifiedLengthParser' />
> bitPosition: 200
> data:
> ├──═ ├═
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000000: 6567 4244 2b31 3420 2020 3134 2020 2020 egBD+14␣␣␣14␣␣␣␣
> 00000010: 3838 3620 3931 3738 3120 2020 3749 2757 886␣91781␣␣␣7I'W
> 00000020: 2020 2020 2020 2047 616d 2050 6567 2020 ␣␣␣␣␣␣␣Gam␣Peg␣␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> </record>
> diff:
> bitLimit: (no value) -> 200
> bitPosition: 112 -> 200
> ----------------------------------------------------------------- 9
> parser: <Element
> name='HD'><SimpleNilOrValueParser>......</SimpleNilOrValueParser></Element>
> bitPosition: 200
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD></HD>
> </record>
> diff:
> bitLimit: 200 -> (no value)
> childIndex: 3 -> 4
> groupIndex: 3 -> 4
> ----------------------------------------------------------------- 10
> parser: <HD nilValue='List(%ES;)'/>
> bitPosition: 248
> data:
> ├─────────────┤ ├────┤
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000010: 3420 2020 2038 3836 2039 3137 3831 2020 4␣␣␣␣886␣91781␣␣
> 00000020: 2037 4927 5720 2020 2020 2020 4761 6d20 ␣7I'W␣␣␣␣␣␣␣Gam␣
> 00000030: 5065 6720 2030 3030 3830 352e 312b 3134 Peg␣␣000805.1+14
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD></HD>
> </record>
> diff:
> bitLimit: (no value) -> 248
> bitPosition: 200 -> 248
> failure:
> Parse Error: HD - Does not contain a nil literal!
> Schema context: HD Location line 61 column 18 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 31 limit(bytes) 31
> ----------------------------------------------------------------- 11
> parser: <HD parser='StringOfSpecifiedLengthParser' />
> bitPosition: 248
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD></HD>
> </record>
> diff:
> (no differences)
> ----------------------------------------------------------------- 12
> parser: <ConvertTextNumberParser/>
> bitPosition: 248
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> </record>
> diff:
> bitLimit: 248 -> (no value)
> ----------------------------------------------------------------- 13
> parser: <Element
> name='SAO'><SimpleNilOrValueParser>......</SimpleNilOrValueParser></Element>
> bitPosition: 248
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000010: 3134 2020 2031 3420 2020 2038 3836 2039 14␣␣␣14␣␣␣␣886␣9
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> 00000030: 2020 4761 6d20 5065 6720 2030 3030 3830 ␣␣Gam␣Peg␣␣00080
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO></SAO>
> </record>
> diff:
> childIndex: 4 -> 5
> groupIndex: 4 -> 5
> ----------------------------------------------------------------- 14
> parser: <SAO nilValue='List(%ES;)'/>
> bitPosition: 296
> data:
> ├═ ║
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000010: 3420 2020 2038 3836 2039 3137 3831 2020 4␣␣␣␣886␣91781␣␣
> 00000020: 2037 4927 5720 2020 2020 2020 4761 6d20 ␣7I'W␣␣␣␣␣␣␣Gam␣
> 00000030: 5065 6720 2030 3030 3830 352e 312b 3134 Peg␣␣000805.1+14
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO></SAO>
> </record>
> diff:
> bitLimit: (no value) -> 296
> bitPosition: 248 -> 296
> failure:
> Parse Error: SAO - Does not contain a nil literal!
> Schema context: SAO Location line 62 column 18 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 37 limit(bytes) 37
> ----------------------------------------------------------------- 15
> parser: <SAO parser='StringOfSpecifiedLengthParser' />
> bitPosition: 296
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO></SAO>
> </record>
> diff:
> (no differences)
> ----------------------------------------------------------------- 16
> parser: <ConvertTextNumberParser/>
> bitPosition: 296
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO>91781</SAO>
> </record>
> diff:
> bitLimit: 296 -> (no value)
> ----------------------------------------------------------------- 17
> parser: <Element
> name='FK5'><SimpleNilOrValueParser>......</SimpleNilOrValueParser></Element>
> bitPosition: 296
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> 00000030: 2020 4761 6d20 5065 6720 2030 3030 3830 ␣␣Gam␣Peg␣␣00080
> 00000040: 352e 312b 3134 3337 3430 3030 3133 3134 5.1+143740001314
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO>91781</SAO>
> <FK5></FK5>
> </record>
> diff:
> childIndex: 5 -> 6
> groupIndex: 5 -> 6
> ----------------------------------------------------------------- 18
> parser: <FK5 nilValue='List(%ES;)'/>
> bitPosition: 328
> data:
> ├────────┤ ├──┤
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000020: 2020 2037 4927 5720 2020 2020 2020 4761 ␣␣␣7I'W␣␣␣␣␣␣␣Ga
> 00000030: 6d20 5065 6720 2030 3030 3830 352e 312b m␣Peg␣␣000805.1+
> 00000040: 3134 3337 3430 3030 3133 3134 2e32 2b31 143740001314.2+1
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO>91781</SAO>
> <FK5></FK5>
> </record>
> diff:
> bitLimit: (no value) -> 328
> bitPosition: 296 -> 328
> failure:
> Parse Error: FK5 - Does not contain a nil literal!
> Schema context: FK5 Location line 63 column 18 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 41 limit(bytes) 41
> ----------------------------------------------------------------- 19
> parser: <FK5 parser='StringOfSpecifiedLengthParser' />
> bitPosition: 328
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO>91781</SAO>
> <FK5></FK5>
> </record>
> diff:
> (no differences)
> ----------------------------------------------------------------- 20
> parser: <ConvertTextNumberParser/>
> bitPosition: 328
> data:
> │ │
> 87654321 0011 2233 4455 6677 8899 aabb ccdd eeff 0123456789abcdef
> 00000020: 3137 3831 2020 2037 4927 5720 2020 2020 1781␣␣␣7I'W␣␣␣␣␣
> infoset:
> <?xml version="1.0" encoding="UTF-8"?>
> <record>
> <HR>39</HR>
> <Name>88Gam PegB</Name>
> <DM>D+14 14</DM>
> <HD>886</HD>
> <SAO>91781</SAO>
> <FK5></FK5>
> </record>
> diff:
> bitLimit: 328 -> (no value)
> failure:
> Parse Error: Unable to parse xs:int from text: 7I
> Schema context: FK5 Location line 63 column 18 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 41
> ----------------------------------------------------------------- 21
> [error] Parse Error: Failed to populate record[1]. Cause: Parse Error:
> Unable to parse xs:int from text: 7I
> Schema context: FK5 Location line 63 column 18 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 41
> Schema context: record Location line 55 column 10 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 41
> [error] Parse Error: Unable to parse xs:int from text: 7I
> Schema context: FK5 Location line 63 column 18 in
> file:/home/mbeckerle/Downloads/v7/bsc.dfdl.xsd
> Data location was preceding byte 41
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Apr 5, 2022 at 3:34 PM Mike Beckerle <[email protected]> wrote:
>
> Venkataraman,
>
>
>
> I started taking a look at this.
>
>
>
> A few observations. The data appears to be fixed length 196 characters,
> and I don't see any line-endings in the file.
>
> So your main "file" element is a separated sequence with separator %NL;,
> but that doesn't match the data file.
>
>
>
> I'm going to clean up your schema a bit and send back to you. A few quick
> things will make it much easier to work with.
>
>
>
>
>
> On Tue, Apr 5, 2022 at 11:46 AM Sethuraman, Venkataraman <
> [email protected]> wrote:
>
> Here is the attachment again.
>
> Please unzip it.
>
> Thanks
>
>
>
> *From:* Sethuraman, Venkataraman
> *Sent:* Tuesday, April 5, 2022 10:38 AM
> *To:* [email protected]; [email protected]
> *Subject:* Request help with creating annotations for fixed length to XML
> and vice versa and JSON to XML and vice versa
>
>
>
> Hi all
>
> I’m trying to use Apache Daffodil for the following scenarios:
>
> - Convert a fixed length text file to XML or JSON file.
> - Convert XML or JSON back to fixed length text file.
> - Convert XML file to JSON file.
> - Convert JSON file to XML file.
>
> Will it be possible to get some sample XSD files to implement the above
> scenarios?
>
> Also I tried to parse the attached sample files and their xsd from the url
> at .
>
> I got error messages that I have attached as well.
>
> Please let me know what corrections I need to make to the xsd for the
> sample to run against daffodil version 3.3.
>
> Thanks
> Venkat
>
>
