Re: XML module question?

[William Michels via perl6-users]

Dear Bruce and David,

Thank you both for an excellent discussion of this issue! David, thank you
for your code example. Bruce, it is good to know the correct XML file
format and specification, in particular, the fact that "...the order of
attribute specifications in a start-tag or empty-element tag is not
significant" (Bruce's references below):

https://www.w3.org/TR/REC-xml/#sec-starttags
> Note that the order of attribute specifications
> in a start-tag or empty-element tag is not significant.
>
> https://stackoverflow.com/questions/57756005/writing-xml-attributes-and-namespace-declarations-in-a-specific-order
>
> https://community.adobe.com/t5/coldfusion/maintain-attribute-order-in-xml-with-a-structure/m-p/3838398
>

Bruce and David, as I look through my bash history I see I tried
XML::Writer::serialize, but kept encountering the error, "The XML tree must
have a single root node". Of course, this is using the method whereby I
create an @-sigiled array first, which Bruce states is unnecessary. But it
would be nice to know the correct way to convert an @-sigiled array into a
valid XML file:

raku -MXML::Writer -e 'my @reordered = do for lines()
{(.words[3..*],.words[0..2]).join(" ")}; say
XML::Writer.serialize(@reordered.flat);' ../lat_lon.txt
The XML tree must have a single root node

As a more general question, I'm still amazed at the effect a single pair of
parentheses can have on data output. Certainly there's something very basic
that I'm overlooking in Raku, probaby because I'm used to other languages
(e.g. R) where parens are primarily used to control precedence. Compare the
following two lines of code (and corresponding output), first one with
parens, second one without:

$ raku -MXML -e 'my @xml = do for lines() {make-xml("wpt",
(:lat(.words[5].subst(";")), :lon(.words[8])), \("time",.words[0..2]))};
.put for @xml;' ../lat_lon.txt
lat 41.858657 lon -91.34514211-06-2014 - 10:49:06PM
lat 42.864653 lon -92.34991411-06-2014 - 10:49:49PM
lat 43.874808 lon -93.35036411-06-2014 - 10:50:35PM
lat 44.885047 lon -94.35005811-06-2014 - 10:51:21PM
lat 45.895078 lon -95.34992011-06-2014 - 10:52:08PM
lat 46.905178 lon -96.34983711-06-2014 - 10:53:30PM
lat 47.910303 lon -97.35060611-06-2014 - 10:54:50PM

$ raku -MXML -e 'my @xml = do for lines() {make-xml("wpt",
:lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put
for @xml;' /lat_lon.txt
11-06-2014 -
10:49:06PM
11-06-2014 -
10:49:49PM
11-06-2014 -
10:50:35PM
11-06-2014 -
10:51:21PM
11-06-2014 -
10:52:08PM
11-06-2014 -
10:53:30PM
11-06-2014 -
10:54:50PM

Presuming this isn't an issue with the XML module per se, I may indeed be
one of those people who needs to use a Raku "slang" to curtail my
pluri-parenthetical tendencies. Something on the order of a 'strict' slang
that absolutely requires an explicit $(...) instead of the standard Raku
behavior wherein (...) understood by Raku to mean $(...).  Would that be a
difficult slang to implement?

Best Regards, Bill.

W. Michels, Ph.D.


On Fri, Jul 16, 2021 at 6:45 PM Bruce Gray 
wrote:

> David,
>
> Aha! I am very glad to be wrong!
>
> $ raku -MXML::Writer -ne 'print XML::Writer.serialize: :wpt[
> :lat(.words[5].subst(";")), :lon(.words[8]), :time[.words.head(3).join("
> ")] ];'  lat_lon.txt
> 11-06-2014 -
> 10:49:06PM
> 11-06-2014 -
> 10:49:49PM
> 11-06-2014 -
> 10:50:35PM
> 11-06-2014 -
> 10:51:21PM
> 11-06-2014 -
> 10:52:08PM
> 11-06-2014 -
> 10:53:30PM
> 11-06-2014 -
> 10:54:50PM
>
> I do note that XML::Writer does not show up in the search at
> https://modules.raku.org/t/XML ,
> and is very under-documented, so that is an opportunity for someone to
> contribute.
>
> --
> Hope this helps,
> Bruce Gray (Util of PerlMonks)
>
> > On Jul 16, 2021, at 2:53 PM, David Warring 
> wrote:
> >
> > The XML::Writer module side-steps any ordering issues by directly
> serializing data structures to xml:
> >
> > $ raku -M XML::Writer -e'say XML::Writer.serialize: :wpt[ :lat<42>,
> :long<69>, "blah" ]'
> > blah
> >
> > On Sat, Jul 17, 2021 at 4:48 AM Bruce Gray 
> wrote:
> > > On Jul 15, 2021, at 11:02 PM, William Michels via perl6-users <
> perl6-users@perl.org> wrote:
> >
> > (and I re-ordered):
> >
> > > Any help appreciated,
> > > Thanks, Bill.
> > > REFERENCE:  https://unix.stackexchange.com/q/166675/227738
> >
> >
> > > I'm trying to put an answer together for SO, and I keep running into
> the same difficulty. Given a CSV file,
> >
> > Note: input is a log file, not in CSV format.
> >
> > > how can one produce an XML file with latitude and longitude values
> properly quoted? Every time I get proper quoting, "lat" and "lon" are
> scrambled (i.e. sometimes "lat" first, sometimes "lat" second).
> > ...
> > > using the "make-xml()" function from the XML module, I end up
> scrambling "lat/lon”:
> >
> > You are using the correct XML module, and you are using it correctly.
> >
> > Similar to a Raku hash, the attributes in an XML start-tag are
> 

Re: XML module question?

[Bruce Gray]

David,

Aha! I am very glad to be wrong!

$ raku -MXML::Writer -ne 'print XML::Writer.serialize: :wpt[ 
:lat(.words[5].subst(";")), :lon(.words[8]), :time[.words.head(3).join(" ")] 
];'  lat_lon.txt
11-06-2014 - 10:49:06PM
11-06-2014 - 10:49:49PM
11-06-2014 - 10:50:35PM
11-06-2014 - 10:51:21PM
11-06-2014 - 10:52:08PM
11-06-2014 - 10:53:30PM
11-06-2014 - 10:54:50PM

I do note that XML::Writer does not show up in the search at 
https://modules.raku.org/t/XML ,
and is very under-documented, so that is an opportunity for someone to 
contribute.

-- 
Hope this helps,
Bruce Gray (Util of PerlMonks)

> On Jul 16, 2021, at 2:53 PM, David Warring  wrote:
> 
> The XML::Writer module side-steps any ordering issues by directly serializing 
> data structures to xml:
> 
> $ raku -M XML::Writer -e'say XML::Writer.serialize: :wpt[ :lat<42>, 
> :long<69>, "blah" ]'
> blah
> 
> On Sat, Jul 17, 2021 at 4:48 AM Bruce Gray  wrote:
> > On Jul 15, 2021, at 11:02 PM, William Michels via perl6-users 
> >  wrote:
> 
> (and I re-ordered):
> 
> > Any help appreciated,
> > Thanks, Bill.
> > REFERENCE:  https://unix.stackexchange.com/q/166675/227738
> 
> 
> > I'm trying to put an answer together for SO, and I keep running into the 
> > same difficulty. Given a CSV file,
> 
> Note: input is a log file, not in CSV format.
> 
> > how can one produce an XML file with latitude and longitude values properly 
> > quoted? Every time I get proper quoting, "lat" and "lon" are scrambled 
> > (i.e. sometimes "lat" first, sometimes "lat" second). 
> ...
> > using the "make-xml()" function from the XML module, I end up scrambling 
> > "lat/lon”:
> 
> You are using the correct XML module, and you are using it correctly.
> 
> Similar to a Raku hash, the attributes in an XML start-tag are *unordered*.
> While the inconsistant ordering may be displeasing to our human eyes, any 
> program reading XML should treat your current output identically to your 
> desired output.
> 
> In Raku-speak, these two hashes have the same contents:
> my %wpt1 = lat => 'A', lon => 'z';
> my %wpt2 = lon => 'z', lat => 'A’;
> 
> Designers of XML libraries are reluctant to add the ability to force an 
> ordering; it would lead to XML-reading programs making improper assumptions.
> Like many major XML libraries in other languages, I see no such ability in 
> the Raku XML module.
> See:
> https://www.w3.org/TR/REC-xml/#sec-starttags
> Note that the order of attribute specifications
> in a start-tag or empty-element tag is not significant.
> 
> https://stackoverflow.com/questions/57756005/writing-xml-attributes-and-namespace-declarations-in-a-specific-order
> 
> https://community.adobe.com/t5/coldfusion/maintain-attribute-order-in-xml-with-a-structure/m-p/3838398
> 
> > Conversely, if I manage to restrict "lat/lon" to the correct order, I lose 
> > the equals sign and proper (value) quotation.
> 
> 
> Unless you just hardcode a template and insert the lat/lon/datetime values 
> (never a good idea), you will have to live with the varying order.
> I will point out that the XML library also does not output the multi-line 
> indentation that the OP shows; it is irrelevant in XML.
> 
> 
> > $ raku -MXML -e 'my @xml = do for lines() {make-xml("wpt", 
> > :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put 
> > for @xml;’  lat_lon.txt
> 
> Minor critiques:
> That use of `@array = do for lines() {…}` is more commonly written as `@array 
> = lines().map: {…}`
> raku -MXML -e 'my @xml = lines().map: {make-xml("wpt", 
> :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put 
> for @xml;'  lat_lon.txt
> 
> Nothing is gained by creating an array, just to print each line of the array, 
> so your can just `put` each time you construct an XML line.
> raku -MXML -e 'for lines() { put make-xml("wpt", 
> :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2])) }'  
> lat_lon.txt
> 
> When all the output is per-line based on the input, the `-p` or `-n` flag 
> removes the need for `for lines`.
> raku -MXML -ne 'put make-xml("wpt", :lat(.words[5].subst(";")), 
> :lon(.words[8]), \("time",.words[0..2]))'  lat_lon.txt
> 
> 
> 
> > So far, I can correctly swap the date/time to the end:
> > 
> > $ raku -MXML -e 'my @reordered = do for lines() 
> > {(.words[3..*],.words[0..2])}; .put for @reordered;’  lat_lon.txt
> 
> I am sure that `-MXML` is leftover from other experiments.
> Removing `@reordered` and switching to `-n` like my critiques above, this 
> becomes:
> raku -ne 'put (.words[3..*],.words[0..2]);' lat_lon.txt
> To “wow” the SO crowd, you might note that @array[N..*],@array[0..N-1] has a 
> shortcut in Raku (.rotate), making this the tightest of all the solutions I 
> see so far:
> raku -ne 'put .words.rotate(3);'  lat_lon.txt
> 
> -- 
> Hope this helps,
> Bruce Gray (Util of PerlMonks)
> 

Re: XML module question?

[David Warring]

The XML::Writer  module side-steps any
ordering issues by directly serializing data structures to xml:

$ raku -M XML::Writer -e'say XML::Writer.serialize: :wpt[ :lat<42>,
:long<69>, "blah" ]'
blah

On Sat, Jul 17, 2021 at 4:48 AM Bruce Gray 
wrote:

> > On Jul 15, 2021, at 11:02 PM, William Michels via perl6-users <
> perl6-users@perl.org> wrote:
>
> (and I re-ordered):
>
> > Any help appreciated,
> > Thanks, Bill.
> > REFERENCE:  https://unix.stackexchange.com/q/166675/227738
>
>
> > I'm trying to put an answer together for SO, and I keep running into the
> same difficulty. Given a CSV file,
>
> Note: input is a log file, not in CSV format.
>
> > how can one produce an XML file with latitude and longitude values
> properly quoted? Every time I get proper quoting, "lat" and "lon" are
> scrambled (i.e. sometimes "lat" first, sometimes "lat" second).
> ...
> > using the "make-xml()" function from the XML module, I end up scrambling
> "lat/lon”:
>
> You are using the correct XML module, and you are using it correctly.
>
> Similar to a Raku hash, the attributes in an XML start-tag are *unordered*.
> While the inconsistant ordering may be displeasing to our human eyes, any
> program reading XML should treat your current output identically to your
> desired output.
>
> In Raku-speak, these two hashes have the same contents:
> my %wpt1 = lat => 'A', lon => 'z';
> my %wpt2 = lon => 'z', lat => 'A’;
>
> Designers of XML libraries are reluctant to add the ability to force an
> ordering; it would lead to XML-reading programs making improper assumptions.
> Like many major XML libraries in other languages, I see no such ability in
> the Raku XML module.
> See:
> https://www.w3.org/TR/REC-xml/#sec-starttags
> Note that the order of attribute specifications
> in a start-tag or empty-element tag is not significant.
>
> https://stackoverflow.com/questions/57756005/writing-xml-attributes-and-namespace-declarations-in-a-specific-order
>
> https://community.adobe.com/t5/coldfusion/maintain-attribute-order-in-xml-with-a-structure/m-p/3838398
>
> > Conversely, if I manage to restrict "lat/lon" to the correct order, I
> lose the equals sign and proper (value) quotation.
>
>
> Unless you just hardcode a template and insert the lat/lon/datetime values
> (never a good idea), you will have to live with the varying order.
> I will point out that the XML library also does not output the multi-line
> indentation that the OP shows; it is irrelevant in XML.
>
>
> > $ raku -MXML -e 'my @xml = do for lines() {make-xml("wpt",
> :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put
> for @xml;’  lat_lon.txt
>
> Minor critiques:
> That use of `@array = do for lines() {…}` is more commonly written as
> `@array = lines().map: {…}`
> raku -MXML -e 'my @xml = lines().map: {make-xml("wpt",
> :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put
> for @xml;'  lat_lon.txt
>
> Nothing is gained by creating an array, just to print each line of the
> array, so your can just `put` each time you construct an XML line.
> raku -MXML -e 'for lines() { put make-xml("wpt",
> :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2])) }'
> lat_lon.txt
>
> When all the output is per-line based on the input, the `-p` or `-n` flag
> removes the need for `for lines`.
> raku -MXML -ne 'put make-xml("wpt", :lat(.words[5].subst(";")),
> :lon(.words[8]), \("time",.words[0..2]))'  lat_lon.txt
>
>
>
> > So far, I can correctly swap the date/time to the end:
> >
> > $ raku -MXML -e 'my @reordered = do for lines()
> {(.words[3..*],.words[0..2])}; .put for @reordered;’  lat_lon.txt
>
> I am sure that `-MXML` is leftover from other experiments.
> Removing `@reordered` and switching to `-n` like my critiques above, this
> becomes:
> raku -ne 'put (.words[3..*],.words[0..2]);' lat_lon.txt
> To “wow” the SO crowd, you might note that @array[N..*],@array[0..N-1] has
> a shortcut in Raku (.rotate), making this the tightest of all the solutions
> I see so far:
> raku -ne 'put .words.rotate(3);'  lat_lon.txt
>
> --
> Hope this helps,
> Bruce Gray (Util of PerlMonks)
>
>

Re: XML module question?

[Bruce Gray]

> On Jul 15, 2021, at 11:02 PM, William Michels via perl6-users 
>  wrote:

(and I re-ordered):

> Any help appreciated,
> Thanks, Bill.
> REFERENCE:  https://unix.stackexchange.com/q/166675/227738


> I'm trying to put an answer together for SO, and I keep running into the same 
> difficulty. Given a CSV file,

Note: input is a log file, not in CSV format.

> how can one produce an XML file with latitude and longitude values properly 
> quoted? Every time I get proper quoting, "lat" and "lon" are scrambled (i.e. 
> sometimes "lat" first, sometimes "lat" second). 
...
> using the "make-xml()" function from the XML module, I end up scrambling 
> "lat/lon”:

You are using the correct XML module, and you are using it correctly.

Similar to a Raku hash, the attributes in an XML start-tag are *unordered*.
While the inconsistant ordering may be displeasing to our human eyes, any 
program reading XML should treat your current output identically to your 
desired output.

In Raku-speak, these two hashes have the same contents:
my %wpt1 = lat => 'A', lon => 'z';
my %wpt2 = lon => 'z', lat => 'A’;

Designers of XML libraries are reluctant to add the ability to force an 
ordering; it would lead to XML-reading programs making improper assumptions.
Like many major XML libraries in other languages, I see no such ability in the 
Raku XML module.
See:
https://www.w3.org/TR/REC-xml/#sec-starttags
Note that the order of attribute specifications
in a start-tag or empty-element tag is not significant.

https://stackoverflow.com/questions/57756005/writing-xml-attributes-and-namespace-declarations-in-a-specific-order

https://community.adobe.com/t5/coldfusion/maintain-attribute-order-in-xml-with-a-structure/m-p/3838398

> Conversely, if I manage to restrict "lat/lon" to the correct order, I lose 
> the equals sign and proper (value) quotation.


Unless you just hardcode a template and insert the lat/lon/datetime values 
(never a good idea), you will have to live with the varying order.
I will point out that the XML library also does not output the multi-line 
indentation that the OP shows; it is irrelevant in XML.


> $ raku -MXML -e 'my @xml = do for lines() {make-xml("wpt", 
> :lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put 
> for @xml;’  lat_lon.txt

Minor critiques:
That use of `@array = do for lines() {…}` is more commonly written as `@array = 
lines().map: {…}`
raku -MXML -e 'my @xml = lines().map: {make-xml("wpt", 
:lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put for 
@xml;'  lat_lon.txt

Nothing is gained by creating an array, just to print each line of the array, 
so your can just `put` each time you construct an XML line.
raku -MXML -e 'for lines() { put make-xml("wpt", 
:lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2])) }'  
lat_lon.txt

When all the output is per-line based on the input, the `-p` or `-n` flag 
removes the need for `for lines`.
raku -MXML -ne 'put make-xml("wpt", :lat(.words[5].subst(";")), 
:lon(.words[8]), \("time",.words[0..2]))'  lat_lon.txt



> So far, I can correctly swap the date/time to the end:
> 
> $ raku -MXML -e 'my @reordered = do for lines() 
> {(.words[3..*],.words[0..2])}; .put for @reordered;’  lat_lon.txt

I am sure that `-MXML` is leftover from other experiments.
Removing `@reordered` and switching to `-n` like my critiques above, this 
becomes:
raku -ne 'put (.words[3..*],.words[0..2]);' lat_lon.txt
To “wow” the SO crowd, you might note that @array[N..*],@array[0..N-1] has a 
shortcut in Raku (.rotate), making this the tightest of all the solutions I see 
so far:
raku -ne 'put .words.rotate(3);'  lat_lon.txt

-- 
Hope this helps,
Bruce Gray (Util of PerlMonks)

XML module question?

[William Michels via perl6-users]

Hi,

I'm trying to put an answer together for SO, and I keep running into the
same difficulty. Given a CSV file, how can one produce an XML file with
latitude and longitude values properly quoted? Every time I get proper
quoting, "lat" and "lon" are scrambled (i.e. sometimes "lat" first,
sometimes "lat" second). Conversely, if I manage to restrict "lat/lon" to
the correct order, I lose the equals sign and proper (value) quotation. The
input file is as below:

11-06-2014 - 10:49:06PM lat = 41.858657; lon = -91.345142
11-06-2014 - 10:49:49PM lat = 42.864653; lon = -92.349914
11-06-2014 - 10:50:35PM lat = 43.874808; lon = -93.350364
11-06-2014 - 10:51:21PM lat = 44.885047; lon = -94.350058
11-06-2014 - 10:52:08PM lat = 45.895078; lon = -95.349920
11-06-2014 - 10:53:30PM lat = 46.905178; lon = -96.349837
11-06-2014 - 10:54:50PM lat = 47.910303; lon = -97.350606

While the desired output gives lat, lon, and moves the date/time to the end
of the line:


11-06-2014 - 10:49:06PM


11-06-2014 - 10:49:49PM


11-06-2014 - 10:50:35PM


11-06-2014 - 10:51:21PM


11-06-2014 - 10:52:08PM


11-06-2014 - 10:53:30PM


11-06-2014 - 10:54:50PM



So far, I can correctly swap the date/time to the end:

$ raku -MXML -e 'my @reordered = do for lines()
{(.words[3..*],.words[0..2])}; .put for @reordered;'  lat_lon.txt
lat = 41.858657; lon = -91.345142 11-06-2014 - 10:49:06PM
lat = 42.864653; lon = -92.349914 11-06-2014 - 10:49:49PM
lat = 43.874808; lon = -93.350364 11-06-2014 - 10:50:35PM
lat = 44.885047; lon = -94.350058 11-06-2014 - 10:51:21PM
lat = 45.895078; lon = -95.349920 11-06-2014 - 10:52:08PM
lat = 46.905178; lon = -96.349837 11-06-2014 - 10:53:30PM
lat = 47.910303; lon = -97.350606 11-06-2014 - 10:54:50PM

But using the "make-xml()" function from the XML module, I end up
scrambling "lat/lon":

$ raku -MXML -e 'my @xml = do for lines() {make-xml("wpt",
:lat(.words[5].subst(";")), :lon(.words[8]), \("time",.words[0..2]))}; .put
for @xml;'  lat_lon.txt
11-06-2014 -
10:49:06PM
11-06-2014 -
10:49:49PM
11-06-2014 -
10:50:35PM
11-06-2014 -
10:51:21PM
11-06-2014 -
10:52:08PM
11-06-2014 -
10:53:30PM
11-06-2014 -
10:54:50PM

Any help appreciated,

Thanks, Bill.

REFERENCE:  https://unix.stackexchange.com/q/166675/227738