Re: [basex-talk] Constructing and querying map structures in a dynamic way
Well - but are such concerns related to using maps as the output? I do not think so, and therefore I suggest to avoid mixing specific how-to's with general challenges. By now you know the idiom how to construct maps dynamically, as you had already known how to construct node trees dynamically. I agree with Graydon - best to stick to specific questions. General questions are also welcome, but they should be phrased very carefully, in order to enable meaningful answers. A yardstick worth trying is to pose the question in the form "How can I ..."? or "How should one ..."? If you try, for example, to construct a question starting with "How can I" and then introducing referential integrity, you may find it difficult to come up with a question making real sense. Graydon wrote: "Understanding the whole comes from understanding an accumulation of parts and understanding the individual parts comes from making specific mistakes" Yes, but I would like to add something: PRINCIPLES. XQuery is a passionately principled language, and it is of prime importance to grasp the principles - a process that takes time, but one should be aware of it and welcome it. Most important is to understand the "everything-is-an-expression principle", which makes the language composable - expressions are defined in terms of operands which are expressions. Another key principle is the Effectvie Boolean Value, defined by a little mantra mapping value to boolean value. (With a couple of nasty pitfalls!) And, of course, the underlying data model - XDM - with its beautiful stream of consciousness "Information is a value. A value is a sequence of items. An item is a node, or an atom, or a mapping. A node has one of six kinds. An atom is a string plus a schema type. A mapping is a mapping of values to value, can be a map, or an array, or a function item." The firmer your understanding of the principles, the less your need to "try out" - you know. Kind regards,Hans-Jürgen Am Freitag, 27. Mai 2022, 15:38:12 MESZ hat Markus Elfring Folgendes geschrieben: > Gaps? First, to avoid a misunderstanding: whereas map keys must not be the > empty sequence, map values may. Special data sources might contain challenges for further clarification of the desired referential integrity. Regards, Markus
Re: [basex-talk] Encoding hassle ....
Oh yes thanks. Forgot to mention this. Forcing utf8 doesn't help.
Il ven 27 mag 2022, 19:26 Bridger Dyson-Smith ha
scritto:
> Marco -
> I'm sorry but I can only corroborate your findings, and that trying to
> force UTF-8 by adding the encoding parameter to the functions doesn't seem
> to help; e.g.
>
> ) ./bin/basex
> BaseX 9.7.1 [Standalone]
> Try 'help' to get more information.
> > xquery file:current-dir()
> /usr/home/bridger/bin/basex/
> Query executed in 886.62 ms.
> > xquery file:write-text("a1.txt", "°" || out:nl(), "UTF-8")
>
> Query executed in 4.32 ms.
> > xquery file:read-text("a1.txt")
> °
>
> Query executed in 1.99 ms.
> > xquery file:write-text("a2.txt", file:read-text("a1.txt", "UTF-8"),
> "UTF-8")
>
> Query executed in 1.83 ms.
> > xquery file:read-text("a2.txt")
> [file:io-error] Decoding error: xb0
> > xquery file:read-text("a2.txt", "UTF-8")
> [file:io-error] Decoding error: xb0
> > xquery file:read-text("a2.txt", "ISO-8859-1")
> °
>
> Query executed in 2.01 ms.
>
> On Fri, May 27, 2022 at 1:00 PM Marco Lettere wrote:
>
>> Dear all,
>>
>> after wrapping our heads around this for hours today, we don't know how
>> to get rid of this inconsistency. Thus I ask for help ...
>>
>> SSCE:
>>
>> BaseX 9.6.4 [Standalone]
>> Try 'help' to get more information.
>> > xquery file:write-text("a1.txt", "°" || out:nl()) (: Same with
>> codepoints-to-string(176) instead of "°" :)
>>
>> Query executed in 183.94 ms.
>> > xquery file:read-text("a1.txt")
>> °
>>
>> Query executed in 1.49 ms.
>> > xquery file:write-text("a2.txt", file:read-text("a1.txt"))
>> Query executed in 3.4 ms.
>>
>> > xquery file:read-text("a2.txt")
>> [file:io-error] Decoding error: xb0
>>
>> Testing the files with linux command-line tool "file", this is the output:
>>
>> > file a1.txt
>> a1.txt: Unicode text, UTF-8 text
>>
>> > file a2.txt
>> a2.txt: ISO-8859 text
>>
>> Reading the file after "copying" it seems to change the encoding. How is
>> this supposed to be handled?
>>
>> Regards,
>>
>> Marco.
>>
>>
Re: [basex-talk] Encoding hassle ....
Definitely looks like a bug. I’m currently on the road, but I’ll get to the
bottom of this once I’m back.
Bridger Dyson-Smith schrieb am Fr., 27. Mai 2022,
19:27:
> Marco -
> I'm sorry but I can only corroborate your findings, and that trying to
> force UTF-8 by adding the encoding parameter to the functions doesn't seem
> to help; e.g.
>
> ) ./bin/basex
> BaseX 9.7.1 [Standalone]
> Try 'help' to get more information.
> > xquery file:current-dir()
> /usr/home/bridger/bin/basex/
> Query executed in 886.62 ms.
> > xquery file:write-text("a1.txt", "°" || out:nl(), "UTF-8")
>
> Query executed in 4.32 ms.
> > xquery file:read-text("a1.txt")
> °
>
> Query executed in 1.99 ms.
> > xquery file:write-text("a2.txt", file:read-text("a1.txt", "UTF-8"),
> "UTF-8")
>
> Query executed in 1.83 ms.
> > xquery file:read-text("a2.txt")
> [file:io-error] Decoding error: xb0
> > xquery file:read-text("a2.txt", "UTF-8")
> [file:io-error] Decoding error: xb0
> > xquery file:read-text("a2.txt", "ISO-8859-1")
> °
>
> Query executed in 2.01 ms.
>
> On Fri, May 27, 2022 at 1:00 PM Marco Lettere wrote:
>
>> Dear all,
>>
>> after wrapping our heads around this for hours today, we don't know how
>> to get rid of this inconsistency. Thus I ask for help ...
>>
>> SSCE:
>>
>> BaseX 9.6.4 [Standalone]
>> Try 'help' to get more information.
>> > xquery file:write-text("a1.txt", "°" || out:nl()) (: Same with
>> codepoints-to-string(176) instead of "°" :)
>>
>> Query executed in 183.94 ms.
>> > xquery file:read-text("a1.txt")
>> °
>>
>> Query executed in 1.49 ms.
>> > xquery file:write-text("a2.txt", file:read-text("a1.txt"))
>> Query executed in 3.4 ms.
>>
>> > xquery file:read-text("a2.txt")
>> [file:io-error] Decoding error: xb0
>>
>> Testing the files with linux command-line tool "file", this is the output:
>>
>> > file a1.txt
>> a1.txt: Unicode text, UTF-8 text
>>
>> > file a2.txt
>> a2.txt: ISO-8859 text
>>
>> Reading the file after "copying" it seems to change the encoding. How is
>> this supposed to be handled?
>>
>> Regards,
>>
>> Marco.
>>
>>
Re: [basex-talk] Encoding hassle ....
Marco -
I'm sorry but I can only corroborate your findings, and that trying to
force UTF-8 by adding the encoding parameter to the functions doesn't seem
to help; e.g.
) ./bin/basex
BaseX 9.7.1 [Standalone]
Try 'help' to get more information.
> xquery file:current-dir()
/usr/home/bridger/bin/basex/
Query executed in 886.62 ms.
> xquery file:write-text("a1.txt", "°" || out:nl(), "UTF-8")
Query executed in 4.32 ms.
> xquery file:read-text("a1.txt")
°
Query executed in 1.99 ms.
> xquery file:write-text("a2.txt", file:read-text("a1.txt", "UTF-8"),
"UTF-8")
Query executed in 1.83 ms.
> xquery file:read-text("a2.txt")
[file:io-error] Decoding error: xb0
> xquery file:read-text("a2.txt", "UTF-8")
[file:io-error] Decoding error: xb0
> xquery file:read-text("a2.txt", "ISO-8859-1")
°
Query executed in 2.01 ms.
On Fri, May 27, 2022 at 1:00 PM Marco Lettere wrote:
> Dear all,
>
> after wrapping our heads around this for hours today, we don't know how
> to get rid of this inconsistency. Thus I ask for help ...
>
> SSCE:
>
> BaseX 9.6.4 [Standalone]
> Try 'help' to get more information.
> > xquery file:write-text("a1.txt", "°" || out:nl()) (: Same with
> codepoints-to-string(176) instead of "°" :)
>
> Query executed in 183.94 ms.
> > xquery file:read-text("a1.txt")
> °
>
> Query executed in 1.49 ms.
> > xquery file:write-text("a2.txt", file:read-text("a1.txt"))
> Query executed in 3.4 ms.
>
> > xquery file:read-text("a2.txt")
> [file:io-error] Decoding error: xb0
>
> Testing the files with linux command-line tool "file", this is the output:
>
> > file a1.txt
> a1.txt: Unicode text, UTF-8 text
>
> > file a2.txt
> a2.txt: ISO-8859 text
>
> Reading the file after "copying" it seems to change the encoding. How is
> this supposed to be handled?
>
> Regards,
>
> Marco.
>
>
[basex-talk] Encoding hassle ....
Dear all,
after wrapping our heads around this for hours today, we don't know how
to get rid of this inconsistency. Thus I ask for help ...
SSCE:
BaseX 9.6.4 [Standalone]
Try 'help' to get more information.
> xquery file:write-text("a1.txt", "°" || out:nl()) (: Same with
codepoints-to-string(176) instead of "°" :)
Query executed in 183.94 ms.
> xquery file:read-text("a1.txt")
°
Query executed in 1.49 ms.
> xquery file:write-text("a2.txt", file:read-text("a1.txt"))
Query executed in 3.4 ms.
> xquery file:read-text("a2.txt")
[file:io-error] Decoding error: xb0
Testing the files with linux command-line tool "file", this is the output:
> file a1.txt
a1.txt: Unicode text, UTF-8 text
> file a2.txt
a2.txt: ISO-8859 text
Reading the file after "copying" it seems to change the encoding. How is
this supposed to be handled?
Regards,
Marco.
Re: [basex-talk] Constructing and querying map structures in a dynamic way
On Fri, May 27, 2022 at 03:37:57PM +0200, Markus Elfring scripsit:
> > Gaps? First, to avoid a misunderstanding: whereas map keys must not
> > be the empty sequence, map values may.
>
> Special data sources might contain challenges for further
> clarification of the desired referential integrity.
Well, sure!
But if you want a useful answer you've got to ask a specific question
with reference to actual code and actual XML. Otherwise the answer is
always the entire language.
Understanding the whole comes from understanding an accumulation of
parts and understanding the individual parts comes from making specific
mistakes. An unwillingness to make mistakes in public is a hindrance to
learning.
--
Graydon Saunders | graydon...@gmail.com
Þæs oferéode, ðisses swá mæg.
-- Deor ("That passed, so may this.")
Re: [basex-talk] Constructing and querying map structures in a dynamic way
> Gaps? First, to avoid a misunderstanding: whereas map keys must not be the > empty sequence, map values may. Special data sources might contain challenges for further clarification of the desired referential integrity. Regards, Markus
Re: [basex-talk] Constructing and querying map structures in a dynamic way
Hi Markus, you wrote
"How would you determine where unexpected gaps occur?"
Gaps? First, to avoid a misunderstanding: whereas map keys must not be the
empty sequence, map values may.
An elegant way to construct map entries safely - taking care that the key is
not empty - is to use the map operator (!) or a path step, in both cases with
the map:entry() on the right-hand side.
Example, map operator:.
$thingy/@name/normalize-space(.)[string()] ! map:entry(., $tingy/string())
Example, path step:
map:merge(doc('things.xml')//thingy/@name[normalize-space(.)]/map:entry(.,
../string()))
Kind regards,
Hans-Jürgen
Am Freitag, 27. Mai 2022, 10:51:00 MESZ hat Markus Elfring
Folgendes geschrieben:
> although I do not quite understand your description
Which wording variant would you have found clearer for the mentioned use case?
> (why should I care for the representation of the map I construct?),
* Selection of a general data structure for the handling of key/value pairs
* I guess that you would like to put customised data into keys and associated
values.
> it may be that one piece of information is useful to you,
Thanks for your constructive feedback.
> as it is not obvious from the spec: it's about how to construct a map
> dynamically,
Can such a view trigger any further consequences?
> that is,
>
> (1) when the keys are not known beforehand,
> (2) and/or when the entry values are assigned in "iterations" of a FLWOR
> expression
>
> (Especially (2) might be what puzzles you.)
I thought also about this data processing area in special ways for a moment.
> (1) and (2) can't be accomplished when using the map constructor expression,
> like this: basex "map{'x':1, 'y': false()}"
Will any information sources help to understand this technical detail better
(besides the XPath function library)?
> The trick consists of using a combination of the functions map:merge() [1]
> and map:entry() [2].
I became curious how application considerations will evolve further.
> This code may for example be (most often is) a FLWOR expression.
> In this code, each entry is constructed by a call of function map:entry().
Now I stumble on the error message “[XPTY0004] Item expected, empty sequence
found.”
from my software test according to a special data source.
How would you determine where unexpected gaps occur?
Regards,
Markus
Re: [basex-talk] Constructing and querying map structures in a dynamic way
On Fri, May 27, 2022 at 10:50:57AM +0200, Markus Elfring scripsit:
> > This code may for example be (most often is) a FLWOR expression. In
> > this code, each entry is constructed by a call of function
> > map:entry().
>
> Now I stumble on the error message “[XPTY0004] Item expected, empty
> sequence found.” from my software test according to a special data
> source.
So either the key or the value isn't in the source data.
> How would you determine where unexpected gaps occur?
let $simple as element(bucket) :=
likes well-drained partial shade
zone 7
marsh plant, not a swamp plant
let $thingMap as map(xs:string,xs:string) := map:merge(
for $thing in $simple/descendant::thingy
let $key as xs:string? := $thing/@name/string()
let $value as xs:string? := $thing/string()
where normalize-space($key) and normalize-space($value)
return map:entry($key,$value)
)
return $thingMap
The map entry can be constructed only when it has meaning. Which means
the code has to allow for finding neither the key nor the value, which
is why the ? (0 or 1) quantifier on the xs:string types.
(The map-constructing FLOWR expression should also consider that there might be
duplicate keys, but one thing at a time.)
If you really want to know where in the data something is missing
let $simple as element(bucket) :=
likes well-drained partial shade
zone 7
marsh plant, not a swamp plant
let $thingProblems as xs:string* :=
for $thing in $simple/descendant::thingy
let $key as xs:string? := $thing/@name/string()
let $value as xs:string? := $thing/string()
where not(normalize-space($key)) or not(normalize-space($value))
return $thing ! concat(base-uri(.),path(.))
return $thingProblems
will tell you which nodes in the input can't create map entries.
This approach can be made more robust but this is the general idea.
--
Graydon Saunders | graydon...@gmail.com
Þæs oferéode, ðisses swá mæg.
-- Deor ("That passed, so may this.")
Re: [basex-talk] Constructing and querying map structures in a dynamic way
> although I do not quite understand your description
Which wording variant would you have found clearer for the mentioned use case?
> (why should I care for the representation of the map I construct?),
* Selection of a general data structure for the handling of key/value pairs
* I guess that you would like to put customised data into keys and associated
values.
> it may be that one piece of information is useful to you,
Thanks for your constructive feedback.
> as it is not obvious from the spec: it's about how to construct a map
> dynamically,
Can such a view trigger any further consequences?
> that is,
>
> (1) when the keys are not known beforehand,
> (2) and/or when the entry values are assigned in "iterations" of a FLWOR
> expression
>
> (Especially (2) might be what puzzles you.)
I thought also about this data processing area in special ways for a moment.
> (1) and (2) can't be accomplished when using the map constructor expression,
> like this: basex "map{'x':1, 'y': false()}"
Will any information sources help to understand this technical detail better
(besides the XPath function library)?
> The trick consists of using a combination of the functions map:merge() [1]
> and map:entry() [2].
I became curious how application considerations will evolve further.
> This code may for example be (most often is) a FLWOR expression.
> In this code, each entry is constructed by a call of function map:entry().
Now I stumble on the error message “[XPTY0004] Item expected, empty sequence
found.”
from my software test according to a special data source.
How would you determine where unexpected gaps occur?
Regards,
Markus
