On 31.03.2020 18:32, Ben Engbers wrote:
Hi,
For (my personal) clarity, I have split up the original function in two
parts:
declare function local:step_one($nodes as node()*) as array(*)*
{
let $text := for $node in $nodes
return $node/text() =>
tokenize() => distinct-values()
let $idf := $text =>
tidyTM:wordCount_arr()
return $idf
};
In local:step_one(), I first create a sequence with the distinct tokens
for each $node. All the sequences are joined in $text.
I then call wordCount_arr to count the occurences of each word in $text:
declare function tidyTM:wordCount_arr(
$Words as xs:string*)
as array(*) {
for $w in $Words
let $f := $w
group by $f
order by count($w) descending
return ([$f, count($w)])
} ;
I would say that tidyTM:wordCount_arr returns a sequence of arrays but I
am not certain if I have specified the correct return-type?
Reading the code I agree that the return type seems to be a sequence of
arrays but therefore I wonder why you don't get a similar error as later
on with declaring
array(*)
and not
array(*)*
Calling local:step_one(tidyTM:remove_Stopwords($nodes, "Stp", $Stoppers))
returns:
["probleem", 703]
["opgelost.", 248]
....
I had hoped that calling the following local:wordFreq, would add the
idf to each element but instead I get an error
declare function local:wordFreq_idf($nodes as node()*) as array(*)
{
let $count := count($nodes)
let $idf := local:step_one($nodes)
let $result := for-each( $idf,
function($z) {array:append ($z, math:log($count div $z(2) ) ) } )
return $result
};
[XPTY0004] Cannot promote (array(xs:anyAtomicType))+ to array(*): $idf
:= ([ "probleem", 703 ], [ "opgelost.", 248 ], ...).
The message tries to tell you that the declared return type
array(*)
is a single array while the function returns a (non-empty) sequence of
arrays so using
declare function local:wordFreq_idf($nodes as node()*) as array(*)*
would remove that error.
To insert the third value into each array I think you want
let $result := $idf ! array:append(., math:log($count div .(2) ))
