A last update, which may illuminate a little. After reindexing the database using Norwegian (snowball), stemming, and keeping diacritis, RESTXQ processes neither the special characters (treats them as closest ascii), nor inflected forms.
The words "mannen" (=the man, definite) and "spaserer" (=walks, present tense), result in no output, while using the naked stems "mann" and "spaser" the full result is displayed. In contrast to REST which behaves as expected. Cheers Lars 2015-05-18 15:28 GMT+02:00 Lars Johnsen <yoon...@gmail.com>: > As an update, after rebuilding database with > > text index, > full text index (no language, no stemming, keep diacritics) > > restarting server: > BaseX 8.1.1 [Server] > Server was started (port: 29084) > [main] INFO org.eclipse.jetty.server.AbstractConnector - Started > SelectChannelConnector@0.0.0.0:8984 > HTTP Server was started (port: 8984) > > RESTXQ: Norwegian characters are converted using full text index, changing > to text index takes forever. > REST: Full-text works as expected, and text index works as expected (same > as runing in GUI for both). > > It looks as if the index structure is treated differently. > > > 2015-05-18 15:07 GMT+02:00 Lars Johnsen <yoon...@gmail.com>: > >> The full text query is blisteringly fast for both, the text index query >> is fast only for REST queries and seems not to be used with queries in >> RESTXQ. I am rebuilding the whole database now to see how it goes, and will >> restart everything for a new assessment. >> >> >> >> 2015-05-18 15:00 GMT+02:00 Christian Grün <christian.gr...@gmail.com>: >> >>> > However, when using text index instead of full text the results are >>> the same >>> > for both, except that RESTXQ takes almost forever >>> >>> What about the original query: Has it been slow as well, or do you >>> think this is a new problem? >>> >>> >>> > 2015-05-18 14:28 GMT+02:00 Christian Grün <christian.gr...@gmail.com>: >>> >> >>> >> It could be that your URL is decoded in a wrong way.. What happens if >>> >> you run the following function with REST and RESTXQ and "føre" as >>> >> word? >>> >> >>> >> declare >>> >> %rest:path("/test/encoding/{$word}") >>> >> function page:test-encoding($word) { >>> >> string-to-codepoints($word) >>> >> }; >>> >> >>> >> Thanks, >>> >> Christian >>> >> >>> >> >>> >> string-to-codepoints() >>> >> > REST output (2 first lines): >>> >> > føre >>> >> > fø - re 219 >>> >> > >>> >> > RESTXQ >>> >> > føre >>> >> > fo - re 123 >>> >> > >>> >> > The first word quoted is "føre" in both cases and is what the >>> scripts >>> >> > see, >>> >> > so the full text is given the same in both cases. Could it be that >>> >> > within >>> >> > RESTXQ the full text index is treated differently? >>> >> > >>> >> > I will work closer on a self contained example, but thought this >>> might >>> >> > point to something. >>> >> > >>> >> > Cheers >>> >> > Lars >>> >> > >>> >> > >>> >> > 2015-05-18 13:44 GMT+02:00 Lars Johnsen <yoon...@gmail.com>: >>> >> >> >>> >> >> Hi Christian - and thanks for fast response. Latest version 8.11 >>> is in >>> >> >> use >>> >> >> (same behaviour as previous). Let me see if I can make a self >>> contained >>> >> >> example. >>> >> >> >>> >> >> best, >>> >> >> Lars >>> >> >> >>> >> >> 2015-05-18 13:40 GMT+02:00 Christian Grün < >>> christian.gr...@gmail.com>: >>> >> >>> >>> >> >>> Hi Lars, >>> >> >>> >>> >> >>> hm, that's difficult to tell. All I can say is that this sounds >>> >> >>> unusual, so I'm coming up with my standard questions: Do you >>> think you >>> >> >>> could build us a little example that allows us to reproduce the >>> >> >>> problem? Have you tried the latest version of BaseX? >>> >> >>> >>> >> >>> Best, >>> >> >>> Christian >>> >> >>> >>> >> >>> >>> >> >>> On Mon, May 18, 2015 at 1:35 PM, Lars Johnsen <yoon...@gmail.com> >>> >> >>> wrote: >>> >> >>> > >>> >> >>> > I am running a web script in two identical versions (identical >>> as in >>> >> >>> > "cut >>> >> >>> > and paste"), one via RESTXQ and one vi REST. The response is >>> >> >>> > different, >>> >> >>> > and >>> >> >>> > I wondered what may be the trouble. >>> >> >>> > >>> >> >>> > For example the output (the URLs only works locally) for >>> >> >>> > http://ljohnsen:8984/hyphens/mellom >>> >> >>> > is the same as >>> >> >>> > http://ljohnsen:8984/rest?run=hyphen-show.xq&word=mellom >>> >> >>> > >>> >> >>> > which is a set of hyphenation data: >>> >> >>> > mellom >>> >> >>> > mel - lom 17005 >>> >> >>> > Mel - lom 144 >>> >> >>> > mel - lom. 50 >>> >> >>> > >>> >> >>> > but if "mellom" is exchanged with "nasjonalbiblioteket" only >>> the >>> >> >>> > REST >>> >> >>> > version shows any result, which then is the same as I get >>> >> >>> > experimenting >>> >> >>> > in >>> >> >>> > the GUI. >>> >> >>> > >>> >> >>> > The actual script is added below, and which runs in both >>> versions >>> >> >>> > (identical apart form the rest and restxq interfaces), it uses >>> full >>> >> >>> > text >>> >> >>> > search, but results differ when run under the REST-regime. >>> >> >>> > >>> >> >>> > All the best >>> >> >>> > Lars G Johnsen >>> >> >>> > National Library of Norway >>> >> >>> > >>> >> >>> > module namespace page = 'http://basex.org/modules/web-page'; >>> >> >>> > >>> >> >>> > declare >>> >> >>> > %rest:path("/hyphens/{$word}") >>> >> >>> > %output:method("html") >>> >> >>> > >>> >> >>> > function page:show-hyphens($word) { >>> >> >>> > let $db := db:open('hyphen-data') >>> >> >>> > let $hyphens := for $hyp in $db/hyphens/hyphens[full >>> contains >>> >> >>> > text >>> >> >>> > {$word}] >>> >> >>> > group by $first := $hyp/first, $second := $hyp/second >>> >> >>> > let $count := count($hyp) >>> >> >>> > order by xs:int($count) descending >>> >> >>> > return element p { >>> >> >>> > attribute freq {$count}, >>> >> >>> > $first, " - ", $second, $count >>> >> >>> > } >>> >> >>> > >>> >> >>> > let $total := sum($hyphens//@freq) >>> >> >>> > let $div := element div { >>> >> >>> > element p {$word}, >>> >> >>> > for $hyp in $hyphens >>> >> >>> > return element div { >>> >> >>> > attribute class {"hyph"}, >>> >> >>> > attribute style {"font-size:", 1 >>> >> >>> > +round(xs:int($hyp//@freq/data()) >>> >> >>> > div $total,1) || "em"}, >>> >> >>> > $hyp >>> >> >>> > >>> >> >>> > } >>> >> >>> > } >>> >> >>> > return >>> >> >>> > <html encoding="UTF-8"> >>> >> >>> > <head> >>> >> >>> > <meta http-equiv="Content-Type" content="text/html" >>> >> >>> > charset="UTF-8" >>> >> >>> > /> >>> >> >>> > <title>Orddelinger</title> >>> >> >>> > </head> >>> >> >>> > <body>{$div} >>> >> >>> > </body> >>> >> >>> > </html> >>> >> >>> > >>> >> >>> > }; >>> >> >> >>> >> >> >>> >> > >>> > >>> > >>> >> >> >