Hi Eliot, If the length of a search strings exceeds MAXLEN [1], a sequential search will be performed instead. You can achieve index-based lookups by increasing the limit when creating/optimizing a database.
Hope this helps, Christian [1] https://docs.basex.org/wiki/Options#MAXLEN On Wed, Oct 11, 2023 at 1:16 AM Eliot Kimber <eliot.kim...@servicenow.com> wrote: > Sorry, my times are off—I was doing a nonsense multiplication of the > result of prof:track. > > Reporting the raw times returned by db:token(), I get: > > "lookup by docPath took 0.255696ms" > > "lookup by docPath took 0.054183ms" > > "lookup by docPath took 0.211373ms" > > "lookup by docPath took 0.175889ms" > > "lookup by docPath took 0.220833ms" > > "lookup by docPath took 0.156875ms" > > "lookup by docPath took 0.139612ms" > > "lookup by docPath took 0.007018ms" > > "lookup by docPath took 0.179297ms" > > "lookup by docPath took 0.00964ms" > > "lookup by docPath took 0.093668ms" > > > > Which adds up to about 400ms for 1300 topics looked up. > > > > Using db:attribute() I get: > > "lookup by docPath took 3.794676ms" > > "lookup by docPath took 0.032425ms" > > "lookup by docPath took 0.092237ms" > > "lookup by docPath took 0.097815ms" > > "lookup by docPath took 1.246617ms" > > "lookup by docPath took 0.050855ms" > > "lookup by docPath took 0.065685ms" > > "lookup by docPath took 0.00511ms" > > "lookup by docPath took 0.057833ms" > > > > Cheers, > > > > E. > > > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *Eliot Kimber <eliot.kim...@servicenow.com> > *Date: *Tuesday, October 10, 2023 at 6:02 PM > *To: *basex-talk@mailman.uni-konstanz.de < > basex-talk@mailman.uni-konstanz.de> > *Subject: *Db:token() vs db:attribute(): Attribute is much slower -- why > > Using 10.6 on macOS: > > > > I have a data file with elements like: > > <record language="en-US" family="pce-test-data" > bundlename="bundle-appportman-application-portfolio-management" > topicpath="product/application-portfolio-management/concept/application-portfolio-management-landing-page.dita" > notviewed="viewed" viewcount="13579"> > > > > One such entry for each topic in our public docs server that is accessed > at least once (Adobe page views data). > > > > I generate a report for all the topics used from a root map that includes > the page views data for each topic for which there is an entry. > > > > The database with this data set is indexed with an attribute and a token > index. > > > > In my testing I’m finding that lookup using db:attribute() on the > @topicpath attribute (which is always a single value) is about 10x slower > than using db:token(). > > > > That is, I have this code: > > $records := db:token($analyticsmgmt:analyticsDatabase, $docPath, > 'topicpath')/.. > > > > When I run this for a set of about 1200 topics, the db:token version takes > about 400ms but the db:attribute version takes 9 seconds! > > > > I’m surprised at this result—I would expect the two forms of index to > perform with the same speed: the attribute is a single blank-delimited > token and I would expect the string match under the covers to be using the > same code path. > > > > But using the db:token database is clearly faster by a lot. > > > > If I report the time each lookup uses, the times vary quite a lot, but for > the token lookup they tend to be in the 10-60ms range, but for the > attribute lookup, many are in the 1 or 2 second range, which explains the > much longer run time. > > > > I’m wondering if this result is expected due to implementation details of > the attribute index as distinct from the token index or if this indicates > some logic bug in the attribute index implementation. > > > > Cheers, > > > > E. > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> >
