Hi Ron,
There is only one document where searchPath = ‘/content’ I did find that the constraining query is finding 6992 docs where I assumed it would be 1 : declare default element namespace 'http://developer.envisn.com/xmlns/envisn/netvisn/'; let $cq := cts:and-query(( cts:collection-query('content'), cts:element-value-query(xs:QName('searchPath'), '/content'), cts:element-value-query(xs:QName('navType'), 'content') )) return count(cts:search(doc(), $cq, 'unfiltered')) Recently I had turned on ‘fast case sensitive searches’ and ‘fast diacritic sensitive searches’ so wondering if that changed the way cts:element-value-query works. I had the understanding that a range element index was not necessary to query element value matches. Question: Is it necessary to reindex the database after changing these options? I understand the xpath filter should not be needed, but the code has been on many databases and this wait time issue has not been reported before, but anything is possible. Maybe a not perfect query was handled better in older versions? Thanks for your help, <http://www.envisn.com/> Envisn Gary Larsen | <http://www.envisn.com/> Envisn, Inc. | 508.259.6465 | <http://www.envisn.com/blog> Envisn's IBM Cognos Blog From: [email protected] [mailto:[email protected]] On Behalf Of Ron Hitchens Sent: Wednesday, August 02, 2017 5:47 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] cts:element-value-query performance Question: Why do you have the XPath filter following cts:search()? As far as I can see, it’s not needed because the "cts:element-value-query(xs:QName('searchPath'), '/content’)" term does the same thing. Nothing will come back from the cts:search where the “searchPath" element does not contain “/content”. When you append a filtering XPath to a cts:search like this, you’re effectively creating a for loop which compares every matching result from the cts:search call. In this case it appears to be an effective no-op because it should match everything returned, unless you have other “searchPath” elements in your documents at different paths. Iterative matching like this can effectively defeat the efficiency of index-only matching (which you’re forcing by using the “unfiltered” option), unless the optimizer can deduce that the XPath is unneeded or can be rewritten to only use indexes. Even though you’re specifying an unfiltered search, your XPath is a filter. Another Question: how many documents in the “content” collection, with “navType” = “content” and “searchPath” = “/content” might be matched by cts:search? It doesn’t appear that you’re taking any steps to limit the size of the result sequence or paginate through the candidates. The cts:search function returns an internal iterator that will be lazily evaluated when needed, it does not reify documents immediately. That makes things like “cts:search (…)[101 to 110]” very efficient because items 1 to 100 can be skipped without ever looking at those documents. If you have a filtering XPath, it can’t be optimized that way. You’d want something like this: return (cts:search (…)[$first to $last])/some/path[foo=‘bar'] I suspect you may be matching far more documents than you think you are and you’re iterating over all of them, which is what’s taking so long. You may also want to specify the “exact” option on the value queries to make sure that there are no unexpected matches, since you have a non-word character in the value. Cheers. ---- Ron Hitchens [email protected], +44 7879 358212 On August 2, 2017 at 3:57:01 PM, Gary Larsen ([email protected]) wrote: This query is taking a long time to execute which seems to have happened in recent versions for MarkLogic. I’m currently using 8.0-6.4 declare default element namespace 'http://developer.envisn.com/xmlns/envisn/netvisn/'; let $cq := cts:and-query(( cts:collection-query('content'), cts:element-value-query(xs:QName('searchPath'), '/content'), cts:element-value-query(xs:QName('navType'), 'content') )) return cts:search(doc(), $cq, 'unfiltered')/content/lookupInfo[searchPath='/content'] A large number of documents have searchPath which start with ‘/content’, for example: '/content/package[@name='BI Reporting']/folder[@name='BI Sales']/report[@name='A Test1 - Order Summary']' The longer search path returns very quickly but the ‘/content’ path executes upwards of 10 minutes in large databases. Thanks for any advice, Gary _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
