OK, if you’re confident that in the database there is only one document with a “searchPath” element that contains the string “/content”, but you’re getting 6,992 of them, then there’s a good chance that there’s something wrong with your query.
You are correct that you should not need a range index. In fact, it wouldn’t really help for this. Range indexes are an ordered list of values that occur in documents. They help with fast sorting and selecting, uh, ranges of values. A simple value match can be resolved from the standard indexes. Basically like a hash lookup. When a document is added to the database, the values of elements are indexed. However, the values are parsed and not simply stored verbatim. The value “/content” is a sequence of one punctuation character and seven word characters. The various query options can affect how a value is actually matched by the query. Hence the various sensitive/insensitive and collation options on queries. If it’s important that you match an exact string value, then you need to make sure that the right matching options are in effect for the query. There’s an option for that: “exact”, which is an alias for the options that will match a target string exactly. Try adding that option to your two cts:element-value-query constructors and see if it makes a difference. ---- Ron Hitchens [email protected], +44 7879 358212 On August 2, 2017 at 11:41:52 PM, Gary Larsen ([email protected]) wrote: Hi Ron, There is only one document where searchPath = ‘/content’ I did find that the constraining query is finding 6992 docs where I assumed it would be 1 : declare default element namespace ' http://developer.envisn.com/xmlns/envisn/netvisn/'; let $cq := cts:and-query(( cts:collection-query('content'), cts:element-value-query(xs:QName('searchPath'), '/content'), cts:element-value-query(xs:QName('navType'), 'content') )) return count(cts:search(doc(), $cq, 'unfiltered')) Recently I had turned on ‘fast case sensitive searches’ and ‘fast diacritic sensitive searches’ so wondering if that changed the way cts:element-value-query works. I had the understanding that a range element index was not necessary to query element value matches. Question: Is it necessary to reindex the database after changing these options? I understand the xpath filter should not be needed, but the code has been on many databases and this wait time issue has not been reported before, but anything is possible. Maybe a not perfect query was handled better in older versions? Thanks for your help, [image: Envisn] <http://www.envisn.com/> *Gary Larsen* | *Envisn, Inc.* <http://www.envisn.com/> | 508.259.6465 | *Envisn's IBM Cognos Blog* <http://www.envisn.com/blog> *From:* [email protected] [mailto: [email protected]] *On Behalf Of *Ron Hitchens *Sent:* Wednesday, August 02, 2017 5:47 PM *To:* MarkLogic Developer Discussion *Subject:* Re: [MarkLogic Dev General] cts:element-value-query performance Question: Why do you have the XPath filter following cts:search()? As far as I can see, it’s not needed because the "cts:element-value-query(xs:QName('searchPath'), '/content’)" term does the same thing. Nothing will come back from the cts:search where the “searchPath" element does not contain “/content”. When you append a filtering XPath to a cts:search like this, you’re effectively creating a for loop which compares every matching result from the cts:search call. In this case it appears to be an effective no-op because it should match everything returned, unless you have other “searchPath” elements in your documents at different paths. Iterative matching like this can effectively defeat the efficiency of index-only matching (which you’re forcing by using the “unfiltered” option), unless the optimizer can deduce that the XPath is unneeded or can be rewritten to only use indexes. Even though you’re specifying an unfiltered search, your XPath is a filter. Another Question: how many documents in the “content” collection, with “navType” = “content” and “searchPath” = “/content” might be matched by cts:search? It doesn’t appear that you’re taking any steps to limit the size of the result sequence or paginate through the candidates. The cts:search function returns an internal iterator that will be lazily evaluated when needed, it does not reify documents immediately. That makes things like “cts:search (…)[101 to 110]” very efficient because items 1 to 100 can be skipped without ever looking at those documents. If you have a filtering XPath, it can’t be optimized that way. You’d want something like this: return (cts:search (…)[$first to $last])/some/path[foo=‘bar'] I suspect you may be matching far more documents than you think you are and you’re iterating over all of them, which is what’s taking so long. You may also want to specify the “exact” option on the value queries to make sure that there are no unexpected matches, since you have a non-word character in the value. Cheers. ---- Ron Hitchens [email protected], +44 7879 358212 On August 2, 2017 at 3:57:01 PM, Gary Larsen ([email protected]) wrote: This query is taking a long time to execute which seems to have happened in recent versions for MarkLogic. I’m currently using 8.0-6.4 declare default element namespace ' http://developer.envisn.com/xmlns/envisn/netvisn/'; let $cq := cts:and-query(( cts:collection-query('content'), cts:element-value-query(xs:QName('searchPath'), '/content'), cts:element-value-query(xs:QName('navType'), 'content') )) return cts:search(doc(), $cq, 'unfiltered')/content/lookupInfo[searchPath='/content'] A large number of documents have searchPath which start with ‘/content’, for example: '/content/package[@name='BI Reporting']/folder[@name='BI Sales']/report[@name='A Test1 - Order Summary']' The longer search path returns very quickly but the ‘/content’ path executes upwards of 10 minutes in large databases. Thanks for any advice, Gary _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
[email protected]
Description: Binary data
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
