Leaving the search API aside, the most efficient techniques are probably doc($uri-list) and cts:document-query($uri-list). Both doc() and cts:document-query() are much more efficient at handling large inputs than a "where" clause can be. However, I'm not sure what will happen when you try this approach with search:search and a searchable-expression option. Building the options XML is likely to be fairly expensive, and could dominate the profile.
Now, you've expressed your requirements in a very low-level way, and I think that's painting you into a corner. Try moving up a layer or two, and getting back to the business problem that you're trying to solve. I'm not sure what that is, but I'll suggest trying to rethink the problem so that the function returns a relatively straightforward cts:query to match the required documents. You can supply that query as an extra option to search:search(). That way you have a relatively compact query to pass around, instead of a list of 10^5 documents. -- Mike On 10 Jun 2011, at 07:14 , Wang, Ningjun (LNG-NPV) wrote: > We have a requirement to do keyword search on a set of documents returned by > some function. For example the function getRelatedDocUrls() return a list of > document URLs (could be hundreds of thousands of document). I need to do a > keyword search within those documents and return the correct total count and > return result with 10 documents per page. What is the most efficient way to > do that? > > One option I am thinking is to do the keyword search first then join the > result with the set of documents, e.g. > > Let $urls := getRelatedDocUrls() > for $result in search:search(keyword)/search:result, $url in $urls > where $result/@uri = $url > > There are two problems > 1. search:search(keyword) return one page at a time. Suppose I make it > return 20 docs per page, then the first page may not have any document > matching $urls. > 2. search:search(keyword) return the total count of document matching > keyword. However I want the total count of documents not only matching the > keyword, but also with url in $urls > > Of course I can make search:search(keyword) to return all result in one page > and then through the join to find out the answer above. However this will be > very inefficient because search:search(keyword) can return millions of > documents. I just want the correct total count and the first 10 documents > that match the keyword and with url match $urls. > > What is the most efficient way to do this? > > Thanks > > > > Ningjun Wang > Consulting Software Engineer > > LexisNexis Global Solutions Dev > New Providence, NJ > (908) 665-6787 > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
