Trying http://docs.marklogic.com/cts:sum-aggregate is a good idea. I'll throw out some others, roughly in order of probability and difficulty. Stop whenever you achieve acceptable response times.
A full 20-30 sec of warmup sounds excessive to me. A large part of the problem is likely to be disk I/O and a rather slow disk subsystem. Monitor the OS to test this theory. If disk I/O doesn't seem to be the culprit, come back with output from the profiler, from xdmp:query-meters, and from xdmp:query-trace before proceeding. The rest of my ideas assume that disk *is* the bottleneck, and likely won't help if it isn't. Assuming disk I/O, check to see if the OS paging or swapping. If it needs more memory, fix that first. You might be able to tune MarkLogic to use less, but it's usually easier to upgrade the hardware. If you can't find any evidence of paging, it's time to start tuning. You can try the database setting "preload mapped data". Also the evq(id=X) portion may not be in list-cache. You might address that with a range index on id, too. That way you can use cts:element-range-query(xs:QName("id"),'=',$id) instead of cts:element-value-query(), avoiding most list lookups. If that still doesn't do it, consider upgrading the disk. Worst case, my guess is you are asking for about 3000 disk reads - but it could be significantly more or less, depending on configuration. So a faster disk might help, or this might be a good use-case for SSD. Or you might paginate across the ids, as suggested. For example display the first 100 ids at a time. If the performance is acceptable, you're done. Looking further, it would be nice to reduce the 500 trips through cts:sum and cts:element-values down to one trip. It seems to me that what you're after is really the co-occurrence of id and sum(valuation). That's tricky to get directly from the database, but here's an approach that might perform better in the case where you have relatively few valuations and relatively many ids. let $map := cts:element-value-co-occurrences( xs:QName('id'), xs:QName('valuation'), ('map'), cts:and-query( (cts:collection-query('/project'), cts:element-range-query(xs:QName("id"), '=', 1 to 500)))) for $id in map:keys($map) return element valuation-sum { attribute id { $id }, sum(map:get($map, $id) ! (cts:frequency(.) * .)) } That last bit uses the simple mapping operator, which only appears in ML6: http://developer.marklogic.com/blog/simple-mapping-operator - if you can't use ML6, substitute a nested FLWOR expression. ... sum(for $v in map:get($map, $id) return cts:frequency($v) * $v) } Finally, with ML6 you might be able to move the summation into an aggregate user-defined function (UDF): http://docs.marklogic.com/guide/app-dev/aggregateUDFs -- Mike On 23 Oct 2012, at 22:20 , Abhishek53 S <abhishek5...@tcs.com> wrote: > > Hi All, > > I am having performance penalty (Average time 23-30 Sec for the first time > and less than 1 sec for next onward hit[which is accecpted]) with cts:sum > over one of the element stores valuation information > > ----------------------------------------------------------------------- > for $id in (1 to 500) > > return cts:sum(cts:element-values( > > xs:QName("valuation"), > > (), > > (), > > cts:and-query(( > > > cts:element-value-query(xs:QName("id"),$id), > > cts:collection-query("/project") > > )) > > ) > > ) > > > Do we have any different way to over come performance overhead. > > Thanks > Abhishek Srivastav > I T Analyst > Tata Consultancy Services > Cell:- +91-9883389968 > Mailto: abhishek5...@tcs.com > Website: http://www.tcs.com > ____________________________________________ > Experience certainty. IT Services > Business Solutions > Outsourcing > ____________________________________________ > =====-----=====-----===== > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > > _______________________________________________ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general