Off the top of my head, collections seem like a reasonable way to do this. If you set up a collection lexicon, it could allow you to get interesting counts without actually touching any of the documents as well (like the URI lexicon, the collection lexicon takes a cts:query as a parameter, allowing you do filter by cts:query).
I'd say it is worth a try. -Danny From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Lee, David Sent: Tuesday, July 19, 2011 2:35 PM To: General Mark Logic Developer Discussion (general@developer.marklogic.com) Subject: [MarkLogic Dev General] Lots of collections ... Thanks to some tips from this group (and especially Kelly !) I've started leveraging collections instead of directories. So far really fantastic results !!! Thank you all !! Of course one success opens the doors to a million questions ... Question ... Is there a significant cost to having a 'large' number of overlapping documents in collections ? In my use case I may have millions of very similar small documents all with some basic set of attributes which have a small set of possible values. I've implemented attribute value range indexes, but was wondering if collections might work better ? A typical use case would be to filter a result set by only those documents with a particular attribute set to one value. If I had collections for each attribute/value combination (maybe 100 collections max) A collection query could do the equivalent of a range index. Example: <logfile host="host1" system="tomcat" ...> ... Instead of making a range index on logfile/@host and logfile/@system Make collections called host-host1 host-host2 host-host3 ... and system-tomcat system-mysql ... Then this xpath //logfile[@host eq 'host1'] would be equivalent to a collection search on 'host-host1' Is this brilliant or stupid ? Obviously there will be a tradeoff ... but I'm thinking in this case since the number of possible values is very small that collections might actually be a good thing. -David ---------------------------------------- David A. Lee Senior Principal Software Engineer Epocrates, Inc. d...@epocrates.com<mailto:d...@epocrates.com> 812-482-5224
_______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
