Have you tested 4.2 with this problem? There are changes which should help. I can't say that they will make the difference you're looking for, but in my testing the difference was dramatic.
As Tim suggested, the fundamental problem with the technique below is that it is single-threaded. You can do better with his suggestion using xdmp:spawn, or perhaps improve on that with a batch-oriented technique like http://marklogic.markmail.org/thread/k56wrer44ufgaml5 - or use https://github.com/marklogic/corb to drive the work through a Java application. -- Mike On 6 Jun 2011, at 19:06 , [email protected] wrote: > I need to delete all of the documents in a collection in Marklogic. The > database in which we have the documents stored has over 4.4 million > documents. The collection itself has approximately 1.7 million documents. > We are currently running 4.0-4 of MarkLogic. We will be upgrading in the > near future to 4.2, but that does not help us with this problem. > > I have attempted to use the code below to perform the delete, but I am only > able to delete 5 documents per second using this method. > > > (: Delete documents in a collection :) > xquery version "1.0-ml"; > > let $assert := > if (xdmp:request-timestamp()) then () > else > error((), 'NOTIMESTAMP', text { 'outer query is not read-only' }) > let $path := '/documents/7615990080550346622/' > let $map := map:map() > let $list-uris := > for $i in xdmp:directory($path, 'infinity')[1 to 10000] > return map:put($map, xdmp:node-uri($i), true()) > let $do := xdmp:eval(' > declare variable $URIS as map:map external; > xdmp:document-delete(map:keys($URIS)) > ', > (xs:QName('URIS'), $map), > <options xmlns="xdmp:eval"> > <isolation>different-transaction</isolation> > <prevent-deadlocks>true</prevent-deadlocks> > </options> > ) > return count(map:keys($map)) > > I read another post on this mailing list that indicated that if you set > directory creation to "manual" and did not have any triggers enabled in the > database, that doing a directory delete would be extremely fast. However, we > DO have CPF enabled (and cannot disable it) and cannot set directory creation > to manual since we depend on the automatic directory creation in our > application. > > > Any suggestions as to how we can get better throughput for deleting all of > the documents in a collection OR deleting a large number of documents from a > large database? Clearing a collection as quickly as possible while > retaining the collection itself is the goal at this time. > > > Thanks, > > > Allen Shatzer > > > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
