Correction on #2) " Xdmp:node-insert and similar operate in-memory only" I was thinking of a different api, the 'in memory' update https://github.com/marklogic/commons/blob/master/memupdate/in-mem-update.xqy
The xdmp:node-insert* functions do operative directly on the document 'on disk' Internally they perform the equivalent of a complete read/delete/write of the entire document each call - multiple calls in the same transaction should be avoided vs reading the document in memory, modifying it and writing it out as one operation ( exceptions in cases of very large documents may be a wash) I fell victim to the exact confusion mentioned by "jimuller2002" aka "anonymous" right there in doc comments - "RTFM" I should have :) https://docs.marklogic.com/xdmp:node-insert-child " A common issue I've seen with developers updating nodes is not understanding the difference between updating a node in a document, in the database ... versus updating a node in a document that is in memory." In my case its not 'not understand' but rather 'not remembering' which API does what -- the result equally confusing. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation d...@marklogic.com Phone: +1 812-482-5224 Cell: +1 812-630-7622 www.marklogic.com<http://www.marklogic.com/> From: David Lee Sent: Sunday, June 21, 2015 9:59 AM To: MarkLogic Developer Discussion Subject: RE: [MarkLogic Dev General] Search and update 1) See #3 2) Xdmp:node-insert and similar operate in-memory only, so themselves do not cause lock problems because they don’t write the document. See #3 3) Yes, to split up an XQuery program into separate transactions to avoid locking issues, you can use one of the xdmp:eval* or xdmp:invoke* functions. I encourage you to look at xdmp:invoke-function() as it doesn’t require a separately stored module (like xdmp:invoke) and it doesn’t have issues with escaping XQuery as strings and the associated risks of XQuery injection (by mistake or intent). Note: it’s not sufficient to run the update portion in a separate transaction if you have already run the query/search in the main transaction – the read locks are already acquired, and running an update as a synchronous child transaction will block attempting to write to the same document searched (depending on the details of the query and update) Like this [ Main Transaction ] Results = Query --> Creates Read Locks Apply [ Update -> Waits on Read Locks ] --> Potential deadlock [ END Main transaction ] To avoid this run *both* search and query as child transactions of an ‘auto’ transaction. Like this [ Main transaction ] Results = [ Query Transaction – Creates Read Locks – Query – End Transaction Releases Locks ] Apply [ Update Transaction – Creates Write lock on 1 doc – no blocking - Release Lock ] [ END ] This way the Query transaction has completed and released its locks before running the update. The above example could run the Apply in the parent Transaction – but only very carefully, If the Apply uses an update method that the XQuery parser can detect on parsing it may upgrade the auto Transaction to a Update transaction before the query occurs. Note also that this is no longer an atomic operation – If your logic requires that the Update is synchronized, ( 2 parallel requests updating based on ‘stale’ information ) you may need to an additional check in the update to make sure the document being updated has not changed since the query – This won’t cause any locking problems if you make the check and update In the same transaction *for a single document* - Take a look at xdmp:invoke-function() for examples – XQuery 3.0 anonymous functions are really handy for this. From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [mailto:general-boun...@developer.marklogic.com] On Behalf Of RAVINDER MAAN Sent: Sunday, June 21, 2015 6:21 AM To: general@developer.marklogic.com<mailto:general@developer.marklogic.com> Subject: [MarkLogic Dev General] Search and update Hi all I have a update job which search for documents matching criteria and then insert or delete a node in the matching docs. I know that if search and update is in same transaction it causes lock problems. So search return URI of the docs and in second request I do fn:doc and node insert or replace. Which seems to be working. I have below questions 1. How other people are handling search and update ? Is there any other way? 2. Does the XPATH search and replace also cause locking problem? for example xdmp:node-replace(/root/old-node, $new-node) 3. Is it good idea to perform search and do the update in xdmp:eval with different-transaction ? Thanks & regards, Ravi
_______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
