Danny, I think you explained almost all. I ran the examples from the dev guide and created some of my own, understanding a lot more about undetectable deadlocks etc.
But one question still remains: what is the difference with document-locks instead of get-document-properties and lock-acquire instead of document-insert (or set-document-properties for instance).. Kind regards, Geert > -----Original Message----- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of > Danny Sokolsky > Sent: dinsdag 10 februari 2009 23:34 > To: General Mark Logic Developer Discussion > Subject: RE: [MarkLogic Dev General] Atomic > andexclusivelockondocuments(accross a cluster) > > To explain why storing the "lock" on the property will work > transactionally, let's briefly review the transaction > semantics of MarkLogic Server (most of this is described in > the Developer's Guide "understanding Transactions in > MarkLogic Server" chapter-- > http://developer.marklogic.com/pubs/4.0/books/dev_guide.pdf). > > MarkLogic Server has two types of requests (we'll call them > statements): > query statements and update statements. Query statements run > at a timestamp and simply look at the latest versions of > documents up to the timestamp in which the query runs. Query > statements do not lock any documents. > > An update statement occurs if there is any kind of update > operation in the statement. This is determined during static > analysis. Update statements run in what is called > readers/writers mode. During the statement evaluation, each > time a document is referenced the system automatically > attempts to get a read lock on the document. It does this > for each document the statement accesses. When it comes time > to update the document, the system tries to promote the read > lock on any documents that will be updated to a write lock. > If it cannot get the write lock, the statement will either > retry until it can get the lock or will throw an exception > and the update will fail. > > So if you have the property stored with the document URI and > you use the technique of checking the property for a known > value, that value will be checked while the system has a lock > on the document, it will not be allowed to be updated unless > it can get a write lock on the document. > Any other thread trying to update this document while it is > locked will have to wait until the lock is released, and as > long as you update the document and the property (or however > you are storing the "user" name) in the same transaction, > they will remain in sync. > > > -Danny > > -----Original Message----- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of > Geert Josten > Sent: Tuesday, February 10, 2009 1:37 PM > To: General Mark Logic Developer Discussion > Subject: RE: [MarkLogic Dev General] Atomic > andexclusivelockondocuments(accross a cluster) > > > > Yes, and to be clear, what I was talking about was to *not* use > > xdmp:lock-acquire at all, but to use the property (or > another document > > as Frank is suggesting) instead. > > I know you were not talking about lock-acquire, but do you > mean that it makes big difference and I really should not use > it to make it work like we want? (Or only use it in > combination with a document-insert as described by Frank) > > And next to this, if I supply the uri of the document that is > to be locked as a request param, how does MarkLogic Server > know which document to write-lock for the transaction? Or is > the write-lock started at the moment the document-insert is called? > > It sounds to me that by using a document-insert on a helper > doc, you are forcing the MarkLogic Server to synchronize the > lock requests, right? So if all lock requesting calls would > start with a document-insert to 'lock.xml' (not really doing > anything actually), it will result in all lock requests being > synchronized? > > I appreciate the input, I am really learning from these details.. > > Kind regards, > Geert > > > > -----Original Message----- > > From: general-boun...@developer.marklogic.com > > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Frank > > Sanders > > Sent: Tuesday, February 10, 2009 1:23 PM > > To: General Mark Logic Developer Discussion > > Subject: RE: [MarkLogic Dev General] Atomic and > > exclusivelockondocuments(accross a cluster) > > > > Geert, > > > > Using a process similar to what Danny described, and you've already > > implemented, those (micro?)seconds shouldn't matter. > > As he mentioned, MarkLogic Server guarantees that for any > two queries > > trying to update a given document the second query will see the > > results of the first query. > > We can use this to our advantage, but we need to make sure > that we're > > seen as an update query by the server: > > > > let $force-update-mode := > xdmp:document-insert("/mutex-doc.xml", <foo > > />) let $user := "user-the-first" > > return > > (: do lock checking and updating here :) > > > > For this to be performant you'll have to create multiple mutex docs > > that in some way correspond to the documents you wish to > update. But > > all queries that want to update a given document are going > to have to > > block on the call to > > xdmp:document-insert() until the current query, with the > lock on the > > mutex, has finished executing. > > > > -fs > > > > -----Original Message----- > > From: general-boun...@developer.marklogic.com > > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Geert > > Josten > > Sent: Tuesday, February 10, 2009 3:18 PM > > To: General Mark Logic Developer Discussion > > Subject: RE: [MarkLogic Dev General] Atomic and exclusive > > lockondocuments(accross a cluster) > > > > Hi Danny, > > > > Thanks for your input. > > > > Actually, we have similar logic, though we use the 'owner' > > property of locks themselves. It is more or less like this: > > > > Acquiring the lock: > > > > if (not(document-locks($uri))) then > > lock-acquire($uri, 'exclusive', 0, $owner, ()) else > > error('already locked') > > > > Performing update: > > > > if (document-locks($uri)//lock:owner = $owner) then > > do-update($uri) > > Else > > error('locked by someone else') > > > > The issue is in acquiring the lock. There is time between > the moment > > of checking for locks and acquiring the actual lock. > > It is very small (perhaps just a few CPU cycles), but time > nontheless, > > which leaves room for other threads or requests to make a > same call to > > lock-acquire at 'the same' time. > > Lock-acquire is atomic across a cluster, but since all lock-acquire > > calls are executed by the same user, it never fails, not > providing us > > the exclusive lock we hoped to get. > > > > Kind regards, > > Geert > > > > > -----Original Message----- > > > From: general-boun...@developer.marklogic.com > > > [mailto:general-boun...@developer.marklogic.com] On > Behalf Of Danny > > > Sokolsky > > > Sent: dinsdag 10 februari 2009 20:16 > > > To: General Mark Logic Developer Discussion > > > Subject: RE: [MarkLogic Dev General] Atomic and exclusive lock > > > ondocuments(accross a cluster) > > > > > > You can store your locks in a separate document (or in a > > property on > > > the same document), and have your application logic only > > allow updates > > > > > if the "user" (that is, from your app's perspective, > since you are > > > only using one MarkLogic > > > user) is the one with the lock. The main difference > > between this and > > > the lock-acquire approach is that your application must > manage the > > > locks. > > > > > > If you implement this correctly, it should solve the > problem of the > > > second update request seeing what happened from the first update > > > request. Internally in MarkLogic Server, updates must > get a write > > > lock on a document before it can be updated. If someone > > else has the > > > write lock, then another update request must wait for the > > lock to be > > > released before it can proceed with its transaction. > > > > > > Imagine this scenario: > > > > > > /doc/mydoc.xml has the following property: > > > > > > <owner>user-a</owner> > > > > > > Your application code has a variable $user that is set to > > the current > > > user. Before any updates can occur, your code has some logic like > > > this: > > > > > > let $user := "user-a" > > > return > > > if ($user ne fn:data( > > > xdmp:document-get-properties("/doc/mydoc.xml", fn:QName("", > > > "owner"))) ) > > > then fn:error("NOUPDATE", "You cannot update because you > > are the wrong > > > user") > > > else "put update code here" > > > > > > I might be missing something, but it seems like this > approach would > > > work. Of course implementing a content management system is > > > complicated, and you might find that managing the security in the > > > application layer will quickly get very complex. For the > > simple case, > > > > > however, it seems doable from my naive perspective. > > > > > > -Danny > > > > > > -----Original Message----- > > > From: general-boun...@developer.marklogic.com > > > [mailto:general-boun...@developer.marklogic.com] On > Behalf Of Geert > > > Josten > > > Sent: Monday, February 09, 2009 11:52 PM > > > To: General Mark Logic Developer Discussion > > > Subject: RE: [MarkLogic Dev General] Atomic and exclusive lock > > > ondocuments(accross a cluster) > > > > > > Hi Danny, > > > > > > Tjerk is my colleague. We already have something like that, > > using the > > > owner field of locks themselves, as we needed locks over > > > multi-transactions anyhow. But we need absolute 100 percent > > guarantee > > > that if two requests are challenging for an exclusive lock > > on the same > > > > > document, only one can be acquired. Unfortunately, since we > > are using > > > an application user, the xdmp:lock-acquire function never > > fails. And > > > however small, there is time between checking for a lock > > and acquiring > > > > > the lock itself, so leaving a tiny gap for two requests to > > get a lock > > > on the same document. > > > > > > Moreover, I noticed that the first lock is being > overwritten by the > > > second, effectively extending the lock-period (which could be > > > usefull). > > > If the second request comes from some other user, and the first > > > response is is a success, the evidence of the first request > > will get > > > lost. We really would like to prevent this to happen. > > > > > > One workaround we thought of ourselves is to create > temporary users > > > (one for each lock) and use xdmp:eval to acquire the lock as that > > > user. But that sounds like unnecessary overhead and is > likely to be > > > error prone (error handling, releasing locks and removing > > those temp > > > users is tricky). > > > > > > Any suggestion at all would be welcome.. > > > > > > Kind regards, > > > Geert > > > > > > > -----Original Message----- > > > > From: general-boun...@developer.marklogic.com > > > > [mailto:general-boun...@developer.marklogic.com] On > > Behalf Of Danny > > > > Sokolsky > > > > Sent: maandag 9 februari 2009 20:07 > > > > To: General Mark Logic Developer Discussion > > > > Subject: RE: [MarkLogic Dev General] Atomic and > exclusive lock on > > > > documents(accross a cluster) > > > > > > > > Hi Tjerk, > > > > > > > > One approach you can take is to store the current owner of the > > > > document somewhere in the document, or in the document > properties. > > > > Then, in your application, you can have logic that only > > allows that > > > > user to update the document. For example, if you had a > > > property that > > > > looked something like > > > > this: > > > > > > > > <owner>user-a</owner> > > > > > > > > Then you might have some logic in your update module that > > > checked that > > > > the user was user-a before allowing him to update the > > document. Of > > > > course it would be up to you to write that logic correctly. > > > > > > > > During the update in MarkLogic Server, the document is > > > locked across > > > > the cluster--that is a key part of the transaction semantics of > > > > MarkLogic Server. > > > > > > > > -Danny > > > > > > > > -----Original Message----- > > > > From: general-boun...@developer.marklogic.com > > > > [mailto:general-boun...@developer.marklogic.com] On > > Behalf Of Tjerk > > > > Stroband > > > > Sent: Monday, February 09, 2009 6:31 AM > > > > To: general@developer.marklogic.com > > > > Subject: [MarkLogic Dev General] Atomic and exclusive lock on > > > > documents(accross a cluster) > > > > > > > > Hi all, > > > > > > > > During development of a particular application a number of > > > > architectural design decisions have been made. One of these > > > decisions > > > > was to not used the internal ML user-database since that > > > would require > > > > the client application to do user-impersonation etc and require > > > > complex synchronization procedures with the LDAP (which > > is used to > > > > store all the users). Instead the application will use a single > > > > application user-account to access ML. > > > > > > > > However, the application requires support for exclusively and > > > > atomically (across the cluster) locking documents > > > (regardless of the > > > > user since only one user accesses ML) as part of a pessimistic > > > > locking-scheme for use by an external workflow. > > > > The logical way to achieve this would be to use the > > > xdmp:lock-acquire > > > > and xdmp:lock-release functions. However, these functions > > > only lock a > > > > document for a particular user. > > > > Two consecutive calls to the xdmp:lock-acquire function > > > using the same > > > > user will not cause the second call to fail, regardless of the > > > > description. > > > > Instead they both succeed and the description will have > > that of the > > > > last call. > > > > > > > > This poses a significant challenge as it seems it is > > > impossible to get > > > > an exclusive and atomic lock on a document when not using > > > named users > > > > in ML. > > > > > > > > Is our analysis correct or are we missing something? > > > > > > > > Best regards, > > > > > > > > Tjerk Stroband > > > > Daidalos > > > > The Netherlands > > > > > > > > > > > > > > > > T. I. Stroband > > > > Senior consultant > > > > GSM: +31 (0)6 22 52 44 45 > > > > > > > > > > > > Daidalos BV > > > > Source of Innovation > > > > Hoekeindsehof 1-4 > > > > 2665 JZ Bleiswijk > > > > Tel.: +31 (0) 10 850 1200 > > > > Fax: +31 (0) 10 850 1199 > > > > http://www.daidalos.nl/ > > > > KvK 27164984 > > > > De informatie - verzonden in of met dit emailbericht - is > > afkomstig > > > > van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. > > > > Indien u dit bericht onbedoeld hebt ontvangen, verzoeken > > > wij u het te > > > > verwijderen. Aan dit bericht kunnen geen rechten worden > ontleend. > > > > > > > > > > > > > > > > _______________________________________________ > > > > General mailing list > > > > General@developer.marklogic.com > > > > http://xqzone.com/mailman/listinfo/general > > > > _______________________________________________ > > > > General mailing list > > > > General@developer.marklogic.com > > > > http://xqzone.com/mailman/listinfo/general > > > > _______________________________________________ > > > General mailing list > > > General@developer.marklogic.com > > > http://xqzone.com/mailman/listinfo/general > > > _______________________________________________ > > > General mailing list > > > General@developer.marklogic.com > > > http://xqzone.com/mailman/listinfo/general > > > _______________________________________________ > > General mailing list > > General@developer.marklogic.com > > http://xqzone.com/mailman/listinfo/general > > _______________________________________________ > > General mailing list > > General@developer.marklogic.com > > http://xqzone.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > General@developer.marklogic.com > http://xqzone.com/mailman/listinfo/general > _______________________________________________ > General mailing list > General@developer.marklogic.com > http://xqzone.com/mailman/listinfo/general > _______________________________________________ General mailing list General@developer.marklogic.com http://xqzone.com/mailman/listinfo/general
