> -----Original Message----- > From: Julian Foad [mailto:julian.f...@wandisco.com] > Sent: donderdag 27 januari 2011 20:44 > To: Bert Huijben > Cc: 'Branko Čibej'; dev@subversion.apache.org > Subject: RE: Ref-counting for pristine texts > > On Thu, 2011-01-27, Bert Huijben wrote: > > > > > -----Original Message----- > > > From: Julian Foad [mailto:julian.f...@wandisco.com] > > > Sent: donderdag 27 januari 2011 18:06 > > > To: Bert Huijben > > > Cc: 'Branko Čibej'; dev@subversion.apache.org > > > Subject: RE: Ref-counting for pristine texts > > > > > > On Wed, 2011-01-26, Bert Huijben wrote: > > > > > -----Original Message----- > > > > > From: Julian Foad [mailto:julian.f...@wandisco.com] > > > > > > > > > > Can anyone help me work out the rules for guaranteeing > consistency > > > of > > > > > the pristine text store? > > > > > > > > Previously we used a somewhat reversed definition: as long as > there > > > > are work queue items or working-copy locks, you can't assume > > > pristines > > > > are unreferenced. > > > > > > Ah, WC-locks. Yes, that sounds more likely. (I don't recall that > we > > > have ever used that rule yet. The pristine cleanup code has only > ever > > > checked "WQ is empty", which isn't good enough.) > > > > > > > Note that there can be multiple clients accessing/modifying a > working > > > > copy at the same time even though they don't have a sqlite > > > transaction > > > > open. (An update doesn’t keep it's transaction open for the > entire > > > > update process). > > > > > > Yes. Nor does the update process keep work items items in the WQ > > > during > > > the whole time - it runs the WQ at various points. But there is a > WC > > > lock through the whole update process. > > > > > > So we can write a rule: > > > > > > "You may purge unreferenced pristines only when > > > there are no WC locks in the DB." > > > > > > More precisely, the access control rules for the pristine store in > a > > > given WC shall be: > > > > > > * A process may add a new (initially unreferenced) pristine text > > > to the store > > > IFF this process has a WC lock. > > > > > > * A process may add or remove references to any pristine text > that > > > is in the store > > > IFF this process has a WC lock. > > > > Maybe this one can be loosened a bit; not sure though... > (incrementing > > or decrementing a reference is probably safe when the other rules are > > followed) > > My thinking was that if I don't have a lock, I can't safely add a > reference because another process might delete the text just before I > get around to executing that statement. But in fact I don't > necessarily > need a WC lock, I need *either* a WC lock *or* a SQL txn. And within > that lock or txn I need to check the pristine text exists in the store > before I add the reference.
You can't change the database without a sql transaction. If you don't open a transaction yourself, SQLite will use one for you. So this is probably the same thing. > > To remove a reference, I can't think why I should need a lock or a txn. You can't change the database without using some kind of SQLite transaction. > > > > * A process that has a WC lock may assume that no pristine text, > > > even if unreferenced, will be deleted from the store as long as > > > this process holds any WC lock in this WC. > > > > > > * A process may purge an unreferenced pristine text > > > IFF no other process has a WC lock. > > > ### Must this process have a WC lock? > > > > > > (I use the word "process" loosely.) > > > > > > Does that make sense? > > > > I think it does. > > Thanks. > > Talking on IRC, we thought maybe involving the WC lock was making the > rules a bit too complex. It would be simpler to just define the rules > with respect to maintaining a valid DB state [1] outside any DB txn. > So > something like: > > * To add a new pristine text, you must add it and the NODES row > that references it within the same txn. Note that we also prepared referencing pristine from conflicts (= a row in ACTUAL_NODES with one of a handful of columns filled) > * To add a reference to an existing text, you must do so within > the same txn where you checked that it still exists. > > * You may remove a reference to a pristine text; no restrictions. > > * Any process may delete unreferenced pristine texts, as long as > the check for "unreferenced" and the deletion are in the same txn. > > However, to do this, we'd have to upgrade existing code to work like > this. Currently, some important code paths (update and commit) have > been designed to put unreferenced pristines into the store and then > later reference them. > > Because of this, I wonder whether it's better to continue down the > existing code's route and ensure unreferenced pristines are never > deleted while the process holds a WC lock. That would appear to be > simpler from the point of view of what changes are required to get it > working. > > > [1] A "valid DB state" meaning "a state that represents some possible > repository state, but not necessarily the actual repository state". > > > Also TODO: pristine_install() and pristine_remove() should do the > file-move-into-place or file-delete via the Work Queue. > > - Julian Bert