An approach that comes immediately to mind is implementing a lock mechanism. Write a "lock object" to the space (either a global one or a lock for each E) and allow S or P to do anything with any E only when they've successfully taken the lock.
With an appropriate transaction and lease time-outs, you should be able to recover pretty easily if either S or P fail at any stage. S must always write E first and then the lock for E. For example, P takes the lock under a transaction and then fails. So the transaction is rolled back, and the lock is replaced allowing another P to begin processing. Before S re-writes E to the space, it must attempt to take the lock. If there is no lock and E is still present in the space, then there is a P still processing it (or a failed P and the transaction is being rolled back or the lease it waiting to expire.) Once P is finished it can remove E from the space. The biggest problem I can see here is the situation where S writes E for the first time but then fails to write the lock for it. Perhaps these two writes can be done under the same transaction to make them atomic? I'm sure there are some gaps and edge cases here, but synchronizing on some lock object is fairly simple. On Wed, Jan 7, 2015 at 9:58 AM, Zsolt Kúti <[email protected]> wrote: > ( I've just realyzed, that my letter went to the former incubator address > which still exists. So resend it to the current list.) > > > Hi, > > I turn with my problem to you experienced people for help. > It seems to be a basic problem still cant see the correct solution. > > Consider the following requirement: > - given two services, one of them (S) periodically wants to write entries > (Es) > for ones (P) that consume and make processing based on Es > - processing may take so long that a new E that would induce (almost) the > same processing may arrive in the meantime and its writing to the space > must be prevented until the previous E's processing is finished > > > All the schemes I could come up till now involves a small time window where > problem can occur. > One of them is: > - E has a state field, that can have either TO_BE_PROCESSED, or PROCESSING > - S reads if E of TO_BE_PROCESSED exists, if not, writes one of that state > - a P takes E with this state (thereby not letting other Ps to chip-in), > changes E to PROCESSING, > and writes back; when finished removes E > > Here in the last step there is a small chance for S where it cannot see > that E and writes a new one, so an other P will start a new useless > processing. > > I would appreciate your suggestions. > > Zsolt >
