There's no way to rollback an put/putAll, unless in TX. On Sun, Sep 10, 2017 at 4:21 PM, Jason Huynh <jhu...@pivotal.io> wrote:
> 1.) Does anyone know of a way to do a rollback where the put is already > reflected in the region? If that is the desired behavior, then perhaps we > will have to live with the current (leaving the region and indexes in a bad > state, wan and other callbacks that occur after index maintenance will not > occur for the one operation but the put has made it into the region) until > someone can figure out how to roll a put back and revert the update to all > the indexes. > > How should this affect putAll, if at all? > > Any callbacks that occur before index update have already been called > (cache writers?). I am not sure how those should be affected by a > rollback... > > 2.) So the index behavior changes if they are marked for sync/async. In > sync the index would reject the put, but in async they would just be marked > as invalid. > > > > > On Sat, Sep 9, 2017 at 6:48 AM John Blum <jb...@pivotal.io> wrote: > > > +1 to both of Anil's points. > > > > On Fri, Sep 8, 2017 at 3:04 PM, Anilkumar Gingade <aging...@pivotal.io> > > wrote: > > > > > Indexes are critical for querying; most of the databases doesn't allow > > > insert/update if there is any failure with index maintenance... > > > > > > As Geode OQL supports two ways (sync and async) to maintain the > indexes, > > we > > > need be careful about the error handling in both cases... > > > > > > My take is: > > > 1. For synchronous index maintenance: > > > If there is any failure in updating any index (security/auth or logical > > > error) on the region; throw an exception and rollback the cache > update/op > > > (index management id done under region.entry lock - we should be able > to > > > revert the op). If index or cache is left in bad state, then its a bug > > that > > > needs to be addressed. > > > > > > Most of the time, If there is any logical error in index, it will be > > > detected as soon as index is created (on existing data) or when first > > > update is done to the cache. > > > > > > 2. For Asynchronous index maintenance: > > > As this is async (assuming) user has good understanding of the risk > > > involved with async, any error with index maintenance, the index should > > be > > > invalidated... > > > > > > About the security/auth, the user permission with region read/write > > needs > > > to be applied for index updates, there should not be different > permission > > > on index. > > > > > > -Anil. > > > > > > > > > > > > On Fri, Sep 8, 2017 at 2:01 PM, Nabarun Nag <n...@pivotal.io> wrote: > > > > > > > Hi Mike, > > > > > > > > Please do find our answers below: > > > > *Question:* What if there were multiple indices that were in flight > and > > > > only the third > > > > one errors out, will they all be marked invalid? > > > > > > > > *Answer:* Only the third will be marked invalid and only the third > one > > > will > > > > not be used for query execution. > > > > > > > > *Question/Statement:* If anything goes wrong with the put it should > > > > probably still throw back to > > > > the caller. Silent invalidation of the index is probably not > desirable. > > > > > > > > *Answer: * > > > > In our current design this the flow of execution of a put operation: > > > > entry put into region -> update index -> other wan related > executions / > > > > callbacks etc. > > > > > > > > If an exception happens while updating the index, the cache gets > into a > > > bad > > > > state, and we may end up getting different results depending on the > > index > > > > we are using. As the failure happens half way in a put operation, the > > > > regions / cache are now in a bad state. > > > > -------------------------- > > > > We are thinking that if index is created over a method invocation in > > an > > > > empty region and then we do puts, but method invocation is not > allowed > > as > > > > per security policies. The puts will now be successful but the index > > will > > > > be rendered invalid. Previously the puts will fail with exception and > > put > > > > the entire cache in a bad state. > > > > > > > > > > > > > > > > Regards > > > > Nabarun > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2017 at 10:43 AM Michael Stolz <mst...@pivotal.io> > > wrote: > > > > > > > > > Just to help me understand, the index is corrupted in a way beyond > > just > > > > the > > > > > field that errors out? > > > > > What if there were multiple indices that were in flight and only > the > > > > third > > > > > one errors out, will they all be marked invalid? > > > > > If anything goes wrong with the put it should probably still throw > > back > > > > to > > > > > the caller. Silent invalidation of the index is probably not > > desirable. > > > > > > > > > > -- > > > > > Mike Stolz > > > > > Principal Engineer, GemFire Product Manager > > > > > Mobile: +1-631-835-4771 <(631)%20835-4771> <(631)%20835-4771> > > > > > > > > > > On Fri, Sep 8, 2017 at 12:34 PM, Dan Smith <dsm...@pivotal.io> > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > -Dan > > > > > > > > > > > > On Thu, Sep 7, 2017 at 9:14 PM, Nabarun Nag <n...@apache.org> > > wrote: > > > > > > > > > > > > > *Proposal:* > > > > > > > * Index interface will include an API - isValid() which will > > return > > > > > true > > > > > > if > > > > > > > the index is still valid / uncorrupted, else will return false > if > > > it > > > > > > > corrupted / invalid. > > > > > > > * gfsh command "list index" will have one more column "Is > Valid" > > > > which > > > > > > will > > > > > > > state the status as "true" or "false". > > > > > > > * Invalid indexes will not be used during query executions. > > > > > > > * In case the index is found to be invalid, the user will be > able > > > to > > > > > > > remove/destroy the index. > > > > > > > * When a put operation corrupts an index, it will be logged. > > > > > > > > > > > > > > *Reasoning:* > > > > > > > * Currently if a put operation raises an exception while > updating > > > the > > > > > > > index, the put operation fails with an exception to the putter. > > > > > > > * For example, if an index is created on an object method, and > > that > > > > > > method > > > > > > > causes an exception while updating the index as a part of a put > > > > > > operation, > > > > > > > then the put operation fails for that particular entry and the > > > index > > > > is > > > > > > > left in a bad state. > > > > > > > * This may occur also due to lack of permission to update index > > but > > > > > have > > > > > > > permission to do puts. > > > > > > > * We are proposing that in the above mentioned scenarios, the > put > > > > > > succeeds > > > > > > > in putting the entry in the region but the index which it was > > > trying > > > > to > > > > > > > update will be marked invalid and will not be used for query > > > > > executions. > > > > > > > * This can be justified because the corrupted index will not > > > > correctly > > > > > > > represent the region entries. > > > > > > > > > > > > > > *Status Quo:* > > > > > > > * Index creation will still fail if we are trying to create an > > > index > > > > > > over a > > > > > > > region containing an entry/entries which will cause an > exception > > > > while > > > > > > > loading the entry into the index. > > > > > > > > > > > > > > Regards > > > > > > > Nabarun Nag > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > -John > > john.blum10101 (skype) > > >
