Re: Custom update processor and race condition with concurrent requests

2020-03-04 Thread Sachin Divekar
Thanks, Chris. I think I should stop thinking about doing it in Solr. Anyway, I was just trying to see how far I can go. On Wed, Mar 4, 2020 at 11:50 PM Chris Hostetter wrote: > > : So, I thought it can be simplified by moving this state transitions and > : processing logic into Solr by

Re: Custom update processor and race condition with concurrent requests

2020-03-04 Thread Chris Hostetter
: So, I thought it can be simplified by moving this state transitions and : processing logic into Solr by writing a custom update processor. The idea : occurred to me when I was thinking about Solr serializing multiple : concurrent requests for a document on the leader replica. So, my thought :

Re: Custom update processor and race condition with concurrent requests

2020-03-04 Thread Walter Underwood
o achieve? >> >> https://people.apache.org/~hossman/#xyproblem >> XY Problem >> >> Your question appears to be an "XY Problem" ... that is: you are dealing >> with "X", you are assuming "Y" will help you, and you are asking about "Y&qu

Re: Custom update processor and race condition with concurrent requests

2020-03-04 Thread Sachin Divekar
So, the idea was that Solr serializes writes while appending records to tlogs. Combining that with realtime gets to make decisions to modify the request would achieve the result. > I do wonder if it’s possible to insure that a given doc is always updated from the same thread? I’m assuming that

Re: Custom update processor and race condition with concurrent requests

2020-03-03 Thread Sachin Divekar
all? > See Also: http://www.perlmonks.org/index.pl?node_id=542341 > > > > > > : Date: Tue, 3 Mar 2020 23:52:38 +0530 > : From: Sachin Divekar > : Reply-To: solr-user@lucene.apache.org > : To: solr-user@lucene.apache.org > : Subject: Re: Custom update processor and race

Re: Custom update processor and race condition with concurrent requests

2020-03-03 Thread Erick Erickson
I guess I’m missing something. Assuming that S1 and S2 are sent in different batches from different threads from your client, there are any number of ways they could arrive out of order. Network delays, client delays, etc. So I don’t se any way to serialize them reliably. If they’re sent

Re: Custom update processor and race condition with concurrent requests

2020-03-03 Thread Chris Hostetter
tand the full issue. Perhaps the best solution doesn't involve "Y" at all? See Also: http://www.perlmonks.org/index.pl?node_id=542341 : Date: Tue, 3 Mar 2020 23:52:38 +0530 : From: Sachin Divekar : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: Re:

Re: Custom update processor and race condition with concurrent requests

2020-03-03 Thread Sachin Divekar
Thank, Erick. I think I was not clear enough. With the custom update processor, I'm not using optimistic concurrency at all. The update processor just modifies the incoming document with updated field values and atomic update instructions. It then forwards the modified request further in the

Re: Custom update processor and race condition with concurrent requests

2020-03-03 Thread Erick Erickson
I don’t really see how this test setup can work, I think you’re just getting lucky with the 4 threads. But let’s be specific about what optimistic concurrency is. If you update a document that has a _version_ field, and that document already exists with a value in the _version_ field higher

Custom update processor and race condition with concurrent requests

2020-03-03 Thread Sachin Divekar
Hi, We are using Solr where there are many update operations. This may not be the right use case for Solr but it's an old application and at this moment we are in no mood to replace Solr with something else. For one of our use case, we had to use optimistic concurrency for handling concurrent