Hi, Not claiming that is the case here, but I think I've read some comments (I think in some Jira) that suggest that inplace updates are not as cheap as one might think and that does not reindex dox but dies rewrite of doc values for the segment. Did not look at code, but if someone is familiar with this, can one please jump in and comment hiw cheap in place updates are.
Thanks, Emir On Oct 17, 2017 2:11 PM, "James" <ja...@ohrt.info> wrote: > I found a solution which works for me: > > Add a document with very little tokenized text and write down QTime (for > me: 5ms) > Add another document with very much text (I used about 1MB of Lorem Ipsum > sample text) and write down QTime (for me: 70ms). > Perform an update operation on document 2 which you want to test whether > it is "in-place" and compare QTime. > For me it was again 70ms. So I assume that my operation did re-index the > whole document and was thus not an in-place update. > > > -----Ursprüngliche Nachricht----- > Von: Amrit Sarkar [mailto:sarkaramr...@gmail.com] > Gesendet: Dienstag, 17. Oktober 2017 12:43 > An: solr-user@lucene.apache.org > Betreff: Re: Howto verify that update is "in-place" > > James, > > @Amrit: Are you saying that the _version_ field should not change when > > performing an atomic update operation? > > > It should change. a new version will be allotted to the document. I am not > that sure about in-place updates, probably a test run will verify that. > > Amrit Sarkar > Search Engineer > Lucidworks, Inc. > 415-589-9269 > www.lucidworks.com > Twitter http://twitter.com/lucidworks > LinkedIn: https://www.linkedin.com/in/sarkaramrit2 > > On Tue, Oct 17, 2017 at 4:06 PM, James <ja...@ohrt.info> wrote: > > > Hi Emir and Amrit, thanks for your reponses! > > > > @Emir: Nice idea but after changing any document in any way and after > > committing the changes, all Doc counter (Num, Max, Deleted) are still > > the same, only thing that changes is the Version (increases by steps of > 2) . > > > > @Amrit: Are you saying that the _version_ field should not change when > > performing an atomic update operation? > > > > Thanks > > James > > > > > > -----Ursprüngliche Nachricht----- > > Von: Amrit Sarkar [mailto:sarkaramr...@gmail.com] > > Gesendet: Dienstag, 17. Oktober 2017 11:35 > > An: solr-user@lucene.apache.org > > Betreff: Re: Howto verify that update is "in-place" > > > > Hi James, > > > > As for each update you are doing via atomic operation contains the > > "id" / "uniqueKey". Comparing the "_version_" field value for one of > > them would be fine for a batch. Rest, Emir has list them out. > > > > Amrit Sarkar > > Search Engineer > > Lucidworks, Inc. > > 415-589-9269 > > www.lucidworks.com > > Twitter http://twitter.com/lucidworks > > LinkedIn: https://www.linkedin.com/in/sarkaramrit2 > > > > On Tue, Oct 17, 2017 at 2:47 PM, Emir Arnautović < > > emir.arnauto...@sematext.com> wrote: > > > > > Hi James, > > > I did not try, but checking max and num doc might give you info if > > > update was in-place or atomic - atomic is reindexing of existing doc > > > so the old doc will be deleted. In-place update should just update > > > doc values of existing doc so number of deleted docs should not change. > > > > > > HTH, > > > Emir > > > -- > > > Monitoring - Log Management - Alerting - Anomaly Detection Solr & > > > Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > > > > > > > > > On 17 Oct 2017, at 09:57, James <ja...@ohrt.info> wrote: > > > > > > > > I am using Solr 6.6 and carefully read the documentation about > > > > atomic and in-place updates. I am pretty sure that everything is > > > > set up as it > > > should. > > > > > > > > > > > > > > > > But how can I make certain that a simple update command actually > > > performs an > > > > in-place update without internally re-indexing all other fields? > > > > > > > > > > > > > > > > I am issuing this command to my server: > > > > > > > > (I am using implicit document routing, so I need the "Shard" > > > > parameter.) > > > > > > > > > > > > > > > > { > > > > > > > > "ID":1133, > > > > > > > > "Property_2":{"set":124}, > > > > > > > > "Shard":"FirstShard" > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > The log outputs: > > > > > > > > > > > > > > > > 2017-10-17 07:39:18.701 INFO (qtp1937348256-643) [c:MyCollection > > > > s:FirstShard r:core_node27 x:MyCollection_FirstShard_replica1] > > > > o.a.s.u.p.LogUpdateProcessorFactory > > > > [MyCollection_FirstShard_replica1] > > > > webapp=/solr path=/update > > > > params={commitWithin=1000&boost=1.0&overwrite=true&wt= > > > json&_=1508221142230}{ > > > > add=[1133 (1581489542869811200)]} 0 1 > > > > > > > > 2017-10-17 07:39:19.703 INFO (commitScheduler-283-thread-1) > > > [c:MyCollection > > > > s:FirstShard r:core_node27 x:MyCollection_FirstShard_replica1] > > > > o.a.s.u.DirectUpdateHandler2 start > > > > commit{,optimize=false,openSearcher=false,waitSearcher=true, > > > expungeDeletes=f > > > > alse,softCommit=true,prepareCommit=false} > > > > > > > > 2017-10-17 07:39:19.703 INFO (commitScheduler-283-thread-1) > > > [c:MyCollection > > > > s:FirstShard r:core_node27 x:MyCollection_FirstShard_replica1] > > > > o.a.s.s.SolrIndexSearcher Opening > > > > [Searcher@32d539b4[MyCollection_FirstShard_replica1] main] > > > > > > > > 2017-10-17 07:39:19.703 INFO (commitScheduler-283-thread-1) > > > [c:MyCollection > > > > s:FirstShard r:core_node27 x:MyCollection_FirstShard_replica1] > > > > o.a.s.u.DirectUpdateHandler2 end_commit_flush > > > > > > > > 2017-10-17 07:39:19.703 INFO > > > > (searcherExecutor-268-thread-1-processing-n:192.168.117.142:8983_s > > > > ol > > > > r > > > > x:MyCollection_FirstShard_replica1 s:FirstShard c:MyCollection > > > > r:core_node27) [c:MyCollection s:FirstShard r:core_node27 > > > > x:MyCollection_FirstShard_replica1] o.a.s.c.QuerySenderListener > > > > QuerySenderListener sending requests to > > > > Searcher@32d539b4[MyCollection_FirstShard_replica1] > > > > main{ExitableDirectoryReader(UninvertingDirectoryReader( > > > Uninverting(_i(6.6.0 > > > > ):C5011/1) Uninverting(_j(6.6.0):C478) Uninverting(_k(6.6.0):C345) > > > > Uninverting(_l(6.6.0):C4182) Uninverting(_m(6.6.0):C317) > > > > Uninverting(_n(6.6.0):C399) Uninverting(_q(6.6.0):C1)))} > > > > > > > > 2017-10-17 07:39:19.703 INFO > > > > (searcherExecutor-268-thread-1-processing-n:192.168.117.142:8983_s > > > > ol > > > > r > > > > x:MyCollection_FirstShard_replica1 s:FirstShard c:MyCollection > > > > r:core_node27) [c:MyCollection s:FirstShard r:core_node27 > > > > x:MyCollection_FirstShard_replica1] o.a.s.c.QuerySenderListener > > > > QuerySenderListener done. > > > > > > > > 2017-10-17 07:39:19.703 INFO > > > > (searcherExecutor-268-thread-1-processing-n:192.168.117.142:8983_s > > > > ol > > > > r > > > > x:MyCollection_FirstShard_replica1 s:FirstShard c:MyCollection > > > > r:core_node27) [c:MyCollection s:FirstShard r:core_node27 > > > > x:MyCollection_FirstShard_replica1] o.a.s.c.SolrCore > > > > [MyCollection_FirstShard_replica1] Registered new searcher > > > > Searcher@32d539b4[MyCollection_FirstShard_replica1] > > > > main{ExitableDirectoryReader(UninvertingDirectoryReader( > > > Uninverting(_i(6.6.0 > > > > ):C5011/1) Uninverting(_j(6.6.0):C478) Uninverting(_k(6.6.0):C345) > > > > Uninverting(_l(6.6.0):C4182) Uninverting(_m(6.6.0):C317) > > > > Uninverting(_n(6.6.0):C399) Uninverting(_q(6.6.0):C1)))} > > > > > > > > > > > > > > > > If I issue another, non-in-place update to another field which is > > > > not a DocValue, the log output is very similar. Can I increase > > > > verbosity? Will > > > it > > > > tell me more about the type of update then? > > > > > > > > > > > > > > > > Thank you! > > > > > > > > James > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
