Re: Atomic updates to increase single field bulk updates?

2017-02-17 Thread Erick Erickson
Well, "it depends". The Atomic update has to first go out to disk and decompress the original stored fields in 16K blocks, then overlay the atomic update on the uncompressed doc, then re-index the doc. 40K times in your example. So yes, the stream going to Solr will be smaller if you do atomic

Re: Atomic updates to increase single field bulk updates?

2017-02-17 Thread Bram Van Dam
> I am aware of the requirements to use atomic updates, but as I understood, > those would not have a big impact on performance and only a slight increase > in index size? AFAIK there won't be a difference in index size between atomic updates and full updates, as the end result is the same.

RE: Atomic updates to increase single field bulk updates?

2017-02-16 Thread Chris Hostetter
: partial update or a complete document. Under the hood a partial update : is a complete object anyway. Using partial updates you gain a little : bandwidth at the expense of additional stored fields. FWIW: once SOLR-5944 lands in a released version, that won't always be true -- atomic updates

RE: Atomic updates to increase single field bulk updates?

2017-02-15 Thread Markus Jelsma
mer <s.rie...@littera.eu> > Sent: Wednesday 15th February 2017 19:31 > To: solr-user@lucene.apache.org > Subject: Atomic updates to increase single field bulk updates? > > Dear solr users, > > when updating documents in bulk (i.e. 40.000 documents at once), and only &g

Atomic updates to increase single field bulk updates?

2017-02-15 Thread Sebastian Riemer
Dear solr users, when updating documents in bulk (i.e. 40.000 documents at once), and only changing the value of a single Boolean-Flag, I currently re-index all whole 40.000 objects. However, the process of obtaining all relevant information for each object from the database is one of