Re: Solr Atomic Updates by Query

2015-06-04 Thread Erick Erickson
Not to my knowledge. In Solr terms this would be a _very_ heavyweight operation, potentially re-indexing millions and millions of documents. Imagine if your q were id:* for instance. Plus routing that to all shards and dealing with other updates coming in would be a nightmare. Best, Erick On

Re: Solr Atomic Updates

2015-06-04 Thread Erick Erickson
NP. It's something of a step when moving to SolrCloud to let go of the details you've had to (painfully) pay attention to, but worth it. The price is, of course, learning to do things a new way ;)... Best, Erick On Thu, Jun 4, 2015 at 10:04 AM, Ксения Баталова batalova...@gmail.com wrote:

Re: Solr Atomic Updates by Query

2015-06-04 Thread Ксения Баталова
Is it planned soon? Or may be not soon.. _ _ _ Batalova Kseniya There is no equivalent of, say a SQL update...where... so no, atomic updates by query... Best, Erick On Thu, Jun 4, 2015 at 2:49 AM, Ксения Баталова batalova...@gmail.com wrote: Hi! I have one more question about atomic

Re: Solr Atomic Updates

2015-06-04 Thread Ксения Баталова
Hope I'll succeed) Anyway, solr-user community surprised me in a good way. Thanks again. _ _ Batalova Kseniya NP. It's something of a step when moving to SolrCloud to let go of the details you've had to (painfully) pay attention to, but worth it. The price is, of course, learning to do

Re: Solr Atomic Updates

2015-06-04 Thread Ксения Баталова
Erick, Thank you so much. It became a bit clearer. It was decided to upgrade Solr to 5.2 and use SolrCloud in our next release. I think I'll write here about it yet :) _ _ Batalova Kseniya I have to ask then why you're not using SolrCloud with multiple shards? It seems to me that that gives

Re: Solr Atomic Updates by Query

2015-06-04 Thread Erick Erickson
There is no equivalent of, say a SQL update...where... so no, atomic updates by query... Best, Erick On Thu, Jun 4, 2015 at 2:49 AM, Ксения Баталова batalova...@gmail.com wrote: Hi! I have one more question about atomic updates in Solr (Solr 4.4.0). Is it posible to generate atomic update by

Re: Solr Atomic Updates by Query

2015-06-04 Thread Ксения Баталова
Oh, I see. May be it's not such a good idea.) Thanks. _ _ Batalova Kseniya Not to my knowledge. In Solr terms this would be a _very_ heavyweight operation, potentially re-indexing millions and millions of documents. Imagine if your q were id:* for instance. Plus routing that to all shards

Solr Atomic Updates by Query

2015-06-04 Thread Ксения Баталова
Hi! I have one more question about atomic updates in Solr (Solr 4.4.0). Is it posible to generate atomic update by query? I mean I want to update those documents in which IDs contain some string. For example, index has: Doc1, id=123|a,b Doc2, id=123|a,c Doc3, id=345|a,b Doc4, id=345|a,c,d. And

Re: Solr Atomic Updates

2015-06-03 Thread Erick Erickson
Basically, I think about using SolrCloud whenever you have to split your corpus into more than one core (shard in SolrCloud terms). Or when you require fault tolerance in terms of machines going up and down. Despite the name, it does _not_ require AWS or similar, and you can run SolrCloud on a

Re: Solr Atomic Updates

2015-06-03 Thread Jack Krupansky
BTW, does anybody know how SolrCloud got that name? I mean, SolrCluster would make a lot more sense since a cloud is typically a very large collection of machines and more of a place than a specific configuration, while a Solr deployment is more typically a more modest number of machines, a

Re: Solr Atomic Updates

2015-06-03 Thread Shawn Heisey
On 6/3/2015 2:19 PM, Jack Krupansky wrote: BTW, does anybody know how SolrCloud got that name? I mean, SolrCluster would make a lot more sense since a cloud is typically a very large collection of machines and more of a place than a specific configuration, while a Solr deployment is more

Re: Solr Atomic Updates

2015-06-03 Thread Jack Krupansky
Explain a little about why you have separate cores, and how you decide which core a new document should reside in. Your scenario still seems a bit odd, so help us understand. -- Jack Krupansky On Wed, Jun 3, 2015 at 3:15 AM, Ксения Баталова batalova...@gmail.com wrote: Hi! Thanks for your

Re: Solr Atomic Updates

2015-06-03 Thread Upayavira
If you are using stand-alone Solr instances, then it is your responsibility to decide which node a document resides in, and thus to which core you will send your update request. If, however, you used SolrCloud, it would handle that for you - deciding which node should contain a document, and

Re: Solr Atomic Updates

2015-06-03 Thread Ксения Баталова
Hi! Thanks for your quick reply. The problem that all my index is consists of several parts (several cores) and while updating I don't know in advance in which part updated id is lying (in which core the document with specified id is lying). For example, I have two cores (*Core1 *and *Core2*)

Re: Solr Atomic Updates

2015-06-03 Thread Ксения Баталова
Upayavira, I'm using stand-alone Solr instances. I've not learnt SolrCloud yet. Please, give me some advice when SolrCloud is better then stand-alone Solr instances. Or when it is worth to choose SolrCloud. _ _ _ Batalova Kseniya If you are using stand-alone Solr instances, then it is your

Re: Solr Atomic Updates

2015-06-03 Thread Erick Erickson
I have to ask then why you're not using SolrCloud with multiple shards? It seems to me that that gives you the indexing throughput you need (be sure to use CloudSolrServer from your client). At 300M complex documents, you pretty much certainly will need to shard anyway so in some sense you're

Re: Solr Atomic Updates

2015-06-03 Thread Ксения Баталова
Jack, Decision of using several cores was made to increase indexing and searching performance (experimentally). In my project index is about 300-500 millions documents (each document has rather difficult structure) and it may be larger. So, while indexing the documents are being added in

Solr Atomic Updates

2015-06-02 Thread Ксения Баталова
Hi! I'm using *SOLR 4.4.0* for searching in my project. Now I am facing a problem of atomic updates in multiple cores. From wiki: curl *http://localhost:8983/solr/update http://localhost:8983/solr/update *-H 'Content-type:application/json' -d ' [ { *id*: *TestDoc1*, title :

Re: Solr Atomic Updates

2015-06-02 Thread Jack Krupansky
What exactly is the problem? And why do you care about cores, per se - other than to send the update to the core/collection you are trying to update? You should specify the core/collection name in the URL. You should also be using the Solr reference guide rather than the (old) wiki:

Re: Solr atomic updates question

2014-07-09 Thread Steve McKay
Right. Without atomic updates, the client needs to fetch the document (or rebuild it from the system of record), apply changes, and send the entire document to Solr, including fields that haven't changed. With atomic updates, the client sends a list of changes to Solr and the server handles the

Solr atomic updates question

2014-07-08 Thread Bill Au
Solr atomic update allows for changing only one or more fields of a document without having to re-index the entire document. But what about the case where I am sending in the entire document? In that case the whole document will be re-indexed anyway, right? So I assume that there will be no

Re: Solr atomic updates question

2014-07-08 Thread Steve McKay
Atomic updates fetch the doc with RealTimeGet, apply the updates to the fetched doc, then reindex. Whether you use atomic updates or send the entire doc to Solr, it has to deleteById then add. The perf difference between the atomic updates and normal updates is likely minimal. Atomic updates

Re: Solr atomic updates question

2014-07-08 Thread Bill Au
Thanks for that under-the-cover explanation. I am not sure what you mean by mix atomic updates with regular field values. Can you give an example? Thanks. Bill On Tue, Jul 8, 2014 at 6:56 PM, Steve McKay st...@b.abbies.us wrote: Atomic updates fetch the doc with RealTimeGet, apply the

Re: Solr atomic updates question

2014-07-08 Thread Steve McKay
Take a look at this update XML: add doc field name=employeeId05991/field field name=employeeNameSteve McKay/field field name=office update=setWalla Walla/field field name=skills update=addPython/field /doc /add Let's say employeeId is the key. If there's a fourth field,

Re: Solr atomic updates question

2014-07-08 Thread Bill Au
I see what you mean now. Thanks for the example. It makes things very clear. I have been thinking about the explanation in the original response more. According to that, both regular update with entire doc and atomic update involves a delete by id followed by a add. But both the Solr

Re: solr atomic updates stored=true, and copyField limitation

2013-09-21 Thread Shawn Heisey
On 9/19/2013 6:47 AM, Tanguy Moal wrote: Quoting http://wiki.apache.org/solr/Atomic_Updates#Caveats_and_Limitations : all fields in your SchemaXml must be configured as stored=true except for fields which are copyField/ destinations -- which must be configured as stored=false For fields

solr atomic updates stored=true, and copyField limitation

2013-09-19 Thread Tanguy Moal
Hello, I'm using solr 4.4. I have a solr core with a schema defining a bunch of different fields, and among them, a date field: - date: indexed and stored // the date used at search time In practice it's a TrieDateField but I think that's not relevant for the concern. It also has a multi