Re: Deleting by query
Thanks Jan, That worked. Kind Regards, Marc On 02/26/2016 01:43 PM, Jan Høydahl wrote: Hi Try this instead /solr/de/update?stream.body=last_seen:[* TO 2016-02-24T00:00:00Z]=true …that is if you have streaming enabled in solrconfig. Else do a POST instead Note that I put a commit=true at the end, so you will see the changes immediately. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com 26. feb. 2016 kl. 13.57 skrev Marc Burt: Hi, I'm trying to delete by query using the following: /solr/de/update?last_seen:[* TO 2016-02-24T00:00:00.00Z] /solr/de/select?q=last_seen:[* TO 2016-02-24T00:00:00.00Z] returns the correct documents to be deleted. Last time I attempted this using the above I somehow managed to delete all documents in the node rather than only the documents returned by the query. Can anyone confirm that this is the correct method to delete documents by query? -- Kind Regards, Marc
Re: Deleting by query
Hi Try this instead /solr/de/update?stream.body=last_seen:[* TO 2016-02-24T00:00:00Z]=true …that is if you have streaming enabled in solrconfig. Else do a POST instead Note that I put a commit=true at the end, so you will see the changes immediately. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 26. feb. 2016 kl. 13.57 skrev Marc Burt: > > Hi, > > I'm trying to delete by query using the following: > > /solr/de/update?last_seen:[* TO > 2016-02-24T00:00:00.00Z] > > /solr/de/select?q=last_seen:[* TO 2016-02-24T00:00:00.00Z] returns the > correct documents to be deleted. > > Last time I attempted this using the above I somehow managed to delete all > documents in the node rather than only the documents returned by the query. > > Can anyone confirm that this is the correct method to delete documents by > query? > > -- > > Kind Regards, > > Marc >
Deleting by query
Hi, I'm trying to delete by query using the following: /solr/de/update?last_seen:[* TO 2016-02-24T00:00:00.00Z] /solr/de/select?q=last_seen:[* TO 2016-02-24T00:00:00.00Z] returns the correct documents to be deleted. Last time I attempted this using the above I somehow managed to delete all documents in the node rather than only the documents returned by the query. Can anyone confirm that this is the correct method to delete documents by query? -- Kind Regards, Marc
Re: Delays when deleting by query
Hmmm, this is unusual. Can we see the code you use to delete? And your solrconfig file? You're not doing something odd like optimizing on commit or anything, right? You shouldn't have to commit after deletes. The fact that you're hanging is very odd (BTW, does hanging mean you're system is locked up or just that you can't find your new documents?). You could try using the default Jetty container just for yucks to see if Tomcat is somehow the culprit, although many people use Tomcat so it'd b something peculiar to your setup. Best Erick On Wed, Dec 7, 2011 at 8:55 AM, Mike Gallan mgal...@hotmail.com wrote: I ran some more tests. I added an explicit commit after each deleteByQuery() call and removed the add/reindex step. This hung up immediately and completed (or timed out?) after 20 minutes. The hangs occur almost exactly 20 minutes apart. Could this be a Tomcat issue? I ran jconsole but didn't see any extraordinary memory or CPU usage. The delays appear on the first delete attempt immediately after start up so I suspect it's not GC related. I also tried adding documents without deleting. This worked with no significant delays on the commit. The delete/commit combo appears to be the source of the problem. Any tips on how to debug this are appreciated! Thanks,Mike From: mgal...@hotmail.com To: solr-user@lucene.apache.org Subject: Delays when deleting by query Date: Tue, 6 Dec 2011 08:25:28 -0500 Hello, We're encountering delays of 10+ minutes when trying to delete from our Solr 3.4 instance. We have 335k documents indexed and interface using SolrJ. Our schema basically consists of a parent object with multiple child objects. Every object is indexed as a separate document with the child documents referencing parents via a 'parentId' field. When any part of a parent object is updated solrServer.deleteByQuery() is called to delete the parent and all the child documents, then solrServer.add() is called to reindex them. We currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s. Deletes work fine on another Solr test instance with 22k documents. Any thoughts? Is this sort of delay common when deleting against this many documents? Thanks, Mike
RE: Delays when deleting by query
Thanks for the response Erick. I actually turned up logging yesterday and noticed spellchecker builds were causing the delays. Setting buildOnCommit to false solved the problem. Our plan is to schedule a nightly timer task that sends a 'spellcheck.build=true' to trigger it. Mike Date: Thu, 8 Dec 2011 08:25:52 -0500 Subject: Re: Delays when deleting by query From: erickerick...@gmail.com To: solr-user@lucene.apache.org Hmmm, this is unusual. Can we see the code you use to delete? And your solrconfig file? You're not doing something odd like optimizing on commit or anything, right? You shouldn't have to commit after deletes. The fact that you're hanging is very odd (BTW, does hanging mean you're system is locked up or just that you can't find your new documents?). You could try using the default Jetty container just for yucks to see if Tomcat is somehow the culprit, although many people use Tomcat so it'd b something peculiar to your setup. Best Erick On Wed, Dec 7, 2011 at 8:55 AM, Mike Gallan mgal...@hotmail.com wrote: I ran some more tests. I added an explicit commit after each deleteByQuery() call and removed the add/reindex step. This hung up immediately and completed (or timed out?) after 20 minutes. The hangs occur almost exactly 20 minutes apart. Could this be a Tomcat issue? I ran jconsole but didn't see any extraordinary memory or CPU usage. The delays appear on the first delete attempt immediately after start up so I suspect it's not GC related. I also tried adding documents without deleting. This worked with no significant delays on the commit. The delete/commit combo appears to be the source of the problem. Any tips on how to debug this are appreciated! Thanks,Mike From: mgal...@hotmail.com To: solr-user@lucene.apache.org Subject: Delays when deleting by query Date: Tue, 6 Dec 2011 08:25:28 -0500 Hello, We're encountering delays of 10+ minutes when trying to delete from our Solr 3.4 instance. We have 335k documents indexed and interface using SolrJ. Our schema basically consists of a parent object with multiple child objects. Every object is indexed as a separate document with the child documents referencing parents via a 'parentId' field. When any part of a parent object is updated solrServer.deleteByQuery() is called to delete the parent and all the child documents, then solrServer.add() is called to reindex them. We currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s. Deletes work fine on another Solr test instance with 22k documents. Any thoughts? Is this sort of delay common when deleting against this many documents? Thanks, Mike
RE: Delays when deleting by query
I ran some more tests. I added an explicit commit after each deleteByQuery() call and removed the add/reindex step. This hung up immediately and completed (or timed out?) after 20 minutes. The hangs occur almost exactly 20 minutes apart. Could this be a Tomcat issue? I ran jconsole but didn't see any extraordinary memory or CPU usage. The delays appear on the first delete attempt immediately after start up so I suspect it's not GC related. I also tried adding documents without deleting. This worked with no significant delays on the commit. The delete/commit combo appears to be the source of the problem. Any tips on how to debug this are appreciated! Thanks,Mike From: mgal...@hotmail.com To: solr-user@lucene.apache.org Subject: Delays when deleting by query Date: Tue, 6 Dec 2011 08:25:28 -0500 Hello, We're encountering delays of 10+ minutes when trying to delete from our Solr 3.4 instance. We have 335k documents indexed and interface using SolrJ. Our schema basically consists of a parent object with multiple child objects. Every object is indexed as a separate document with the child documents referencing parents via a 'parentId' field. When any part of a parent object is updated solrServer.deleteByQuery() is called to delete the parent and all the child documents, then solrServer.add() is called to reindex them. We currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s. Deletes work fine on another Solr test instance with 22k documents. Any thoughts? Is this sort of delay common when deleting against this many documents? Thanks, Mike
Delays when deleting by query
Hello, We're encountering delays of 10+ minutes when trying to delete from our Solr 3.4 instance. We have 335k documents indexed and interface using SolrJ. Our schema basically consists of a parent object with multiple child objects. Every object is indexed as a separate document with the child documents referencing parents via a 'parentId' field. When any part of a parent object is updated solrServer.deleteByQuery() is called to delete the parent and all the child documents, then solrServer.add() is called to reindex them. We currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s. Deletes work fine on another Solr test instance with 22k documents. Any thoughts? Is this sort of delay common when deleting against this many documents? Thanks,Mike