Re: Deleting by query

[Marc Burt]

Thanks Jan,

That worked.

Kind Regards,

Marc

On 02/26/2016 01:43 PM, Jan Høydahl wrote:

Hi

Try this instead

/solr/de/update?stream.body=last_seen:[* TO 
2016-02-24T00:00:00Z]=true
…that is if you have streaming enabled in solrconfig. Else do a POST instead
Note that I put a commit=true at the end, so you will see the changes 
immediately.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com


26. feb. 2016 kl. 13.57 skrev Marc Burt :

Hi,

I'm trying to delete by query using the following:

/solr/de/update?last_seen:[* TO 
2016-02-24T00:00:00.00Z]

/solr/de/select?q=last_seen:[* TO 2016-02-24T00:00:00.00Z] returns the correct 
documents to be deleted.

Last time I attempted this using the above I somehow managed to delete all 
documents in the node rather than only the documents returned by the query.

Can anyone confirm that this is the correct method to delete documents by query?

--

Kind Regards,

Marc

Re: Deleting by query

[Jan Høydahl]

Hi

Try this instead

/solr/de/update?stream.body=last_seen:[* TO 
2016-02-24T00:00:00Z]=true
…that is if you have streaming enabled in solrconfig. Else do a POST instead
Note that I put a commit=true at the end, so you will see the changes 
immediately.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 26. feb. 2016 kl. 13.57 skrev Marc Burt :
> 
> Hi,
> 
> I'm trying to delete by query using the following:
> 
> /solr/de/update?last_seen:[* TO 
> 2016-02-24T00:00:00.00Z]
> 
> /solr/de/select?q=last_seen:[* TO 2016-02-24T00:00:00.00Z] returns the 
> correct documents to be deleted.
> 
> Last time I attempted this using the above I somehow managed to delete all 
> documents in the node rather than only the documents returned by the query.
> 
> Can anyone confirm that this is the correct method to delete documents by 
> query?
> 
> -- 
> 
> Kind Regards,
> 
> Marc
> 

Deleting by query

[Marc Burt]

Hi,

I'm trying to delete by query using the following:

/solr/de/update?last_seen:[* TO 
2016-02-24T00:00:00.00Z]


/solr/de/select?q=last_seen:[* TO 2016-02-24T00:00:00.00Z] returns the 
correct documents to be deleted.


Last time I attempted this using the above I somehow managed to delete 
all documents in the node rather than only the documents returned by the 
query.


Can anyone confirm that this is the correct method to delete documents 
by query?


--

Kind Regards,

Marc

Re: Delays when deleting by query

[Erick Erickson]

Hmmm, this is unusual. Can we see the code you use to delete?
And your solrconfig file? You're not doing something odd like
optimizing on commit or anything, right?

You shouldn't have to commit after deletes. The fact that you're
hanging is very odd (BTW, does hanging mean you're system
is locked up or just that you can't find your new documents?).

You could try using the default Jetty container just for yucks
to see if Tomcat is somehow the culprit, although many people
use Tomcat so it'd b something peculiar to your setup.

Best
Erick

On Wed, Dec 7, 2011 at 8:55 AM, Mike Gallan mgal...@hotmail.com wrote:

 I ran some more tests.  I added an explicit commit after each deleteByQuery() 
 call and removed the add/reindex step.  This hung up immediately and 
 completed (or timed out?) after 20 minutes.  The hangs occur almost exactly 
 20 minutes apart.  Could this be a Tomcat issue?

 I ran jconsole but didn't see any extraordinary memory or CPU usage.  The 
 delays appear on the first delete attempt immediately after start up so I 
 suspect it's not GC related.

 I also tried adding documents without deleting.  This worked with no 
 significant delays on the commit.  The delete/commit combo appears to be the 
 source of the problem.

 Any tips on how to debug this are appreciated!
 Thanks,Mike

 From: mgal...@hotmail.com
 To: solr-user@lucene.apache.org
 Subject: Delays when deleting by query
 Date: Tue, 6 Dec 2011 08:25:28 -0500

 Hello,

 We're encountering delays of 10+ minutes when trying to delete from our Solr 
 3.4 instance.  We have 335k documents indexed and interface using SolrJ.  
 Our schema basically consists of a parent object with multiple child 
 objects.  Every object is indexed as a separate document with the child 
 documents referencing parents via a 'parentId' field.  When any part of a 
 parent object is updated solrServer.deleteByQuery() is called to delete the 
 parent and all the child documents, then solrServer.add() is called to 
 reindex them.  We currently rely on autocommit, with maxDocs set to 100 and 
 maxTime set to 30s.  Deletes work fine on another Solr test instance with 
 22k documents.

 Any thoughts?  Is this sort of delay common when deleting against this many 
 documents?

 Thanks,
 Mike

RE: Delays when deleting by query

[Mike Gallan]

Thanks for the response Erick.  I actually turned up logging yesterday and 
noticed spellchecker builds were causing the delays.  Setting buildOnCommit to 
false solved the problem.  Our plan is to schedule a nightly timer task that 
sends a 'spellcheck.build=true' to trigger it.

Mike

 Date: Thu, 8 Dec 2011 08:25:52 -0500
 Subject: Re: Delays when deleting by query
 From: erickerick...@gmail.com
 To: solr-user@lucene.apache.org
 
 Hmmm, this is unusual. Can we see the code you use to delete?
 And your solrconfig file? You're not doing something odd like
 optimizing on commit or anything, right?
 
 You shouldn't have to commit after deletes. The fact that you're
 hanging is very odd (BTW, does hanging mean you're system
 is locked up or just that you can't find your new documents?).
 
 You could try using the default Jetty container just for yucks
 to see if Tomcat is somehow the culprit, although many people
 use Tomcat so it'd b something peculiar to your setup.
 
 Best
 Erick
 
 On Wed, Dec 7, 2011 at 8:55 AM, Mike Gallan mgal...@hotmail.com wrote:
 
  I ran some more tests.  I added an explicit commit after each 
  deleteByQuery() call and removed the add/reindex step.  This hung up 
  immediately and completed (or timed out?) after 20 minutes.  The hangs 
  occur almost exactly 20 minutes apart.  Could this be a Tomcat issue?
 
  I ran jconsole but didn't see any extraordinary memory or CPU usage.  The 
  delays appear on the first delete attempt immediately after start up so I 
  suspect it's not GC related.
 
  I also tried adding documents without deleting.  This worked with no 
  significant delays on the commit.  The delete/commit combo appears to be 
  the source of the problem.
 
  Any tips on how to debug this are appreciated!
  Thanks,Mike
 
  From: mgal...@hotmail.com
  To: solr-user@lucene.apache.org
  Subject: Delays when deleting by query
  Date: Tue, 6 Dec 2011 08:25:28 -0500
 
  Hello,
 
  We're encountering delays of 10+ minutes when trying to delete from our 
  Solr 3.4 instance.  We have 335k documents indexed and interface using 
  SolrJ.  Our schema basically consists of a parent object with multiple 
  child objects.  Every object is indexed as a separate document with the 
  child documents referencing parents via a 'parentId' field.  When any part 
  of a parent object is updated solrServer.deleteByQuery() is called to 
  delete the parent and all the child documents, then solrServer.add() is 
  called to reindex them.  We currently rely on autocommit, with maxDocs set 
  to 100 and maxTime set to 30s.  Deletes work fine on another Solr test 
  instance with 22k documents.
 
  Any thoughts?  Is this sort of delay common when deleting against this 
  many documents?
 
  Thanks,
  Mike
 
 
  

RE: Delays when deleting by query

[Mike Gallan]

I ran some more tests.  I added an explicit commit after each deleteByQuery() 
call and removed the add/reindex step.  This hung up immediately and completed 
(or timed out?) after 20 minutes.  The hangs occur almost exactly 20 minutes 
apart.  Could this be a Tomcat issue?

I ran jconsole but didn't see any extraordinary memory or CPU usage.  The 
delays appear on the first delete attempt immediately after start up so I 
suspect it's not GC related.

I also tried adding documents without deleting.  This worked with no 
significant delays on the commit.  The delete/commit combo appears to be the 
source of the problem.

Any tips on how to debug this are appreciated!
Thanks,Mike

 From: mgal...@hotmail.com
 To: solr-user@lucene.apache.org
 Subject: Delays when deleting by query
 Date: Tue, 6 Dec 2011 08:25:28 -0500
 
 Hello,
 
 We're encountering delays of 10+ minutes when trying to delete from our Solr 
 3.4 instance.  We have 335k documents indexed and interface using SolrJ.  Our 
 schema basically consists of a parent object with multiple child objects.  
 Every object is indexed as a separate document with the child documents 
 referencing parents via a 'parentId' field.  When any part of a parent object 
 is updated solrServer.deleteByQuery() is called to delete the parent and all 
 the child documents, then solrServer.add() is called to reindex them.  We 
 currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s. 
  Deletes work fine on another Solr test instance with 22k documents.
 
 Any thoughts?  Is this sort of delay common when deleting against this many 
 documents?
 
 Thanks,
 Mike
 
  

Delays when deleting by query

[Mike Gallan]

Hello,
We're encountering delays of 10+ minutes when trying to delete from our Solr 
3.4 instance.  We have 335k documents indexed and interface using SolrJ.  Our 
schema basically consists of a parent object with multiple child objects.  
Every object is indexed as a separate document with the child documents 
referencing parents via a 'parentId' field.  When any part of a parent object 
is updated solrServer.deleteByQuery() is called to delete the parent and all 
the child documents, then solrServer.add() is called to reindex them.  We 
currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s.  
Deletes work fine on another Solr test instance with 22k documents.
Any thoughts?  Is this sort of delay common when deleting against this many 
documents?
Thanks,Mike