Re: solr 1.3 - did something with deleting documents change?
: The notes in the wiki seem to indicate that syntax (with multiple id nodes) : will be supported in Solr 1.4, not 1.3 - but I guess it really just means that : you can't combine those with a query node yet. correct -- the wiki is pointing out htat you can't combine id and query deletes until 1.4 -- 1.3 supported multiple id (note CHANGES.txt refrences to SOLR-133 : I'll miss the deletesPending stat, I think I'm going to have to re-implement : this within our application now - I try to block people from committing unless : there's actually something to commit (and only after certain time interval, : etc), and the deletesPending stat helped to determine this (maybe you have no : new documents to index but you do need to commit). Not a huge deal to do, it : was just convenient. a nice feature for solr to have in the core would be to make commit a no-op if there are no actaul changes to commit (it would need an option or a query param or something in case people are trying to force a postCommit hook) ... it's pretty easy for Solr to know this based on the index version of hte open reader for commits and the one being used for search. wanna submit a patch? -Hoss
Re: solr 1.3 - did something with deleting documents change?
On Feb 28, 2009, at 5:56 PM, Stephen Weiss wrote: Yeah honestly I don't know how it ever worked either. my guess is that the XPP parser did not validate anything -- when we switched to StAX it validates something... ryan
Re: solr 1.3 - did something with deleting documents change?
Yeah honestly I don't know how it ever worked either. The notes in the wiki seem to indicate that syntax (with multiple id nodes) will be supported in Solr 1.4, not 1.3 - but I guess it really just means that you can't combine those with a query node yet. I'll miss the deletesPending stat, I think I'm going to have to re- implement this within our application now - I try to block people from committing unless there's actually something to commit (and only after certain time interval, etc), and the deletesPending stat helped to determine this (maybe you have no new documents to index but you do need to commit). Not a huge deal to do, it was just convenient. Thanks for the pointers! -- Steve On Feb 27, 2009, at 5:18 PM, Chris Hostetter wrote: : deleteidimage.1/id/deletedeleteidimage.2/id/delete etc... (one : delete node for each image we wanted to delete) : : And that worked in 1.2. that is really suprising ... it's not a legal XML doc (multiple root nodes) so it should have been an error. Support was added in Solr 1.3 to support multiple id elements in a single delete element... deleteidimage.1/ididimage.2/id.../delete : Also, we could get a statistic called deletesPending from the stats.jsp page : in the admin console. : : Now, when we upgraded to 1.3, I noticed right away that deletesPending was : gone, but since it wasn't critical to the application we kinda let it go. : Now, first of all, I'm wondering, is there any way to get this statistic back? Not really Solr kept that stat before because in order to delete by query it had to execute searches and maintain a queue of docs to deletes and then process them explicitly when the commit happened. now the underlying lucene IndexWriter can handle hte deleteByQuery directly. : And that's fine. Just wondering if that was actually a change or what's going : on there or if the fact that it ever worked was a fluke (which seems to be the : case since I find people out on message boards essentially saying the delete : by query method is the only way to delete multiple documents at once). I'm As i mentioned, that was only true before 1.3, but the syntax you were using was definitely a fluke. switching to the multiple id/ nodes inside a single delete/ node should be a lot faster then the query/ using OR syntax you are using right now BTW. -Hoss
Re: solr 1.3 - did something with deleting documents change?
: deleteidimage.1/id/deletedeleteidimage.2/id/delete etc... (one : delete node for each image we wanted to delete) : : And that worked in 1.2. that is really suprising ... it's not a legal XML doc (multiple root nodes) so it should have been an error. Support was added in Solr 1.3 to support multiple id elements in a single delete element... deleteidimage.1/ididimage.2/id.../delete : Also, we could get a statistic called deletesPending from the stats.jsp page : in the admin console. : : Now, when we upgraded to 1.3, I noticed right away that deletesPending was : gone, but since it wasn't critical to the application we kinda let it go. : Now, first of all, I'm wondering, is there any way to get this statistic back? Not really Solr kept that stat before because in order to delete by query it had to execute searches and maintain a queue of docs to deletes and then process them explicitly when the commit happened. now the underlying lucene IndexWriter can handle hte deleteByQuery directly. : And that's fine. Just wondering if that was actually a change or what's going : on there or if the fact that it ever worked was a fluke (which seems to be the : case since I find people out on message boards essentially saying the delete : by query method is the only way to delete multiple documents at once). I'm As i mentioned, that was only true before 1.3, but the syntax you were using was definitely a fluke. switching to the multiple id/ nodes inside a single delete/ node should be a lot faster then the query/ using OR syntax you are using right now BTW. -Hoss
solr 1.3 - did something with deleting documents change?
Hi, This might be a very silly question that's documented everywhere but I just can't find an answer right now. When we first implemented solr we used version 1.2 (of course, 1.3 was released days afterwards). In the brief period we used 1.2, whenever we wanted to delete a bunch of documents, we would send along something like this: deleteidimage.1/id/deletedeleteidimage.2/id/delete etc... (one delete node for each image we wanted to delete) And that worked in 1.2. Also, we could get a statistic called deletesPending from the stats.jsp page in the admin console. Now, when we upgraded to 1.3, I noticed right away that deletesPending was gone, but since it wasn't critical to the application we kinda let it go. Now, first of all, I'm wondering, is there any way to get this statistic back? I've checked the document because I know precisely how many deletes should be pending right now (135 documents), and that statistic doesn't appear to be on the page at all by any name, any more. It was very useful though! If there's any way to poll the index for this information still, that would be great. Also, and this is the part that really gets me, somehow now that delete command that we were using no longer works. I can't find any reference to the fact that it changed anywhere... But I know it did because I have the logs from when we tested it (just went back to check). Now I rewrote our scripts to do a delete by query like this: deletequeryid:(image.1 OR image.2)/query/delete And that's fine. Just wondering if that was actually a change or what's going on there or if the fact that it ever worked was a fluke (which seems to be the case since I find people out on message boards essentially saying the delete by query method is the only way to delete multiple documents at once). I'm kinda worried that syntax for other commands might have changed without my knowledge (but only a little - after all, it took about 4 months before someone noticed deletes didn't work properly - it's not the most common thing for us, but still, that's a long time). If anyone knows the answer to these questions (in particular, the deletesPending statistic) I'd really appreciate the hint. Thanks!! -- Steve
