Thanks for the response Rahul, and sorry for the delay. This email somehow got missed in my inbox.
I am not able to get the thread dump now; here are the other details: - It is stand-alone Solr - I also found later that, while the UI timed out, the request was ultimately processed. In some cases, the request got processed hours after I had made it, but it did get processed. - When I tried via cURL, it did not time out but was stuck forever and never returned. I guess if I had waited for a few hours, it would have returned. - I later found out that we had allocated 4 cores for Solr, so I guess using 110% CPU was ok, since that is still a little over 25% of the CPU available to Solr. Thanks! Goutham On Sun, Sep 27, 2020 at 12:50 AM Rahul Goswami <rahul196...@gmail.com> wrote: > You mention high CPU usage...Can you share the thread dump (using jstack) > for both the delete by id and delete by query? > Also, an output of /solr/<collection>/schema executed on the host? > Lastly, is this standalone Solr or SolrCloud? > Attachments won’t make it to the list, so I would recommend sharing a link > to any file sharing service. > On a side note, I have observed the UI timing out requests after a certain > point even though the actual request is still being processed. In case > something like that is happening here, did you try the delete by id as an > HTTP request through a curl or Postman? Having said that I would still > expect delete by id to execute in reasonable time, so I would start by > looking at what is s eating up the CPU in your request. > > -Rahul > > On Sat, Sep 26, 2020 at 4:50 AM Goutham Tholpadi <gtholp...@gmail.com> > wrote: > > > Thanks Dominique! I just tried deleting a single document using its id. I > > > > tried this: > > > > <delete> > > > > <id> id123 </id> > > > > </delete> > > > > > > > > and this: > > > > <delete> > > > > <query> id:id123 </query> > > > > </delete> > > > > > > > > In each case, I still get the same "Solr connection lost" error. I > checked > > > > that the Solr instance has enough RAM (it was using 73% of the RAM), but > it > > > > was using 110% CPU. Could this be a CPU under-allocation problem (the > Solr > > > > container has 4 cores allocated to it)? > > > > > > > > Thanks > > > > Goutham > > > > > > > > On Fri, Sep 25, 2020 at 7:41 PM Dominique Bejean < > > dominique.bej...@eolya.fr> > > > > wrote: > > > > > > > > > Hi Goutham, > > > > > > > > > > I agree with Rahul, avoid large deletebyquery. > > > > > It you can, prefere one query to get all the ids first than use ids > with > > > > > deletebyid > > > > > > > > > > Regards > > > > > > > > > > Dominique > > > > > > > > > > > > > > > Le ven. 25 sept. 2020 à 06:50, Goutham Tholpadi <gtholp...@gmail.com> > a > > > > > écrit : > > > > > > > > > > > I spoke too soon. I am getting the "Connection lost" error again. > > > > > > > > > > > > I have never faced this problem when there are a small number of docs > > in > > > > > > the index. I was wondering if the size of the index (30M docs) has > > > > > anything > > > > > > to do with this. > > > > > > > > > > > > Thanks > > > > > > Goutham > > > > > > > > > > > > On Fri, Sep 25, 2020 at 9:55 AM Goutham Tholpadi < > gtholp...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Thanks for your response Rahul! > > > > > > > > > > > > > > Yes, all the fields I tried with were indexed=true, but it did not > > > > > work. > > > > > > > > > > > > > > Btw, when I try to today, I am no longer getting the "Connection > > lost" > > > > > > > error. The delete command returns with status=success, however the > > > > > > document > > > > > > > is not actually deleted when I check in the search console again. > > > > > > > > > > > > > > I tried using Document Type as XML just now and I see the same > > > > > behaviour > > > > > > > as above. > > > > > > > > > > > > > > Thanks > > > > > > > Goutham > > > > > > > > > > > > > > On Fri, Sep 25, 2020 at 7:17 AM Rahul Goswami < > rahul196...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Goutham, > > > > > > >> Is the field you are trying to delete by indexed=true in the > schema > > ? > > > > > > >> If the uniqueKey is indexed=true, does delete by id work for you? > > > > > > >> (<delete><id> uniqueKey:value</id></delete>) > > > > > > >> Also, instead of "Solr Command" if you choose the Document type > as > > > > > > "XML" > > > > > > >> does it make any difference? > > > > > > >> > > > > > > >> Rahul > > > > > > >> > > > > > > >> On Thu, Sep 24, 2020 at 1:04 PM Goutham Tholpadi < > > gtholp...@gmail.com > > > > > > > > > > > > >> wrote: > > > > > > >> > > > > > > >> > Hi, > > > > > > >> > > > > > > > >> > Setup: > > > > > > >> > We have a stand-alone Solr (v7.2) with around 30 million > documents > > > > > and > > > > > > >> with > > > > > > >> > 4 cores, 38G of RAM, and a 1TB disk. The documents were not > > directly > > > > > > >> > indexed but came from a restore of a back from another Solr > > > > > instance. > > > > > > >> > > > > > > > >> > Problem: > > > > > > >> > Search queries seem to be working fine. However, when I try to > > > > > delete > > > > > > >> > documents from the Solr console, I get a "Connection to Solr > lost" > > > > > > >> error. I > > > > > > >> > am trying by navigating to the "Documents" section of the chosen > > > > > core, > > > > > > >> > using "Solr Command" as the "Document Type", and entering > > something > > > > > > >> this in > > > > > > >> > the box below: > > > > > > >> > <delete> > > > > > > >> > <query> > > > > > > >> > field:value > > > > > > >> > </query> > > > > > > >> > </delete> > > > > > > >> > > > > > > > >> > I tried with the field being the unique key, and otherwise. I > also > > > > > > tried > > > > > > >> > with values containing wild cards. I got the error in all cases. > > > > > > >> > > > > > > > >> > Any pointers on this? > > > > > > >> > > > > > > > >> > Thanks > > > > > > >> > Goutham > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >
