Re: Reload an unloaded core
I have similar needs but for a slightly different use-case. In my case, I am breaking up cores / indexes based on the month and year so that I can add an alias that always points to the last few months, but beyond that I want to simply unload the other indexes once they get past a few months old. The indexes will remain on disk but I simply don't want my queries to have to go through the older "archived" documents. However, users will occasionally need to have those indexes reloaded for research reasons so what I was doing in ES was simply re-loading all of the indexes that fit within the range being searched for and added those to an alias (let's call it "archived", for example). Once they are finished querying on that older data, I again unload those indexes and remove the alias. From what I'm reading in this thread, this isn't quite as straight-forward in Solr so I'm looking for other options. Thanks, David On 5/2/2017 5:04 PM, Shashank Pedamallu wrote: Thank you Simon, Erick and Shawn for your replies. Unfortunately, restarting Solr is not a option for me. So, I’ll try to follow the steps given by Shawn to see where I’m standing. Btw, I’m using Solr 6.4.2. Shawn, once again thank you very much for the detailed reply. Thanks, Shashank Pedamallu On 5/2/17, 2:51 PM, "Shawn Heisey" wrote: On 5/2/2017 10:53 AM, Shashank Pedamallu wrote: I want to unload a core from Solr without deleting data-dir or instance-dir. I’m performing some operations on the data-dir after this and then I would like to reload the core from the same data-dir. These are the things I tried: 1. Reload api – throws an exception saying no such core exists. 2. Create api – throws an exception saying a core with given name already exists. Can someone point me what api I could use to achieve this. Please note that, I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc. The RELOAD command isn't going to work at all because the core has been unloaded -- Solr doesn't know about the core, so it can't reload it. This is a case where the language used is somewhat confusing, even though it's completely correct. I am about 90 percent certain that the reason the CREATE command gave you an error message is because you tried to make a new core.properties file before you did the CREATE. When things are working correctly, the CREATE command itself is what will create core.properties. If it already exists, CoreAdmin will give you an error. This is the exact text of the error I encountered when trying to use CREATE after building a core.properties file manually: Error CREATEing SolrCore 'foo': Could not create a new core in C:\Users\sheisey\Downloads\solr-6.5.1\server\solr\fooas another core is already defined there That error message is confusing, so I will be fixing it: https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D10599&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=YimxLbwsjFGBwV4_LR5LK4yXu_uafFvvVujg-7MDJFE&e= To verify what you need to do, I fired up Solr 6.5.1 from an extracted download directory. I created two cores, "foo" and "bar", using the commandline "bin\solr create" command. Then I went to the admin UI and unloaded foo. The foo directory was still there, but the core was gone >from Solr's list. By clicking on the "Add Core" button in the Core Admin tab, typing "foo" into name and instanceDir, and clearing the other text boxes, the core was recreated exactly as it was before it was unloaded. This is the log from the CREATE command that the admin UI sent: 2017-05-02 18:02:49.232 INFO (qtp1543727556-18) [ x:foo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={schema=&dataDir=&name=foo&action=CREATE&config=&instanceDir=foo&wt=json&_=1493747904891} status=0 QTime=396 To double-check this and show how it can be done without the admin UI, I accessed these two URLs (in a browser), and accomplished the exact same thing again. The first URL unloads the core, the second asks Solr to find the core and re-add it with default settings. https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DUNLOAD-26core-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=CkhSi_Ik3vbgx1UYDGYcifbIuN8GUpc64dpm_hxYy8U&e= https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DCREATE-26name-3Dfoo-26instanceDir-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=vORlMj_KMQCCLbYsbQM3t2y5Fy8i6IKI0zw3O8zsbcI&e= If you are using additional options with your cores, such as the configset parameter, you would need to include those options on your CREATE call, similar to what you might have done when you initi
Re: Reload an unloaded core
Thank you Simon, Erick and Shawn for your replies. Unfortunately, restarting Solr is not a option for me. So, I’ll try to follow the steps given by Shawn to see where I’m standing. Btw, I’m using Solr 6.4.2. Shawn, once again thank you very much for the detailed reply. Thanks, Shashank Pedamallu On 5/2/17, 2:51 PM, "Shawn Heisey" wrote: >On 5/2/2017 10:53 AM, Shashank Pedamallu wrote: >> I want to unload a core from Solr without deleting data-dir or instance-dir. >> I’m performing some operations on the data-dir after this and then I would >> like to reload the core from the same data-dir. These are the things I tried: >> >> 1. Reload api – throws an exception saying no such core exists. >> 2. Create api – throws an exception saying a core with given name already >> exists. >> >> Can someone point me what api I could use to achieve this. Please note that, >> I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc. > >The RELOAD command isn't going to work at all because the core has been >unloaded -- Solr doesn't know about the core, so it can't reload it. >This is a case where the language used is somewhat confusing, even >though it's completely correct. > >I am about 90 percent certain that the reason the CREATE command gave >you an error message is because you tried to make a new core.properties >file before you did the CREATE. When things are working correctly, the >CREATE command itself is what will create core.properties. If it >already exists, CoreAdmin will give you an error. This is the exact >text of the error I encountered when trying to use CREATE after building >a core.properties file manually: > >Error CREATEing SolrCore 'foo': Could not create a new core in >C:\Users\sheisey\Downloads\solr-6.5.1\server\solr\fooas another core is >already defined there > >That error message is confusing, so I will be fixing it: > >https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D10599&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=YimxLbwsjFGBwV4_LR5LK4yXu_uafFvvVujg-7MDJFE&e= > > >To verify what you need to do, I fired up Solr 6.5.1 from an extracted >download directory. I created two cores, "foo" and "bar", using the >commandline "bin\solr create" command. Then I went to the admin UI and >unloaded foo. The foo directory was still there, but the core was gone >from Solr's list. > >By clicking on the "Add Core" button in the Core Admin tab, typing "foo" >into name and instanceDir, and clearing the other text boxes, the core >was recreated exactly as it was before it was unloaded. > >This is the log from the CREATE command that the admin UI sent: > >2017-05-02 18:02:49.232 INFO (qtp1543727556-18) [ x:foo] >o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores >params={schema=&dataDir=&name=foo&action=CREATE&config=&instanceDir=foo&wt=json&_=1493747904891} >status=0 QTime=396 > >To double-check this and show how it can be done without the admin UI, I >accessed these two URLs (in a browser), and accomplished the exact same >thing again. The first URL unloads the core, the second asks Solr to >find the core and re-add it with default settings. > >https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DUNLOAD-26core-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=CkhSi_Ik3vbgx1UYDGYcifbIuN8GUpc64dpm_hxYy8U&e= > >https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DCREATE-26name-3Dfoo-26instanceDir-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=vORlMj_KMQCCLbYsbQM3t2y5Fy8i6IKI0zw3O8zsbcI&e= > > >If you are using additional options with your cores, such as the >configset parameter, you would need to include those options on your >CREATE call, similar to what you might have done when you initially >created the core. With some of the options you can use, re-adding a >core might be better done by re-creating the correct core.properties >file and restarting Solr so it discovers the core. > >Erick, I seem to remember the "core.properties.unloaded" rename >happening in the past as well, but when I unloaded on 6.5.1, the >core.properties file was simply deleted. I don't think that's a good >idea because it may contain information that's not available anywhere else. > >Thanks, >Shawn >
Re: Reload an unloaded core
On 5/2/2017 10:53 AM, Shashank Pedamallu wrote: > I want to unload a core from Solr without deleting data-dir or instance-dir. > I’m performing some operations on the data-dir after this and then I would > like to reload the core from the same data-dir. These are the things I tried: > > 1. Reload api – throws an exception saying no such core exists. > 2. Create api – throws an exception saying a core with given name already > exists. > > Can someone point me what api I could use to achieve this. Please note that, > I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc. The RELOAD command isn't going to work at all because the core has been unloaded -- Solr doesn't know about the core, so it can't reload it. This is a case where the language used is somewhat confusing, even though it's completely correct. I am about 90 percent certain that the reason the CREATE command gave you an error message is because you tried to make a new core.properties file before you did the CREATE. When things are working correctly, the CREATE command itself is what will create core.properties. If it already exists, CoreAdmin will give you an error. This is the exact text of the error I encountered when trying to use CREATE after building a core.properties file manually: Error CREATEing SolrCore 'foo': Could not create a new core in C:\Users\sheisey\Downloads\solr-6.5.1\server\solr\fooas another core is already defined there That error message is confusing, so I will be fixing it: https://issues.apache.org/jira/browse/SOLR-10599 To verify what you need to do, I fired up Solr 6.5.1 from an extracted download directory. I created two cores, "foo" and "bar", using the commandline "bin\solr create" command. Then I went to the admin UI and unloaded foo. The foo directory was still there, but the core was gone from Solr's list. By clicking on the "Add Core" button in the Core Admin tab, typing "foo" into name and instanceDir, and clearing the other text boxes, the core was recreated exactly as it was before it was unloaded. This is the log from the CREATE command that the admin UI sent: 2017-05-02 18:02:49.232 INFO (qtp1543727556-18) [ x:foo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={schema=&dataDir=&name=foo&action=CREATE&config=&instanceDir=foo&wt=json&_=1493747904891} status=0 QTime=396 To double-check this and show how it can be done without the admin UI, I accessed these two URLs (in a browser), and accomplished the exact same thing again. The first URL unloads the core, the second asks Solr to find the core and re-add it with default settings. http://localhost:8983/solr/admin/cores?action=UNLOAD&core=foo http://localhost:8983/solr/admin/cores?action=CREATE&name=foo&instanceDir=foo If you are using additional options with your cores, such as the configset parameter, you would need to include those options on your CREATE call, similar to what you might have done when you initially created the core. With some of the options you can use, re-adding a core might be better done by re-creating the correct core.properties file and restarting Solr so it discovers the core. Erick, I seem to remember the "core.properties.unloaded" rename happening in the past as well, but when I unloaded on 6.5.1, the core.properties file was simply deleted. I don't think that's a good idea because it may contain information that's not available anywhere else. Thanks, Shawn
Re: Reload an unloaded core
the core properties definitely disappears if you use a configset, as in # #Written by CorePropertiesLocator #Tue May 02 20:19:40 UTC 2017 name=testcore dataDir=/indexes/solrindexes/testcore configSet=myconf Using a conf directory, as in #Written by CorePropertiesLocator #Tue May 02 20:30:44 UTC 2017 name=testcorewithconf schema=conf/schema.xml dataDir=/indexes/solrindexes/testcorewithconf has the same behavior. This is Solr 6.3.0 standalone, and I share your memory that at one point in the distant past core.properties was renamed on an unload. Probably worth submitting a JIRA -Simon On Tue, May 2, 2017 at 4:04 PM, Erick Erickson wrote: > IIRC, the core.properties file _is_ renamed to > core.properties.unloaded or something like that. > > Yeah, this is something of a pain. The inverse of "unload" is "create" > but you have to know exactly how to create a core, and in SolrCloud > mode that's...interesting. It's much safer to bring the Solr node > down, do what you want then start it up, although not always possible. > > Best, > Erick > > On Tue, May 2, 2017 at 10:55 AM, simon wrote: > > I ran into the exact same situation recently. I unloaded from the > browser > > GUI which does not delete the data or instance dirs, but does delete > > core.properties. I couldn't find any API either so I eventually > manually > > recreated core.properties and restarted Solr. > > > > Would be nice if the core.properties file were to be renamed rather than > > deleted and if there were a RESCAN action to scan for unloaded cores and > > reload them. > > > > On Tue, May 2, 2017 at 12:53 PM, Shashank Pedamallu < > spedama...@vmware.com> > > wrote: > > > >> Hi all, > >> > >> I want to unload a core from Solr without deleting data-dir or > >> instance-dir. I’m performing some operations on the data-dir after this > and > >> then I would like to reload the core from the same data-dir. These are > the > >> things I tried: > >> > >> 1. Reload api – throws an exception saying no such core exists. > >> 2. Create api – throws an exception saying a core with given name > >> already exists. > >> > >> Can someone point me what api I could use to achieve this. Please note > >> that, I’m working with Solr in Non-Cloud mode without Zookeeper, > >> Collections, etc. > >> > >> Thanks in advance! > >> > >> Thanks, > >> Shashank Pedamallu > >> >
Re: Reload an unloaded core
IIRC, the core.properties file _is_ renamed to core.properties.unloaded or something like that. Yeah, this is something of a pain. The inverse of "unload" is "create" but you have to know exactly how to create a core, and in SolrCloud mode that's...interesting. It's much safer to bring the Solr node down, do what you want then start it up, although not always possible. Best, Erick On Tue, May 2, 2017 at 10:55 AM, simon wrote: > I ran into the exact same situation recently. I unloaded from the browser > GUI which does not delete the data or instance dirs, but does delete > core.properties. I couldn't find any API either so I eventually manually > recreated core.properties and restarted Solr. > > Would be nice if the core.properties file were to be renamed rather than > deleted and if there were a RESCAN action to scan for unloaded cores and > reload them. > > On Tue, May 2, 2017 at 12:53 PM, Shashank Pedamallu > wrote: > >> Hi all, >> >> I want to unload a core from Solr without deleting data-dir or >> instance-dir. I’m performing some operations on the data-dir after this and >> then I would like to reload the core from the same data-dir. These are the >> things I tried: >> >> 1. Reload api – throws an exception saying no such core exists. >> 2. Create api – throws an exception saying a core with given name >> already exists. >> >> Can someone point me what api I could use to achieve this. Please note >> that, I’m working with Solr in Non-Cloud mode without Zookeeper, >> Collections, etc. >> >> Thanks in advance! >> >> Thanks, >> Shashank Pedamallu >>
Re: Reload an unloaded core
I ran into the exact same situation recently. I unloaded from the browser GUI which does not delete the data or instance dirs, but does delete core.properties. I couldn't find any API either so I eventually manually recreated core.properties and restarted Solr. Would be nice if the core.properties file were to be renamed rather than deleted and if there were a RESCAN action to scan for unloaded cores and reload them. On Tue, May 2, 2017 at 12:53 PM, Shashank Pedamallu wrote: > Hi all, > > I want to unload a core from Solr without deleting data-dir or > instance-dir. I’m performing some operations on the data-dir after this and > then I would like to reload the core from the same data-dir. These are the > things I tried: > > 1. Reload api – throws an exception saying no such core exists. > 2. Create api – throws an exception saying a core with given name > already exists. > > Can someone point me what api I could use to achieve this. Please note > that, I’m working with Solr in Non-Cloud mode without Zookeeper, > Collections, etc. > > Thanks in advance! > > Thanks, > Shashank Pedamallu >