On Thu, May 22, 2014 at 10:30 AM, Erick Erickson <erickerick...@gmail.com>wrote:
> If we manage to extend the "lazy core" loading from stand-alone to > "lazy collection" loading in SolrCloud would that satisfy the > use-case? It still doesn't allow manual unloading of the collection, > but the large collection would "age out" if it was truly not used all > that much. That said, I don't know if there's active work in this > direction right now. > > This is a nice option to have but having the ability to manually load and unload a collection is still needed. For example, if you're doing analytics work and storing a day's data in a collection, you still want to be able to access day 32 and day 64 even if you keep 30 days of data loaded in memory. I also think having a manual option would allow people more flexibility in how they manage the number of collections they keep loaded. > Best, > Erick > > On Thu, May 22, 2014 at 5:35 AM, Saumitra Srivastav > <saumitra.srivast...@gmail.com> wrote: > > Yes, that's what I am doing. > > > > IMO in addition to search, Solr satisfies the needs of lot of analytics > > applications as well, and on-demand loading is a common use case in > > analytics(to keep TCO low), so it would be nice to keep this supported. > > > > > > Regards, > > Saumitra > > > > > > > > On Thu, May 22, 2014 at 5:37 PM, Shalin Shekhar Mangar [via Lucene] < > > ml-node+s472066n4137630...@n3.nabble.com> wrote: > > > >> Ah, I see. So if I understand it correctly, you are sharing the cluster > >> with other collections which are more frequently used and you want to > keep > >> resources available for them so you keep your collection dormant most of > >> the time until requested. > >> > >> No, we don't have such an API. It'd be cool to have a lazy loaded > >> collection though. Thank you for describing the use-case because the way > >> that we're moving towards (ZK as truth etc.), the core admin APIs will > >> gradually be phased out and satisfying your use-case would become > >> impossible. Let me think more on this. > >> > >> > >> On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav < > >> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=0>> > >> wrote: > >> > >> > I don't want to delete the collection/shards. I just want to unload > all > >> > shards/replica of the collection temporarily. > >> > > >> > Let me explain my use case. > >> > > >> > I have a collection alias say *collectionA* which consists of n > >> > collections(n<=5) each with 8 shards and 2 replica over a 16 machine > >> > cluster. > >> > *collectionA* is quite big in size and used very rarely, so we keep > all > >> > shards/replica of *collectionA* unloaded most of the time. Only when > >> user > >> > request to use it, we load it in memory. To load/unload shards/replica > >> of > >> > aliased *collectionA*, we use CLUSTERSTATUS api to get list of all > >> > shards/replicas in aliased collection and then use CORE ADMIN api to > >> > load/unload them. > >> > > >> > As you can see there is lot of manual work involved, so I want to know > >> if > >> > there is an API to load/unload ALL shards/replicas of a collection? > >> > > >> > > >> > Regards, > >> > Saumitra > >> > > >> > > >> > On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] < > >> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=1 > >> > >> wrote: > >> > > >> > > You can use the delete Collection API. > >> > > > >> > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6 > >> > > > >> > > > >> > > > >> > > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav < > >> > > [hidden email] < > http://user/SendEmail.jtp?type=node&node=4137608&i=0>> > >> > >> > > wrote: > >> > > > >> > > > Guys, any suggestions for this?? > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > View this message in context: > >> > > > > >> > > > >> > > >> > http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html > >> > > > Sent from the Solr - User mailing list archive at Nabble.com. > >> > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > Regards, > >> > > Shalin Shekhar Mangar. > >> > > > >> > > > >> > > ------------------------------ > >> > > If you reply to this email, your message will be added to the > >> discussion > >> > > below: > >> > > > >> > > > >> > > >> > http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html > >> > > To start a new topic under Solr - User, email > >> > > [hidden email] < > http://user/SendEmail.jtp?type=node&node=4137630&i=2> > >> > > To unsubscribe from Unload collection in SolrCloud, click here< > >> > > >> > > >> > > . > >> > > NAML< > >> > > >> > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml > >> > >> > > > >> > > > >> > > >> > > >> > > >> > > >> > -- > >> > View this message in context: > >> > > >> > http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html > >> > >> > Sent from the Solr - User mailing list archive at Nabble.com. > >> > > >> > >> > >> > >> -- > >> Regards, > >> Shalin Shekhar Mangar. > >> > >> > >> ------------------------------ > >> If you reply to this email, your message will be added to the > discussion > >> below: > >> > >> > http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137630.html > >> To start a new topic under Solr - User, email > >> ml-node+s472066n472068...@n3.nabble.com > >> To unsubscribe from Unload collection in SolrCloud, click here< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM= > > > >> . > >> NAML< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml > > > >> > > > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137639.html > > Sent from the Solr - User mailing list archive at Nabble.com. >
