Re: Auto-optimization for Solr indexes

You'll probably have to shard before you get to the TB range. At that point, all the optimization is done individually on each shard so it really doesn't matter how many shards you have. Just issuing http://solr:port/solr/collection/update?optimize=true is sufficient, that'll forward the

Re: Auto-optimization for Solr indexes

Rahul: bq: we dont want the index sizes to grow too large and auto optimzie to kick in Not what quite what's going on. There is no "auto optimize". What there is is background merging that will take _some_ segments and merge them together. Very occasionally this will be the same as a full

Re: Replacing a document in Solr5

+1 On Sun, Dec 20, 2015 at 10:28 PM, Debraj Manna wrote: > Thanks Andrea for the detailed explanation. > On Dec 19, 2015 1:34 PM, "Andrea Gazzarini" wrote: > > > That has nothing to do with your topic: addField adds a new value for a > > given

Auto-optimization for Solr indexes

Hi, I would like to find out, will it be good to do write a script to do an auto-opitmization of the indexes at a certain time every day? Is there any advantage to do so? I found that optimization can reduce the index size by quite a signification amount, and allow the searching of the index to

Re: Auto-optimization for Solr indexes

Much depends on how often the index is updated. If your index only changes, say, once a day then it's probably a good idea. If you're constantly updating your index, then I'd recommend that you do _not_ optimize. Optimizing will create one large segment. That segment will be unlikely to be merged

Re: Re: Some problems when upload data to index in cloud environment

Hi Shawn, thanks for your reply. :) It is because the /update/extract handler is not defined in my collection's solrconfig.xml file as I upload the basic_configs/conf to ZooKeeper. When I upload sample_techproducts_configs to ZooKeeper, everything goes well. I am a freshman for Solr. Now I

Re: solrcloud used a lot of memory and memory keep increasing during long time run

Thanks Erick for pointing out the memory change in a sawtooth pattern. The problem troubles me is that the bottom point of the sawtooth keeps increasing. And when the used capacity of old generation exceeds the threshold set by CMS’s CMSInitiatingOccupancyFraction, gc keeps running and uses a lot

Re: Auto-optimization for Solr indexes

Thanks for your information Erick. We have yet to decide how often we will update the index to include new documents that came in. Let's say we update the index once a day, then when the indexed is updated, we do the optimization (this will be done at night when there are not many users using the

Re: facet component and uninverted field

On Sun, Dec 20, 2015, at 01:32 PM, Jamie Johnson wrote: > For those interested I've attached an initial patch to > https://issues.apache.org/jira/browse/SOLR-8096 to start supporting uif > in > FacetComponent via JSON facet api. > On Dec 18, 2015 9:22 PM, "Jamie Johnson"

Re: Auto-optimization for Solr indexes

Thanks Erick! Rahul On Mon, Dec 21, 2015 at 10:07 AM, Erick Erickson wrote: > Rahul: > > bq: we dont want the index sizes to grow too large and auto optimzie to > kick in > > Not what quite what's going on. There is no "auto optimize". What > there is is background

Re: Auto-optimization for Solr indexes

Hi Erick, We index around several million documents/ day and we optimize everyday when the relative load is low. The reason we optimize is, we dont want the index sizes to grow too large and auto optimzie to kick in. When auto optimize kicks in, it results in unpredictable performance as it is CPU

Re: Auto-optimization for Solr indexes

Thanks Erick! Yes, most likely I'll have to do sharing soon before the index size grows too big. Regards, Edwin On 21 December 2015 at 13:18, Rahul Ramesh wrote: > Thanks Erick! > > Rahul > > On Mon, Dec 21, 2015 at 10:07 AM, Erick Erickson >

Re: faceting is unusable slow since upgrade to 5.3.0

Thanks Jamie. On Sat, Dec 19, 2015 at 11:31 PM, Jamie Johnson wrote: > Bill, > > Check out the patch attached to > https://issues.apache.org/jira/browse/SOLR-8096. I had considered making > the method uif after I had done most of the work, it would be trivial to > change and

TPS with Solr Cloud

Hi, I am trying to evaluate solr for one of my project for which i need to check the scalability in terms of tps(transaction per second) for my application. I have configured solr on 1 AWS server as standalone application which is giving me a tps of ~8000 for my query. In order to test the

Re: solrcloud used a lot of memory and memory keep increasing during long time run

Just now, I see about 40 "Searchers@ main" displayed in Solr Web UI: collection -> Plugins/Stats -> CORE I think it’s abnormal! softcommit is set to 1.5s, but warmupTime needs about 3s Does it lead to so many Searchers? maxWarmingSearchers is set to 4 in my solrconfig.xml, doesn’t it will

Re: facet component and uninverted field

For those interested I've attached an initial patch to https://issues.apache.org/jira/browse/SOLR-8096 to start supporting uif in FacetComponent via JSON facet api. On Dec 18, 2015 9:22 PM, "Jamie Johnson" wrote: > I recently saw that the new JSON Facet API supports

Re: Replacing a document in Solr5

Thanks Andrea for the detailed explanation. On Dec 19, 2015 1:34 PM, "Andrea Gazzarini" wrote: > That has nothing to do with your topic: addField adds a new value for a > given field in a SolrInputDocument, while setField replaces any existing > value (of a given field,