Re: Tuning StatsComponent
What field type do you recommend for a float stats.field for optimal Solr 1.4.1 StatsComponent performance ? float, pfloat or tfloat ? Do you recommend to index the field ? 2011/1/12 stockii st...@shopgate.com my field Type is double maybe sint is better ? but i need double ... =( -- View this message in context: http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-tp2225809p2241903.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tuning StatsComponent
i try this: http://host:port /solr/select?q=YOUR_QUERYstats=onstats.field=amountf.amount.stats.facet=currencyrows=0 and this: http://host:portsolr /select?q=amount_us:*+OR+amount_eur:*[+OR+amount_...:*]stats=onstats.field=amount_usdstats.field=amount_eur[stats.field=amount_...]rows=0 of my index. but however i change my request, every request have a Qtime of ~10 seconds ... my result solr StatsComponent cannot be fast on 31 Million documents =( -- View this message in context: http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-tp2225809p2241793.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tuning StatsComponent
my field Type is double maybe sint is better ? but i need double ... =( -- View this message in context: http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-tp2225809p2241903.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tuning StatsComponent
simplest solution is more RAM !? sometimes i think, that is a standard solution for problems with solr ;-) i going to buy 100 GB RAM :P -- View this message in context: http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-tp2225809p2234557.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tuning StatsComponent
On Jan 11, 2011, at 9:37 AM, stockii wrote: simplest solution is more RAM !? sometimes i think, that is a standard solution for problems with solr ;-) FWIW, it's a solution for most computing problems, right? i going to buy 100 GB RAM :P That won't do it. More RAM is sometimes the answer, but not always. Too much RAM to the JVM often equals too much garbage which means stop the world garbage collections. -- Grant Ingersoll http://www.lucidimagination.com
Re: Tuning StatsComponent
On Mon, Jan 10, 2011 at 2:28 PM, stockii st...@shopgate.com wrote: Hello. i`m using the StatsComponent to get the sum of amounts. but solr statscomponent is very slow on a huge index of 30 Million documents. how can i tune the statscomponent ? Not sure about this problem. the problem is, that i have 5 currencys and i need to send for each currency a new request. thats make the solr search sometimes very slow. =( [...] I guess that you mean the search from the front-end is slow. It is difficult to make a guess without details of your index, and of your queries, but one thing that immediately jumps out is that you could shard the Solr index by currency, and have your front-end direct queries for each currency to the appropriate Solr server. Please do share a description of what all you are indexing, how large your index is, and what kind of queries you are running. I take it that you have already taken a look at http://wiki.apache.org/solr/SolrPerformanceFactors Regards, Gora
Re: Tuning StatsComponent
oh thx for your fast reply. i will try the suggestions. in meanwhile more information about my index. i have 2 solr instances with 6 cores. each core have his own index and one core`s index is about 30 million documents. each document have:(stats-relevant) amount amount_euro currency_id user_costs user_costs_euro currency_id_user_costs so i send for each currency an requeston statscompontn like this stats=truejson.nl=mapwt=javabinrows=0version=2fl=uniquekey,scorestart=0stats.field=amountq=QUERYisShard=truefq=product:bla+currency_id:EURfsv=true the stats.field is changing and filter for each of my 5 currencys. so for ONE search-request, i need to send 10 requests to get the sums. and that veeery slow =( i searching over two shards. sometimes more than two. -- View this message in context: http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-tp2225809p2226258.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tuning StatsComponent
when i start statsComponent i get this message:
INFO: UnInverted multi-valued field
{field=product,memSize=4336,tindexSize=46,time=0,phase1=0,nTerms=1,bigTerms=1,termInstances=0,uses=0}
what means this ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Tuning-StatsComponent-tp2225809p2226555.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tuning StatsComponent
I found StatsComponent to be slow only when I didn't have enough RAM allocated to the JVM. I'm not sure exactly what was causing it, but it was pathologically slow -- and then adding more RAM to the JVM made it incredibly fast. On 1/10/2011 4:58 AM, Gora Mohanty wrote: On Mon, Jan 10, 2011 at 2:28 PM, stockiist...@shopgate.com wrote: Hello. i`m using the StatsComponent to get the sum of amounts. but solr statscomponent is very slow on a huge index of 30 Million documents. how can i tune the statscomponent ? Not sure about this problem. the problem is, that i have 5 currencys and i need to send for each currency a new request. thats make the solr search sometimes very slow. =( [...] I guess that you mean the search from the front-end is slow. It is difficult to make a guess without details of your index, and of your queries, but one thing that immediately jumps out is that you could shard the Solr index by currency, and have your front-end direct queries for each currency to the appropriate Solr server. Please do share a description of what all you are indexing, how large your index is, and what kind of queries you are running. I take it that you have already taken a look at http://wiki.apache.org/solr/SolrPerformanceFactors Regards, Gora
Re: Tuning StatsComponent
StatsComponent, like many things, relies on FieldCache (and the related uninverted version in Solr for multivalued fields), which takes up memory and is related to the number of documents in the index. Strings in FieldCache can also be expensive. -Grant On Jan 10, 2011, at 4:10 PM, Jonathan Rochkind wrote: I found StatsComponent to be slow only when I didn't have enough RAM allocated to the JVM. I'm not sure exactly what was causing it, but it was pathologically slow -- and then adding more RAM to the JVM made it incredibly fast. On 1/10/2011 4:58 AM, Gora Mohanty wrote: On Mon, Jan 10, 2011 at 2:28 PM, stockiist...@shopgate.com wrote: Hello. i`m using the StatsComponent to get the sum of amounts. but solr statscomponent is very slow on a huge index of 30 Million documents. how can i tune the statscomponent ? Not sure about this problem. the problem is, that i have 5 currencys and i need to send for each currency a new request. thats make the solr search sometimes very slow. =( [...] I guess that you mean the search from the front-end is slow. It is difficult to make a guess without details of your index, and of your queries, but one thing that immediately jumps out is that you could shard the Solr index by currency, and have your front-end direct queries for each currency to the appropriate Solr server. Please do share a description of what all you are indexing, how large your index is, and what kind of queries you are running. I take it that you have already taken a look at http://wiki.apache.org/solr/SolrPerformanceFactors Regards, Gora -- Grant Ingersoll http://www.lucidimagination.com/
