Re: queryResultCache showing all zeros
: We just configured a new Solr cloud (5 nodes) running Solr 4.3, ran : about 200 000 queries taken from our production environment and measured : the performance of the cloud over a collection of 14M documents with the : default Solr settings. We are now trying to tune the different caches : and when I look at each node of the cloud, all of them are showing no : activity (see below) regarding the queryResultCache... all other caches : are showing some activity. Any idea what could cause this? Can you show us some examples of hte types of queries you are executing? Do you have useFilterForSortedQuery in your solrconfig.xml ? -Hoss
RE: queryResultCache showing all zeros
Looks like the problem might not be related to Solr but to a proprietary system we have on top of it. I made some queries with facets and the cache was updated. We are looking into this... I should not have assumed that the problem was coming from Solr ;) I'll let you know if there is anything From: Chris Hostetter Sent: Wednesday, July 31, 2013 1:58 PM To: solr-user@lucene.apache.org Subject: Re: queryResultCache showing all zeros : We just configured a new Solr cloud (5 nodes) running Solr 4.3, ran : about 200 000 queries taken from our production environment and measured : the performance of the cloud over a collection of 14M documents with the : default Solr settings. We are now trying to tune the different caches : and when I look at each node of the cloud, all of them are showing no : activity (see below) regarding the queryResultCache... all other caches : are showing some activity. Any idea what could cause this? Can you show us some examples of hte types of queries you are executing? Do you have useFilterForSortedQuery in your solrconfig.xml ? -Hoss
RE: queryResultCache showing all zeros
Ok I might have found an Solr issue after I fixed a problem in our system. This the kind of query we are making: http://10.0.5.214:8201/solr/Current/select?fq=position_refreshed_date_id:[2747%20TO%203501]fq=position_soc_2011_8_code:41101100fq=country_id:1fq=position_job_type_id:4fq=position_education_level_id:8fq=position_salary_range_id:2fq=is_dirty:falsefq=is_staffing:falsefq=-position_soc_2011_2_code:99fq=-covering_source_id:(839%20OR%201145%20OR%2025%20OR%20802%20OR%20777%20OR%2085%20OR%20881%20OR%20775%20OR%201558%20OR%20743%20OR%20800%20OR%201580%20OR%201147%20OR%201690%20OR%20674%20OR%20894%20OR%20791)q=%20(title:photographer%20OR%20ad_description:photographer%20OR%20super_alias:photographer)%20AND%20(_val_:%22sum(product(75,div(5000,sum(50,sub(3500,position_refreshed_date_id,product(0.75,job_score),product(0.75,source_score))%22)facet=truefacet.mincount=1f.state_id.facet.limit=10facet.field=state_idfacet.field=position_salary_range_idfacet.field=position_job_type_idfacet.field=position_naics_6_codefacet.field=place_idfacet.field=position_education_level_idfacet.field=position_soc_2011_8_codef.position_salary_range_id.facet.limit=10f.position_job_type_id.facet.limit=10f.position_naics_6_code.facet.limit=10f.place_id.facet.limit=10f.position_education_level_id.facet.limit=10f.position_soc_2011_8_code.facet.limit=10rows=10start=0fl=job_id,position_id,super_alias_id,advertiser,super_alias,credited_source_id,position_first_seen_date_id,position_last_seen_date_id,%20position_posted_date_id,%20position_refreshed_date_id,%20position_job_type_id,%20position_function_id,position_green_code,title_id,semi_clean_title_id,clean_title_id,position_empl_count,place_id,%20state_id,county_id,msa_id,country_id,position_id,position_job_type_mva,%20ad_activity_status_id,%20position_score,%20ad_score,position_salary,position_salary_range_id,position_salary_source,position_naics_6_code,position_education_level_id,%20is_staffing,is_bulk,is_anonymous,is_third_party,is_dirty,ref_num,tags,lat,long,position_duns_number,url,advertiser_id,%20title,%20semi_clean_title,%20ad_description,%20position_description,%20ad_bls_salary,%20position_bls_salary,%20covering_source_id,%20content_model_id,position_soc_2011_8_code,position_noc_2006_4_idgroup.field=position_idgroup=truegroup.ngroups=truegroup.main=truesort=score%20desc it's quite long but this request uses both faceting and grouping. If I remove the grouping then the cache is used. Is this a normal behavior or a bug? Thanks From: Jean-Sebastien Vachon Sent: Wednesday, July 31, 2013 2:38 PM To: solr-user@lucene.apache.org Subject: RE: queryResultCache showing all zeros Looks like the problem might not be related to Solr but to a proprietary system we have on top of it. I made some queries with facets and the cache was updated. We are looking into this... I should not have assumed that the problem was coming from Solr ;) I'll let you know if there is anything From: Chris Hostetter Sent: Wednesday, July 31, 2013 1:58 PM To: solr-user@lucene.apache.org Subject: Re: queryResultCache showing all zeros : We just configured a new Solr cloud (5 nodes) running Solr 4.3, ran : about 200 000 queries taken from our production environment and measured : the performance of the cloud over a collection of 14M documents with the : default Solr settings. We are now trying to tune the different caches : and when I look at each node of the cloud, all of them are showing no : activity (see below) regarding the queryResultCache... all other caches : are showing some activity. Any idea what could cause this? Can you show us some examples of hte types of queries you are executing? Do you have useFilterForSortedQuery in your solrconfig.xml ? -Hoss
RE: queryResultCache showing all zeros
Also we do not have any useFilterForSortedQuery in our config. So we are relying on the default which I guess is false. From: Jean-Sebastien Vachon Sent: Wednesday, July 31, 2013 3:44 PM To: solr-user@lucene.apache.org Subject: RE: queryResultCache showing all zeros Ok I might have found an Solr issue after I fixed a problem in our system. This the kind of query we are making: http://10.0.5.214:8201/solr/Current/select?fq=position_refreshed_date_id:[2747%20TO%203501]fq=position_soc_2011_8_code:41101100fq=country_id:1fq=position_job_type_id:4fq=position_education_level_id:8fq=position_salary_range_id:2fq=is_dirty:falsefq=is_staffing:falsefq=-position_soc_2011_2_code:99fq=-covering_source_id:(839%20OR%201145%20OR%2025%20OR%20802%20OR%20777%20OR%2085%20OR%20881%20OR%20775%20OR%201558%20OR%20743%20OR%20800%20OR%201580%20OR%201147%20OR%201690%20OR%20674%20OR%20894%20OR%20791)q=%20(title:photographer%20OR%20ad_description:photographer%20OR%20super_alias:photographer)%20AND%20(_val_:%22sum(product(75,div(5000,sum(50,sub(3500,position_refreshed_date_id,product(0.75,job_score),product(0.75,source_score))%22)facet=truefacet.mincount=1f.state_id.facet.limit=10facet.field=state_idfacet.field=position_salary_range_idfacet.field=position_job_type_idfacet.field=position_naics_6_codefacet.field=place_idfacet.field=position_education_level_idfacet.field=position_soc_2011_8_codef.position_salary_range_id.facet.limit=10f.position_job_type_id.facet.limit=10f.position_naics_6_code.facet.limit=10f.place_id.facet.limit=10f.position_education_level_id.facet.limit=10f.position_soc_2011_8_code.facet.limit=10rows=10start=0fl=job_id,position_id,super_alias_id,advertiser,super_alias,credited_source_id,position_first_seen_date_id,position_last_seen_date_id,%20position_posted_date_id,%20position_refreshed_date_id,%20position_job_type_id,%20position_function_id,position_green_code,title_id,semi_clean_title_id,clean_title_id,position_empl_count,place_id,%20state_id,county_id,msa_id,country_id,position_id,position_job_type_mva,%20ad_activity_status_id,%20position_score,%20ad_score,position_salary,position_salary_range_id,position_salary_source,position_naics_6_code,position_education_level_id,%20is_staffing,is_bulk,is_anonymous,is_third_party,is_dirty,ref_num,tags,lat,long,position_duns_number,url,advertiser_id,%20title,%20semi_clean_title,%20ad_description,%20position_description,%20ad_bls_salary,%20position_bls_salary,%20covering_source_id,%20content_model_id,position_soc_2011_8_code,position_noc_2006_4_idgroup.field=position_idgroup=truegroup.ngroups=truegroup.main=truesort=score%20desc it's quite long but this request uses both faceting and grouping. If I remove the grouping then the cache is used. Is this a normal behavior or a bug? Thanks From: Jean-Sebastien Vachon Sent: Wednesday, July 31, 2013 2:38 PM To: solr-user@lucene.apache.org Subject: RE: queryResultCache showing all zeros Looks like the problem might not be related to Solr but to a proprietary system we have on top of it. I made some queries with facets and the cache was updated. We are looking into this... I should not have assumed that the problem was coming from Solr ;) I'll let you know if there is anything From: Chris Hostetter Sent: Wednesday, July 31, 2013 1:58 PM To: solr-user@lucene.apache.org Subject: Re: queryResultCache showing all zeros : We just configured a new Solr cloud (5 nodes) running Solr 4.3, ran : about 200 000 queries taken from our production environment and measured : the performance of the cloud over a collection of 14M documents with the : default Solr settings. We are now trying to tune the different caches : and when I look at each node of the cloud, all of them are showing no : activity (see below) regarding the queryResultCache... all other caches : are showing some activity. Any idea what could cause this? Can you show us some examples of hte types of queries you are executing? Do you have useFilterForSortedQuery in your solrconfig.xml ? -Hoss
RE: queryResultCache showing all zeros
: it's quite long but this request uses both faceting and grouping. If I : remove the grouping then the cache is used. Is this a normal behavior or : a bug? I believe that is expected -- i don't think grouping can take advantage of the queryResultCache because of how it collects documents. there is however a group.cache.percent option tha you might look into -- but i honestly have no idea if that toggles the use of queryResultCache or something else, i havn't played with it before... https://wiki.apache.org/solr/FieldCollapsing#Request_Parameters -Hoss
Re: queryResultCache showing all zeros
On Wed, Jul 31, 2013 at 3:49 PM, Chris Hostetter hossman_luc...@fucit.org wrote: there is however a group.cache.percent option tha you might look into -- but i honestly have no idea if that toggles the use of queryResultCache or something else, i havn't played with it before... That's only a single-request cache (caches some ids/scores within a single request and is not reused across different requests). -Yonik http://lucidworks.com
