Re: Multiple Facet prefixes on the same facet field in one request?
Well I looked at SOLR-1351 and applied the patch and that seems to do everything that I need. The local param syntax definitely the way to go. Robert, On 2009-12-11, at 2:53 AM, Shalin Shekhar Mangar wrote: On Thu, Dec 10, 2009 at 9:47 PM, Robert Purdy r...@shoptoit.ca wrote: Hey there Shalin, After scrolling through the Jira Issues I found, http://issues.apache.org/jira/browse/SOLR-1387, discussing this exact thing.Is anyone currently working on a patch for this issue still? If not has the syntax in the last post in the Jira issue been approved so a patch can be made? SOLR-1387 is not final. There's also SOLR-1351. A local param based syntax looks like the right way to go. However, we have not reached consensus yet. You are welcome to take them forward. -- Regards, Shalin Shekhar Mangar.
Re: Multiple Facet prefixes on the same facet field in one request?
Hey there Shalin, After scrolling through the Jira Issues I found, http://issues.apache.org/jira/browse/SOLR-1387, discussing this exact thing.Is anyone currently working on a patch for this issue still? If not has the syntax in the last post in the Jira issue been approved so a patch can be made? Thanks Robert. On 2009-12-10, at 2:19 AM, Shalin Shekhar Mangar wrote: On Wed, Dec 9, 2009 at 5:25 AM, Robert Purdy rdpu...@gmail.com wrote: Hey all, Is there anyway in Solr 1.4/1.5 to perform multiple facet prefixes on the same facet field in one request? Ex. On field 'Foo' I want to perform a facet prefix of A* and B* so I can get a facet response of all terms prefixed with A and all terms prefixed with B, either grouped together in the same facet result list or seperate facet lists labeled by the prefix. Currently, I perform one request per facet prefix and I am hoping that there is some cryptic way using local params that I am missing that will allow me to do this. No, I don't think there is a way to do request for more than one facet.prefix on the same field in one request. -- Regards, Shalin Shekhar Mangar.
Multiple Facet prefixes on the same facet field in one request?
Hey all, Is there anyway in Solr 1.4/1.5 to perform multiple facet prefixes on the same facet field in one request? Ex. On field 'Foo' I want to perform a facet prefix of A* and B* so I can get a facet response of all terms prefixed with A and all terms prefixed with B, either grouped together in the same facet result list or seperate facet lists labeled by the prefix. Currently, I perform one request per facet prefix and I am hoping that there is some cryptic way using local params that I am missing that will allow me to do this. Robert. -- View this message in context: http://old.nabble.com/Multiple-Facet-prefixes-on-the-same-facet-field-in-one-request--tp26702997p26702997.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Error sorting random field with June 1, 2009 Solr 1.4 nightly
Yes I am still getting the NPE. Robert. Shalin Shekhar Mangar wrote: On Tue, Jun 2, 2009 at 4:50 AM, Robert Purdy rdpu...@gmail.com wrote: Hey all, I was just wondering if anyone else is getting an error with today's nightly while sorting the random field. Thanks Rob. Jun 1, 2009 4:52:37 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NullPointerException at org.apache.lucene.search.SortField.getComparator(SortField.java:483) at org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQueue.init(FieldValueHitQueue.java:80) at org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.java:190) at org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java:851) at org.apache.solr.search.SolrIndexSearcher.sortDocSet(SolrIndexSearcher.java:1360) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:868) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:337) at Are you still seeing this issue? It may be related to SOLR-. Yonik mentioned that there might be a problem with RandomSortField: https://issues.apache.org/jira/browse/SOLR-?focusedCommentId=12713271page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12713271 -- Regards, Shalin Shekhar Mangar. -- View this message in context: http://www.nabble.com/Error-sorting-random-field-with-June-1%2C-2009-Solr-1.4-nightly-tp23824012p23899481.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Which caches should use the solr.FastLRUCache
Thanks for the Good information :) Well I haven't had any evictions in any of the caches in years, but the hit ratio is 0.51 in queryResultCache, 0.77 in documentCache, 1.00 in the fieldValueCache, and 0.99 in the filterCache. So in your opinion should the documentCache and queryResultCache use the old way on a single CPU quad core machine? Also right now I have all caches using the solr.FastLRUCache (tried with both the cleanupThread = false or true) and I have noticed some queries that are taking 53 ms on a freshly warmed new searcher (when nothing else is querying the slave), but when the slave is busy the same query, that should be using the caches, is sometimes taking 8 secs? Any thoughts? Thanks Robert. Yonik Seeley-2 wrote: 2009/6/4 Noble Paul നോബിള് नोब्ळ् noble.p...@corp.aol.com: FastLRUCache is designed to be lock free so it is well suited for caches which are hit several times in a request. I guess there is no harm in using FastLRUCache across all the caches. Gets are cheaper, but evictions are more expensive. If the cache hit rate is low, the old synchronized cache may be faster, unless you have a ton of CPUs... not sure where the crossover point is though. -Yonik http://www.lucidimagination.com -- View this message in context: http://www.nabble.com/Which-caches-should-use-the-solr.FastLRUCache-tp23860182p23874898.html Sent from the Solr - User mailing list archive at Nabble.com.
Which caches should use the solr.FastLRUCache
Hey there, Anyone got any advice on which caches (filterCache, queryResultCache, documentCache, fieldValueCache) should be implemented using the solr.FastLRUCache in solr 1.4 and what are the pros cons vs the solr.LRUCache. Thanks Robert. -- View this message in context: http://www.nabble.com/Which-caches-should-use-the-solr.FastLRUCache-tp23860182p23860182.html Sent from the Solr - User mailing list archive at Nabble.com.
Error sorting random field with June 1, 2009 Solr 1.4 nightly
Hey all, I was just wondering if anyone else is getting an error with today's nightly while sorting the random field. Thanks Rob. Jun 1, 2009 4:52:37 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NullPointerException at org.apache.lucene.search.SortField.getComparator(SortField.java:483) at org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQueue.init(FieldValueHitQueue.java:80) at org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.java:190) at org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java:851) at org.apache.solr.search.SolrIndexSearcher.sortDocSet(SolrIndexSearcher.java:1360) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:868) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:337) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:176) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1328) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:51) at org.apache.solr.core.SolrCore$4.call(SolrCore.java:1158) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269) at java.util.concurrent.FutureTask.run(FutureTask.java:123) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:613) -- View this message in context: http://www.nabble.com/Error-sorting-random-field-with-June-1%2C-2009-Solr-1.4-nightly-tp23824012p23824012.html Sent from the Solr - User mailing list archive at Nabble.com.
Dynamic autowarming queries
Hey there, is there a current way to specify dynamic autowarming queries in the solrconfig.xml? If not is it possible for the solrconfg.xml to read from a file that I can update on the fly to warm new dynamic fields in the schema or just new queries to warm to prevent a restart of solr and so the changes are picked up each time a new searcher is warmed? Also is it possible to warm a dynamic field ie (category_*) in the warming section in the solrconfig.xml? If so what does it store in the cache if say I have category_1, category_2, category_3, category_4 dynamically stored in the index? Does it make a separate bitset for each field or does it combine it all of them somehow? Thanks Robert. -- View this message in context: http://www.nabble.com/Dynamic-autowarming-queries-tp14339575p14339575.html Sent from the Solr - User mailing list archive at Nabble.com.
Performance question: Solr 64 bit java vs 32 bit mode.
Would anyone know if solr runs better in 64bit java vs 32 bit and could answer another possible related question. I currently have two servers running solr under identical tomcat installations. One is the production server and is under heavy user load and the other is under no load at all because it is a test box. I was looking in the logs on the production server and noticed some queries were taking about 15 seconds, and this is after auto-warming. So I decided to execute that same query on the other server with nothing in the caches and found that it only took 2 seconds to complete. My question is why an Dual Intel Core Duo Xserve server in 64 bit java mode with 8GB of ram allocated to the tomcat server be slower than a Dual Power PC G5 server running in 32 bit mode with only 2GB of ram allocated? Is it because of the load/concurrrency issues on the production sever that made the time next to the query in the log greater on the production server? If so what is the best way to configure tomcat to deal with that issue? Thanks Robert. -- View this message in context: http://www.nabble.com/Performance-question%3A-Solr-64-bit-java-vs-32-bit-mode.-tf4817186.html#a13781791 Sent from the Solr - User mailing list archive at Nabble.com.
Major CPU performance problems under heavy user load with solr 1.2
Hi there, I am having some major CPU performance problems with heavy user load with solr 1.2. I currently have approximately 4 million documents in the index and I am doing some pretty heavy faceting on multi-valued columns. I know that doing facets are expensive on multi-valued columns but the CPU seems to max out (400%) with apache bench with just 5 identical concurrent requests and I have the potential for a lot more concurrent requests then that with my large number of users that hit our site per day and I am wondering if there are any workarounds. Currently I am running the out of the box solr solution (Example jetty application with my own schema.xml and solrconfig.xml) on a dual Intel Duo core 64 bit box with 8 gigs of ram allocated to the start.jar process dedicated to solr with no slaves. I have set up some aggressive caching in the solrconfig.xml for the filtercache (class=solr.LRUCachesize=300 initialSize=200) and have the HashDocSet set to 1 to help with faceting, but still I am getting some pretty poor performance. I have also tried autowarming the facets by performing a query that hits all my multivalued facets with no facet limits across all the documents in the index. This does seem to reduce my query times by a lot because the filtercache grows to about 2.1 million lookups and finishes the query in about 70 secs. However I have noticed an issue with this because each time I do an optimize or a commit after prewarming the facets the cache gets cleared, according to the stats on the admin page, but the RSize does not shink for the process, and the queries get slow again, so I prewarm the facets again and the memory usage keeps growing like the cache is not being recycled and as a results the prewarm query starts to get slower and slower as each time this occurs (after about 5 times of prewarms and then commit the query takes about 30 mins... ugh) and almost run out of memory. Any thoughts on how to help improve this and fix the memory issue? -- View this message in context: http://www.nabble.com/Major-CPU-performance-problems-under-heavy-user-load-with-solr-1.2-tf4549093.html#a12981540 Sent from the Solr - User mailing list archive at Nabble.com.
Schema version question
I was wondering if anyone could help me, I just completed a full index of my data (about 4 million documents) and noticed that when I was first setting up the schema I set the version number to 1.2 thinking that solr 1.2 uses schema version 1.2... ugh... so I am wondering if I can just set the schema to 1.1 without having to rebuild the full index? I ask because I am hoping that given an invalid schema version number, that version 1.0 is not used by default and all my fields are now mulitvalued. Any help would be greatly appreciated. Thanks in advance -- View this message in context: http://www.nabble.com/Schema-version-question-tf4536802.html#a12948588 Sent from the Solr - User mailing list archive at Nabble.com.
Facet prefix question
Hey there does anyone know how to perform multiple facet prefixes on the same field. I am using the f.fieldname.facet.prefix syntax and cannot get an ORing of faceted results. Thanks Robert. -- View this message in context: http://www.nabble.com/Facet-prefix-question-tf4052754.html#a11511601 Sent from the Solr - User mailing list archive at Nabble.com.
