Index Solr Logs
I'm interested to know if there is a way to have Solr index its own logs, in particular the logging of queries. One project that showed promise was Sogger but I believe the developer is working more closely with LogStash which uses ElasticSearch so my guess is that the Sogger project is no longer being developed. Has anyone else had experience with this and can share their thoughts/findings/solution? Cheers hayden
Re: Advice wanted on approach/architecture
When marking a line in your app, you could re-index that line with a field set to hide=true so that subsequent queries will not show that line. Regarding regex exclude: Will that be same for all users of system or is it per user? If it is system-wide I'd consider using a back-end job which re-indexes matching lines with a field exclude=true. This is more efficient than running all the regexes query-time. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training - www.solrtraining.com On 24. juni 2011, at 14.54, Js wrote: Hi List, I'm looking into some options on what technology to adopt building a specific logfile search solution. At first glance it looks like Solr is the tool I'm looking for. I intend to write a web-based front end for end users What would be a possible approach to tackle following requirements? In other words how could these requirements be translated into Solr on a high level. I'm not asking for solutions, just pointers, approaches, tips, Solr features to look at, possible pitfalls, ... - A query results into a set of results. - Individual records from this query should have the ability to be marked so (although they match the query) those specific records don't show anymore when the same query is rerun. - I don't want to delete data from the db/index - I want to avoid that my application has to take care of excluding parts of the returned data by keeping track which record id's to exclude. - A query should exclude the records which have a match in a possibly large growing list of regexes. Thanks! Jelle
Using RAMDirectoryFactory in Master/Slave setup
PS: Sorry if this is a repost, I was unable to see my message in the mailing list - this may have been due to my outgoing email different from the one I used to subscribe to the list with. Overview – Trying to evaluate if keeping the index in memory using RAMDirectoryFactory can help in query performance.I am trying to perform the indexing on the master using solr.StandardDirectoryFactory and make those indexes accesible to the slave using solr.RAMDirectoryFactory Details: We have set-up Solr in a master/slave enviornment. The index is built on the master and then replicated to slaves which are used to serve the query. The replication is done using the in-built Java replication in Solr. On the master, in the indexDefaults of solrconfig.xml we have directoryFactory name=DirectoryFactory class=solr.StandardDirectoryFactory/ On the slave, I tried to use the following in the indexDefaults directoryFactory name=DirectoryFactory class=solr.RAMDirectoryFactory/ My slave shows no data for any queries. In solrconfig.xml it is mentioned that replication doesn’t work when using RAMDirectoryFactory, however this ( https://issues.apache.org/jira/browse/SOLR-1379) mentions that you can use it to have the index on disk and then load into memory. To test the sanity of my set-up, I changed solrconfig.xml in the slave to and replicated: directoryFactory name=DirectoryFactory class=solr.StandardDirectoryFactory/ I was able to see the results. Shouldn’t RAMDirectoryFactory be used for reading index from disk into memory? Any help/pointers in the right direction would be appreciated. Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Using-RAMDirectoryFactory-in-Master-Slave-setup-tp3111792p3111792.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr clustering configuration
Hi, I'm looking for cluster configuration(multiple Solr servers) instructions for Solr. Can you please post a link for Solr cluster configuration if you familiar with such kind of document? Thanks, Shay -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-clustering-configuration-tp3109965p3109965.html Sent from the Solr - User mailing list archive at Nabble.com.
Using RAMDirectoryFactory in Master/Slave setup
Overview Trying to evaluate if keeping the index in memory using RAMDirectoryFactory can help in query performance.I am trying to perform the indexing on the master using solr.StandardDirectoryFactory and on the slave using solr.RAMDirectoryFactory Details: We have set-up Solr in a master/slave enviornment. The index is built on the master and then replicated to slaves which are used to serve the query. The replication is done using the in-built Java replication in Solr. On the master, in the indexDefaults of solrconfig.xml we have directoryFactory name=DirectoryFactory class=solr.StandardDirectoryFactory/ On the slave, I tried to use the following in the indexDefaults directoryFactory name=DirectoryFactory class=solr.RAMDirectoryFactory/ My slave shows no data for any queries. In solrconfig.xml it is mentioned that replication doesn¹t work when using RAMDirectoryFactory, however this ( https://issues.apache.org/jira/browse/SOLR-1379) mentions that you can use it to have the index on disk and then load into memory. To test the sanity of my set-up, I changed solrconfig.xml in the slave to and replicated: directoryFactory name=DirectoryFactory class=solr.StandardDirectoryFactory/ I was able to see the results. Shouldn¹t RAMDirectoryFactory be used for reading index from disk into memory? Any help/pointers in the right direction would be appreciated. Thanks!
Re: Index Solr Logs
Check out Logg.ly. http://www.loggly.com/. They use SOLR to index all kinds of logs, SOLR included. This is a paid service, so maybe not what you're looking for. I've used it though, works great. -Mike On Sun, Jun 26, 2011 at 5:49 AM, Mr Havercamp mrhaverc...@gmail.com wrote: I'm interested to know if there is a way to have Solr index its own logs, in particular the logging of queries. One project that showed promise was Sogger but I believe the developer is working more closely with LogStash which uses ElasticSearch so my guess is that the Sogger project is no longer being developed. Has anyone else had experience with this and can share their thoughts/findings/solution? Cheers hayden
Re: MultiValued facet behavior question
: And I click on B, does it make sense for the user to display : B : C : E : : after the selection ? Just because items in B are C and E items as well? : As A user I chose B because I'm interested in B items. I do not care if they : are also C and E items. : Technically this is correct, but functional wise, the user doesn't care : because it is not what they searched for. if that's how you want your UI to behavior, then it should be trivial to say once the user clicks on a constraint in the $fieldName field, stop displaying facets for $fieldName -- ie: if you add fq=$fieldName:fieldValue to your URL, then don't add facet.field=$fieldName to that same URL. If you want the same behavior anytime the users query inadvertantly *happens* to be a query that only matches one of many values in a mutifaceted field (ie: the user searches for brown hair and all brown haired docors just happen to be cardiologiests) then you can still deal with this in your UI code by implementing logic such as: * check the numFound for the main query * for each facet: * for each constraint: * if the constraint count is the same as numFound, skip this facet -Hoss
SolrDocumentList in Distributed search
Hi All I have 5 shards. (sh01 ~ sh05) I was debugging using solrJ. When I quiried at each shard, results are right. But when I quiried at all shards, elementData of SolrDocumentList is null. But numFound of SolrDocumentList is right. How can I get the SolrDocumentList in shards? Thanks in Advance -- View this message in context: http://lucene.472066.n3.nabble.com/SolrDocumentList-in-Distributed-search-tp3112580p3112580.html Sent from the Solr - User mailing list archive at Nabble.com.
