Re: Monitoring SolrCloud health
Mark, while thinking about it .. perhaps it'd would a good idea to compute the needed information somewhere in a request-handler/servlet, so that not every tool which shows the health needs to do that on their own? On Sunday, April 15, 2012 at 3:00 AM, Jamie Johnson wrote: Otis, Is SMP a hosted service? My application is not something that is publicly available so I need a solution that I can install and monitor the system with. Zabbix seems very interesting. I will give this a whirl. On Sat, Apr 14, 2012 at 7:20 PM, Lance Norskog goks...@gmail.com (mailto:goks...@gmail.com) wrote: SPM looks cool! If you can get analytics out for the numbers analysts like: search quality (precision, recall, MRR etc.), effectiveness of recommendations, drop-down keystrokes etc., you've got a winner. On Sat, Apr 14, 2012 at 2:20 PM, Otis Gospodnetic otis_gospodne...@yahoo.com (mailto:otis_gospodne...@yahoo.com) wrote: Jamie, We have Performance Monitoring for Solr (+HBase, ElasticSearch, and some other things...). I don't think we've tested it with SolrCloud yet, but that is pretty much next big item for SPM for Solr. The easiest way to find out when SPM for SolrCloud is available is by following @sematext - http://twitter.com/sematext Otis Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html From: Jamie Johnson jej2...@gmail.com (mailto:jej2...@gmail.com) To: solr-user@lucene.apache.org (mailto:solr-user@lucene.apache.org) Sent: Saturday, April 14, 2012 1:58 PM Subject: Re: Monitoring SolrCloud health ah, one last piece, and if you're not some how alert an admin about it, perhaps through email or something. Maybe this question is more application availability monitoring in general? Any opinions would be appreciated. On Sat, Apr 14, 2012 at 1:57 PM, Jamie Johnson jej2...@gmail.com (mailto:jej2...@gmail.com) wrote: Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com (mailto:markrmil...@gmail.com) wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com (http://lucidimagination.com) -- Lance Norskog goks...@gmail.com (mailto:goks...@gmail.com)
Re: Monitoring SolrCloud health
On Apr 15, 2012, at 11:01 AM, Stefan Matheis wrote: Mark, while thinking about it .. perhaps it'd would a good idea to compute the needed information somewhere in a request-handler/servlet, so that not every tool which shows the health needs to do that on their own? Yeah, I guess the hard part about that is that if you are not asking zookeeper, you don't know what server you can talk to (any given server may not be up or may go down). - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
I took a quick look at Zabbix and specifically the lucid imagination template for solr but in trying to import it into Zabbix I don't see the template show up in the list to choose from. What version of zabbix is recommended to use with LWE and specifically the provided zabbix templates? I tried with the latest source available for download (as of last night anyway). I also saw that there was some naggios checks available, but they seemed much more limited, has anyone implemented anything more advanced in naggios? We already have that in our environment so would be nice to plug into it if possible. On Sun, Apr 15, 2012 at 12:39 PM, Mark Miller markrmil...@gmail.com wrote: On Apr 15, 2012, at 11:01 AM, Stefan Matheis wrote: Mark, while thinking about it .. perhaps it'd would a good idea to compute the needed information somewhere in a request-handler/servlet, so that not every tool which shows the health needs to do that on their own? Yeah, I guess the hard part about that is that if you are not asking zookeeper, you don't know what server you can talk to (any given server may not be up or may go down). - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
I think I found the answer here http://lucidworks.lucidimagination.com/display/lweug/Integrating+Monitoring+Services#IntegratingMonitoringServices-HowtointegratewithZabbix2.0%281.9.x%29 I'll go ahead and try on the 1.9 baseline and see if I have any different result. On Sun, Apr 15, 2012 at 3:46 PM, Jamie Johnson jej2...@gmail.com wrote: I took a quick look at Zabbix and specifically the lucid imagination template for solr but in trying to import it into Zabbix I don't see the template show up in the list to choose from. What version of zabbix is recommended to use with LWE and specifically the provided zabbix templates? I tried with the latest source available for download (as of last night anyway). I also saw that there was some naggios checks available, but they seemed much more limited, has anyone implemented anything more advanced in naggios? We already have that in our environment so would be nice to plug into it if possible. On Sun, Apr 15, 2012 at 12:39 PM, Mark Miller markrmil...@gmail.com wrote: On Apr 15, 2012, at 11:01 AM, Stefan Matheis wrote: Mark, while thinking about it .. perhaps it'd would a good idea to compute the needed information somewhere in a request-handler/servlet, so that not every tool which shows the health needs to do that on their own? Yeah, I guess the hard part about that is that if you are not asking zookeeper, you don't know what server you can talk to (any given server may not be up or may go down). - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
Can you be more specific about health? On Sat, 2012-04-14 at 00:03 -0400, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface?
Re: Monitoring SolrCloud health
On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
ah, one last piece, and if you're not some how alert an admin about it, perhaps through email or something. Maybe this question is more application availability monitoring in general? Any opinions would be appreciated. On Sat, Apr 14, 2012 at 1:57 PM, Jamie Johnson jej2...@gmail.com wrote: Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
Jamie, We have Performance Monitoring for Solr (+HBase, ElasticSearch, and some other things...). I don't think we've tested it with SolrCloud yet, but that is pretty much next big item for SPM for Solr. The easiest way to find out when SPM for SolrCloud is available is by following @sematext - http://twitter.com/sematext Otis Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html From: Jamie Johnson jej2...@gmail.com To: solr-user@lucene.apache.org Sent: Saturday, April 14, 2012 1:58 PM Subject: Re: Monitoring SolrCloud health ah, one last piece, and if you're not some how alert an admin about it, perhaps through email or something. Maybe this question is more application availability monitoring in general? Any opinions would be appreciated. On Sat, Apr 14, 2012 at 1:57 PM, Jamie Johnson jej2...@gmail.com wrote: Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com
Re: Monitoring SolrCloud health
Other cool options: Zabbix collects from many things include Solr JVM JMX beans, which is what the solr/admin/stats.jsp page fetches. Zabbix fetches, archives, graphs and alerts. We could not find another monitor that did all of these well. NewRelic is a hosted service for Solr and a lot of other things. It is a JVM bolt-on which sends to NewRelic- there is zero configuration. We use both for monitoring in our LucidWorks for Cloud service. Zabbix is our workhorse. NewRelic is just internal eye candy: it looks great on a big monitor in the office and mesmerizes our managers :) On Sat, Apr 14, 2012 at 10:58 AM, Jamie Johnson jej2...@gmail.com wrote: ah, one last piece, and if you're not some how alert an admin about it, perhaps through email or something. Maybe this question is more application availability monitoring in general? Any opinions would be appreciated. On Sat, Apr 14, 2012 at 1:57 PM, Jamie Johnson jej2...@gmail.com wrote: Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com -- Lance Norskog goks...@gmail.com
Re: Monitoring SolrCloud health
SPM looks cool! If you can get analytics out for the numbers analysts like: search quality (precision, recall, MRR etc.), effectiveness of recommendations, drop-down keystrokes etc., you've got a winner. On Sat, Apr 14, 2012 at 2:20 PM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: Jamie, We have Performance Monitoring for Solr (+HBase, ElasticSearch, and some other things...). I don't think we've tested it with SolrCloud yet, but that is pretty much next big item for SPM for Solr. The easiest way to find out when SPM for SolrCloud is available is by following @sematext - http://twitter.com/sematext Otis Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html From: Jamie Johnson jej2...@gmail.com To: solr-user@lucene.apache.org Sent: Saturday, April 14, 2012 1:58 PM Subject: Re: Monitoring SolrCloud health ah, one last piece, and if you're not some how alert an admin about it, perhaps through email or something. Maybe this question is more application availability monitoring in general? Any opinions would be appreciated. On Sat, Apr 14, 2012 at 1:57 PM, Jamie Johnson jej2...@gmail.com wrote: Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com -- Lance Norskog goks...@gmail.com
Re: Monitoring SolrCloud health
Otis, Is SMP a hosted service? My application is not something that is publicly available so I need a solution that I can install and monitor the system with. Zabbix seems very interesting. I will give this a whirl. On Sat, Apr 14, 2012 at 7:20 PM, Lance Norskog goks...@gmail.com wrote: SPM looks cool! If you can get analytics out for the numbers analysts like: search quality (precision, recall, MRR etc.), effectiveness of recommendations, drop-down keystrokes etc., you've got a winner. On Sat, Apr 14, 2012 at 2:20 PM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: Jamie, We have Performance Monitoring for Solr (+HBase, ElasticSearch, and some other things...). I don't think we've tested it with SolrCloud yet, but that is pretty much next big item for SPM for Solr. The easiest way to find out when SPM for SolrCloud is available is by following @sematext - http://twitter.com/sematext Otis Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html From: Jamie Johnson jej2...@gmail.com To: solr-user@lucene.apache.org Sent: Saturday, April 14, 2012 1:58 PM Subject: Re: Monitoring SolrCloud health ah, one last piece, and if you're not some how alert an admin about it, perhaps through email or something. Maybe this question is more application availability monitoring in general? Any opinions would be appreciated. On Sat, Apr 14, 2012 at 1:57 PM, Jamie Johnson jej2...@gmail.com wrote: Right now my biggest concern is if the systems are all up and running. It would be nice to be able to see the stats that are provided on the current solr admin page across the cluster though, but again that's not my biggest issue now. Just want to know, are you up and running. On Sat, Apr 14, 2012 at 9:56 AM, Mark Miller markrmil...@gmail.com wrote: On Apr 14, 2012, at 12:03 AM, Jamie Johnson wrote: How do people currently monitor the health of a solr cluster? Are there any good tools which can show the health across the entire cluster? Is this something which is planned for the new admin user interface? Work on it happening here: https://issues.apache.org/jira/browse/SOLR-3174 Visualize Cluster State - Mark Miller lucidimagination.com -- Lance Norskog goks...@gmail.com
