Re: How to get mappings using the Java API with ElasticSearch 0.90.10 and 1.0.0 RC1
Just a side note that the API response format changed in 1.0.0RC1. http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/_indices_apis.html On Tuesday, January 28, 2014 3:15:57 AM UTC-5, Thierry Templier wrote: Hello, I wonder what is the best way to get the mappings for an index using the ElasticSearch Java API with version 0.90.10. I saw that there is only a GetFieldMappingsRequest class for this version whereas a class GetMappingsRequest is now present in 1.0RC1. Thanks very much for your answer and your help! Thierry -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/568456a2-c76b-43ef-ac0a-76b1bd4faffd%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: For those wanting to try Marvel
Although I agree with the code transparency in principle, it isn't a sound business idea. Scaling an OSS company on support/services revenue alone is very linear and that's not something investors want. You also risk becoming a consulting company, rather than a software distributor. In the end, all successful OSS companies have a commercial component. This is ES's first stab at it and I think it's a good one. On Tuesday, January 28, 2014 11:02:50 PM UTC-5, Lukáš Vlček wrote: If my memory serves me well one of the issues with plugins is missing classloading isolation. If you have two plugins installed on a single node they can clash (we have been hit by this with our own plugins and it took us some time to figure it out because the symptom was not deterministic). If Marvel contains Java code that is installed as a plugin it would be imo useful if the code were a bit more transparent, especially for plugin developers. Regards, Lukáš On Wed, Jan 29, 2014 at 2:42 AM, Ivan Brusic iv...@brusic.comjavascript: wrote: Marvel doesn't different much from elasticsearch plugins in that the code now runs in the same JVM instead of a separate process. The event data is pushed rather than pulled. It is great not having to re-invent the wheel, but having monitoring outside of elasticsearch is not an issue. Great observation about the tribe node BTW. Given that efficient JSON parsing libraries exist in most languages, I rather go that route over the cat API. Time to re-visit monitoring. Ivan On Tue, Jan 28, 2014 at 5:33 PM, Mark Walkom ma...@campaignmonitor.comjavascript: wrote: Or perhaps something could come from the new cat API if they don't want to go that route. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com javascript: web: www.campaignmonitor.com On 29 January 2014 12:21, joerg...@gmail.com javascript: joerg...@gmail.com javascript: wrote: Marvel comes with hidden Java plugin code for an event pusher. Node events, route events, and shard events of ES can be indexed into ES. Very useful for historic analysis and post mortem views. It seems this was also a motivation for the tribe node mode: grow two separate clusters, one for the data, another one for the metrics. It would be nice if also the event pushing source code could be opened, so other monitoring tools are able to build on this facility too. Or at least documenting the event pushing API, for re-implementing it from scratch. Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-JaWZU4qFQsUaM_UHCwkzMdYnLruUHNmcnkiZ-wX6jA%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Yn5y1bxGYd%3DjeG%3DiUVzowhOsuzKi_8erKrPEmTNbxb%3DQ%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD%3Dh804DJBr2J8V3gDvkSK0ddXHvg0xDBf9XYzUza_n5w%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6de0c52f-f0f8-45d5-b19d-32c149bc1fe2%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: A Question on Plugin redundancy
Lukas, Yes, very similar approach in tech, but different in distribution model. The use of RRD makes sense for them, as I believe their offering monitors more than just ES clusters. It's a New Relic type of model. Alternatively, New Relic has ES monitoring plugins available as well. On Wednesday, January 22, 2014 12:00:11 PM UTC-5, Lukáš Vlček wrote: Roy, Sounds a bit like similar approach to Sematext SPM to me (except they use CollectD - which is based on RRD is I am not mistaken) - but that shouldn't stop you. As for RHQ it is upstream for JON (JBossON). See http://planet.jboss.org/post/jboss_operations_network_jbosson_jon_rhq_whats_in_a_name I think it might have been renamed (this happens in JBoss world :-)). Regards, Lukáš -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a07a40ac-e1fa-4554-834f-6ab8a268ec24%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: A Question on Plugin redundancy
Hi David, Thanks for replying. The problem is that if you have a plugin that lets say is responsible for polling ES _/node/stats and pushing the data somewhere, you don't want 100 nodes doing the same thing (that's a rough example). It just seems like it's a central point of failure, but if deployed per-node, you end up with duplicate data or duplicate service calls with #X-simultaneously-running plugins. You have written a lot of plugins/rivers. Are your meant to be deployed singularly? On Tuesday, January 21, 2014 2:34:49 PM UTC-5, David Pilato wrote: What's wrong with installing plugins in all nodes? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 21 janv. 2014 à 19:57, Roy Russo royr...@gmail.com javascript: a écrit : Hello all, Curious here on what is the recommended way of plugin deployment in a cluster, for redundancy on that plugin. Say I have 10 nodes... I deploy a plugin on Node1. Node1 dies. Now what? I may have a new master node, but my plugin is gone. Is there an idea of deploying plugins on every node, and having only the elected master be the active plugin? Excuse me if it's a dumb question. I can't seem to find the answer anywhere. Regards, Roy Russo http://www.elastichq.org/ -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8cd60da6-7101-4a0d-9d63-d8e6ceac5c5a%40googlegroups.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9df71edf-fb1e-4438-8ac3-1f9c21a8b666%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: A Question on Plugin redundancy
Hi Lukas, Somewhat... The idea is a monitoring 'service'. Hosted. Where its hosted is a separate conversation. I agree with most everything you stated. Having a local agent running is the way to go. I'm avoiding remote polling and little to no advantage in distributing as a plugin. The one advantage is that it is very clean and simple to install, but such a tight dependency has its downsides, as you mentioned. I don't agree on not using java. Folks using elasticsearch already have a JVM installed obviously, so its an easy jar drop and run. I'm not sure why you would consider it a heavy process. Have you been looking at my old jboss code? ;-) rrd is lighter. No objection there. So is Perl. At first the data from elasticsearch is meant to be condensed and sent across the wire... Small payloads. At some point, I may move to storing the full blobs, but that can task a system and certainly add to hardware costs for retention. Across x,000 clusters, storing monitoring metrics form histograms and analysis can get heavy. On Jan 21, 2014 11:24 PM, Lukáš Vlček lukas.vl...@gmail.com wrote: Roy, if you are in fact talking about implementation of monitoring system for ES then the following are my 2 cents: - regarding how to poll data from ES nodes, I would not implement it as a plugin or anything similar that is dependent on ES platform itself. It should be an external process and possibly very light process (Java process is not light). The impact on ES should be as minimal as possible and it should not rely on plugin system of ES either. I would consider looking at something like RRD tools (and extensions on top of it). Really lightweight process that sits on each node, kicks once a while, grabs all metrics via REST API and stores it into local storage. It should then try to send these data into central store but it may not be always available thus storing it locally is important in case the connection to central store is restored at later time. The other argument why not to use plugin system of ES is that you should be able to upgrade monitoring system independently on ES cluster - you can not do this with ES plugins. - should it poll the same stats from every single node? I think it should! If ES cluster gets into trouble is can be because not every part of the cluster sees the same picture. IMO the only way how to learn about this is collecting individual pictures from every node. It is also the most simple way of doing it (and you should definitely aim for simplicity). Solving duplicities on the central storage side later should be possible but you might not even need to do it if the size of collected data is not a problem for you. The part where you need to be very careful when polling the same stats from each node is making sure this is not putting high load on the ES cluster. If it is possible to poll from the local node only (like using _local in URL) then opt for it. Though I am no sure if this option is still available post 1.0.0.RC1 release. Regards, Lukas On Tue, Jan 21, 2014 at 10:09 PM, joergpra...@gmail.com joergpra...@gmail.com wrote: You must deploy plugins to all nodes, so you will still have the plugin available when nodes come and go. You could add an action so that you can turn on/off a plugin explicitly by remote command. Or you can define a condition so a plugin can decide when to run. For example if you want a singleton, you can check if you run on the master node, and execute only then. Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFQUHak0-PQ%3DLW3C0_0GQy8EwJThOKC__RhGgzGJk4MvA%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/UWiNFuu6bw4/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUbcy%2B-F1EdrOAJE%2Bh8Ud8JcbnpWpAf78ijZ4yO0jn6RYA%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANTOpgtZuTk%2BQkNii7SoZWtf%3DkQ%3DYJ5%2BxJP1i3kAa1Yw4M0k0Q%40mail.gmail.com. For more options, visit
1.0.0RC1 breaking changes : GC stats
Well, not sure if it qualifies as a breaking change, unless you cowboy code
javascript like I do ;-)
The node stats api (/_nodes/{this.nodeId}/stats?all=1) is returning a
different format for JVM GC now, splitting old and young generations (which
is really helpful)
I didn't notice this change in Beta1, so I thought I'd point it out and
maybe someone can/might update the doc.
gc: {
collectors: {
young: {
collection_count: 3,
collection_time_in_millis: 136
},
old: {
collection_count: 0,
collection_time_in_millis: 0
}
}
I also didn't notice previously the jvm returning pool information (another
helpful addition!)
pools: {
young: {
used_in_bytes: 30180776,
max_in_bytes: 279183360,
peak_used_in_bytes: 71630848,
peak_max_in_bytes: 279183360
},
survivor: {
used_in_bytes: 8912888,
max_in_bytes: 34865152,
peak_used_in_bytes: 8912896,
peak_max_in_bytes: 34865152
},
old: {
used_in_bytes: 21765456,
max_in_bytes: 724828160,
peak_used_in_bytes: 21765456,
peak_max_in_bytes: 724828160
}
}
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/da092537-d6b7-4870-81fa-ed599fe610ea%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[ANN] ElasticHQ v1.0.0 (ElasticSearch v1.0.0.RC1 Support)
Hello All, Pleased to announce the release of ElasticHQ v1.0.0. This release added: 1. *Support for ElasticSearch v1.0.0RC1* and unbroke the breaking changes. ;-) 2. Support for monitoring multiple file systems 3. Support for G1 GC 4. Allow user to select which nodes are displayed on the Diagnostics Screen *Every HQ release is always backwards compatible*, so there's no magic needed on your part. As always, you can get it here: http://www.elastichq.org/gettingstarted.html Regards, Roy Russo http://www.elastichq.org/blog/ -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8582fbfb-d5a0-4cc4-8310-08fbf7aa0456%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
1.0.0RC1 broken changes
Hello, So it appears a few APIs have changed in 1.0.0RC1 (which is bizarre considering I tested on master just last week and everything worked, but whatever...) Any ideas when the documentation will be updated to reflect the changes? I'm having a hard time mapping old API calls to the new urls... For instance, THIS: /_cluster/nodes/4oL7COyQTNiQPa4xZ76Pfg/stats?all=trueplugin=truehttp://localhost:9200/_cluster/nodes/4oL7COyQTNiQPa4xZ76Pfg/stats?all=trueplugin=true IS NOW WHAT? /_??? The breaking changes page is not fine-grained enough in mapping new to old calls... at least for me. ;-) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0333d260-1232-4f68-8681-88536cbefa5f%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: 1.0.0RC1 broken changes
Yep. Thanks! I discovered the new URL and the updated docs shortly after posting - they looked so similar, I thought the docs hadn't changed. Looks like the new paths actually make more sense, ala REST standards. On Wednesday, January 15, 2014 3:41:07 PM UTC-5, Matt Weber wrote: Use the master docs http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/index.html Looks like your call should be: /_nodes/4oL7COyQTNiQPa4xZ76Pfg/stats?all=trueplugin=true Thanks, Matt Weber On Wed, Jan 15, 2014 at 12:31 PM, Roy Russo royr...@gmail.comjavascript: wrote: Hello, So it appears a few APIs have changed in 1.0.0RC1 (which is bizarre considering I tested on master just last week and everything worked, but whatever...) Any ideas when the documentation will be updated to reflect the changes? I'm having a hard time mapping old API calls to the new urls... For instance, THIS: /_cluster/nodes/4oL7COyQTNiQPa4xZ76Pfg/stats?all=trueplugin=truehttp://localhost:9200/_cluster/nodes/4oL7COyQTNiQPa4xZ76Pfg/stats?all=trueplugin=true IS NOW WHAT? /_??? The breaking changes page is not fine-grained enough in mapping new to old calls... at least for me. ;-) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0333d260-1232-4f68-8681-88536cbefa5f%40googlegroups.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/70713f66-e925-40be-9cca-4459ff9a7e25%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
