Re: Shield: node-to-node communication performance
Well...this is hardly a satisfactory answer. Of course I expect a slowdown because encryption takes down. But how much, and what data does shield encrypt (e.g. only the initial authentication step or every bit of communication)? For example, I would not be surprised if Shield does the simplest thing and naively encrypt every communication between nodes as they happen, i.e. a SSL wrap on top of the transport layer. In that case, it would be super easy to estimate the type of slowdown that one could expect, and similarly back out how much slowdown under a specific cluster setup and data pattern. My inquiry is really to ask for more detailed information. 1. Could you outline how the node-to-node communication is encrypted? 2. Using 1, could you explain, via an example, when the slowdown is minimum, and also via a different example, when the slowdown is significant? Best, Jin On Thursday, January 29, 2015 at 1:00:32 AM UTC-8, David Pilato wrote: > > Shay twitted this about this matter: > https://twitter.com/kimchy/status/560124652472008704 > > Shay Banon @kimchy <https://twitter.com/kimchy> > Follow <https://twitter.com/kimchy> > @m_hughes <https://twitter.com/m_hughes> yes, it affects performance, > though less now with newer JVMs @dadoonet <https://twitter.com/dadoonet> @ > elasticsearch <https://twitter.com/elasticsearch> > 6:18 PM - 27 Jan 2015 > <https://twitter.com/kimchy/status/560124652472008704> > Not specific numbers here though. > > > -- > *David Pilato* | *Technical Advocate* | *Elasticsearch.com > <http://Elasticsearch.com>* > @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr > <https://twitter.com/elasticsearchfr> | @scrutmydocs > <https://twitter.com/scrutmydocs> > > > > Le 29 janv. 2015 à 09:56, Jin Huang > a > écrit : > > Hi, > > Can anyone shed some light on the impact of Shield on performance, > assuming that secured communication is enabled for node to node > communication? > > When Elasticsearch team says that node-to-node encryption is enabled, does > it mean that every bit of data transported on port 9300 is encrypted? Since > the whole cluster could transfer a huge amount of data across different > nodes constantly, would this encryption step severely lower the performance > of the cluster? > > Doe the Elasticsearch team have some ready-made benchmark data to share? > > Can someone elaborate on the architecture? > > Thanks, > Jin > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearc...@googlegroups.com . > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/8af7a106-0365-49a1-a0be-38eb1a7c0514%40googlegroups.com > > <https://groups.google.com/d/msgid/elasticsearch/8af7a106-0365-49a1-a0be-38eb1a7c0514%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9eca170f-8bf9-4dda-afee-f38c099174b5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Shield: node-to-node communication performance
Hi, Can anyone shed some light on the impact of Shield on performance, assuming that secured communication is enabled for node to node communication? When Elasticsearch team says that node-to-node encryption is enabled, does it mean that every bit of data transported on port 9300 is encrypted? Since the whole cluster could transfer a huge amount of data across different nodes constantly, would this encryption step severely lower the performance of the cluster? Doe the Elasticsearch team have some ready-made benchmark data to share? Can someone elaborate on the architecture? Thanks, Jin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8af7a106-0365-49a1-a0be-38eb1a7c0514%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Metrics Aggregations: custom stats functions, script stats, user-defined metrics, custom metrics
Hi people, Can someone provide some detailed information on how to extend the sets of currently available metrics in the aggregation framework? For now, Elasticsearch provides support for the metrics: count, min, max, avg, sum, sum_of_squares, variance, and std_deviation. How can I define my own? I understand that there might some hard constraints on the type of metrics could be defined. But it would be nice if someone can provide a simple example of how to write my own. Here is a couple of suggested examples that I would want to know how to write: 1. I want to calculate all the n-th central moments, with n=2,3,4,..10. By the way, Elasticsearch already provides the 2nd central moment, i.e. square of the standard deviation. 2. I want to get the second order statics. That is, I want to the second smallest number. In general, I would like to know what kind of custom aggregator is easy to write? If the above examples require a deep dive into some of the source codes, please DO! I don't mind if the answer runs 10 pages long as long as someone can provide me with some pointers on how to get started. Thanks, Jin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/814573e2-50e8-4dbd-ba4a-192c932a03fc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Custom Aggregations
Hi Alex, Is there an updates on this topic? I want to know how easy or how complex is it to write a custom aggregator. For example, I want to calculate all the n-th central moments, with n=2,3,4,..10. By the way, Elasticsearch already provides the 2nd central moment, i.e. square of the standard deviation. In general, I would like to know what kind of custom aggregator is easy to write? Thanks, Jin On Wednesday, February 12, 2014 5:49:28 AM UTC-8, Alexander Reelsen wrote: > > Hey, > > sure, you can write a custom plugin, which implements your custom > aggregations, if you want to. > Any specific functionality you are missing (might be interesting to get it > in the core anyway?). > > > --Alex > > > On Tue, Feb 11, 2014 at 9:41 PM, Justin Uang > wrote: > >> Is there any way we can define our own aggregation functions beyond the >> provided metric and bucket aggregations? >> >> Thanks! >> >> Justin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elasticsearc...@googlegroups.com . >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/90accf78-6540-4126-8268-7f997353ad74%40googlegroups.com >> . >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b2b8894d-c26d-4787-af4c-c45f793befc6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
