Re: increase query performance by adding more machines, shouldn't it be linear to # of machines?

2014-07-01 Thread David Pilato 
Sorry. I meant on how many physical bare metal machines your 5 VMs are running?


--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 2 juil. 2014 à 07:59, Seungjin Lee  a écrit :

yes, all same machines on which only ES with same configuration is running


2014-07-02 14:55 GMT+09:00 David Pilato :
> Are you using same physical machine for all your VMs?
> 
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> 
> 
> Le 2 juil. 2014 à 07:09, Seungjin Lee  a écrit :
> 
> Hi all,
> 
> I'm testing percolator performance, 50k/s is required condition with 3~4k 
> rules.
> 
> now I only have 1 simple rule, and 5 es vms with 1 shard and 4 replicas.
> 
> and using Java transport client like below
> 
> new TransportClient(settings)
> .addTransportAddresses(transportAddressList.toArray(new 
> InetSocketTransportAddress[transportAddressList.size()]));
> 
> 
> when I added just 1 address to transport client, percolator perf is about 
> 10k/s
> 
> and when I added all 5 of vms to it, perf is about 15k/s
> 
> so it increases by about 1.5times only even though I added 4 more vm 
> addresses.
> 
> Is it supposed to be like this?
> 
> What I was thinking is, if it runs in for example round-robin fashion, there 
> should be about 5 times performance gain.
> 
> Could you comment anything on this?
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/CAL3_U412ecCDh0GONWJZy6S9zjY%2By8xvoVsPrt1849csfs-zUA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/BAEDE7D4-320E-4123-A783-823FF1EC26ED%40pilato.fr.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL3_U42vJaO0oqfw4DjWP6KK%3DpqtnGEBFGKHOr7Ky7nmbXV5HA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5BFE5974-6B7B-465A-A587-AEBDF6C27D23%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Re: increase query performance by adding more machines, shouldn't it be linear to # of machines?

2014-07-01 Thread Seungjin Lee 
yes, all same machines on which only ES with same configuration is running


2014-07-02 14:55 GMT+09:00 David Pilato :

> Are you using same physical machine for all your VMs?
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 2 juil. 2014 à 07:09, Seungjin Lee  a écrit :
>
> Hi all,
>
> I'm testing percolator performance, 50k/s is required condition with 3~4k
> rules.
>
> now I only have 1 simple rule, and 5 es vms with 1 shard and 4 replicas.
>
> and using Java transport client like below
>
> new TransportClient(settings)
> .addTransportAddresses(transportAddressList.toArray(new
> InetSocketTransportAddress[transportAddressList.size()]));
>
>
> when I added just 1 address to transport client, percolator perf is about
> 10k/s
>
> and when I added all 5 of vms to it, perf is about 15k/s
>
> so it increases by about 1.5times only even though I added 4 more vm
> addresses.
>
> Is it supposed to be like this?
>
> What I was thinking is, if it runs in for example round-robin fashion,
> there should be about 5 times performance gain.
>
> Could you comment anything on this?
>
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAL3_U412ecCDh0GONWJZy6S9zjY%2By8xvoVsPrt1849csfs-zUA%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/BAEDE7D4-320E-4123-A783-823FF1EC26ED%40pilato.fr
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL3_U42vJaO0oqfw4DjWP6KK%3DpqtnGEBFGKHOr7Ky7nmbXV5HA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: increase query performance by adding more machines, shouldn't it be linear to # of machines?

2014-07-01 Thread Patrick 
Also, what's the hardware configuration of the devices? RAM? CPU? Disk? OS?

How much data is in your ES instance(s) already? and are you sure you're
not hitting a limit on something like client side networking?

Thanks,
Patrick


Patrick

http://about.me/patrick.ancillotti
patrick  eefy  net


On Tue, Jul 1, 2014 at 11:55 PM, David Pilato  wrote:

> Are you using same physical machine for all your VMs?
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 2 juil. 2014 à 07:09, Seungjin Lee  a écrit :
>
> Hi all,
>
> I'm testing percolator performance, 50k/s is required condition with 3~4k
> rules.
>
> now I only have 1 simple rule, and 5 es vms with 1 shard and 4 replicas.
>
> and using Java transport client like below
>
> new TransportClient(settings)
> .addTransportAddresses(transportAddressList.toArray(new
> InetSocketTransportAddress[transportAddressList.size()]));
>
>
> when I added just 1 address to transport client, percolator perf is about
> 10k/s
>
> and when I added all 5 of vms to it, perf is about 15k/s
>
> so it increases by about 1.5times only even though I added 4 more vm
> addresses.
>
> Is it supposed to be like this?
>
> What I was thinking is, if it runs in for example round-robin fashion,
> there should be about 5 times performance gain.
>
> Could you comment anything on this?
>
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAL3_U412ecCDh0GONWJZy6S9zjY%2By8xvoVsPrt1849csfs-zUA%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/BAEDE7D4-320E-4123-A783-823FF1EC26ED%40pilato.fr
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL%2BHSrOczux-gdHPNDRJcPU%3D4CNEh2uKb13NE-%2BGPP3Kw_NjEg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: increase query performance by adding more machines, shouldn't it be linear to # of machines?

2014-07-01 Thread David Pilato 
Are you using same physical machine for all your VMs?

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 2 juil. 2014 à 07:09, Seungjin Lee  a écrit :

Hi all,

I'm testing percolator performance, 50k/s is required condition with 3~4k rules.

now I only have 1 simple rule, and 5 es vms with 1 shard and 4 replicas.

and using Java transport client like below

new TransportClient(settings)
.addTransportAddresses(transportAddressList.toArray(new 
InetSocketTransportAddress[transportAddressList.size()]));


when I added just 1 address to transport client, percolator perf is about 10k/s

and when I added all 5 of vms to it, perf is about 15k/s

so it increases by about 1.5times only even though I added 4 more vm addresses.

Is it supposed to be like this?

What I was thinking is, if it runs in for example round-robin fashion, there 
should be about 5 times performance gain.

Could you comment anything on this?



-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL3_U412ecCDh0GONWJZy6S9zjY%2By8xvoVsPrt1849csfs-zUA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/BAEDE7D4-320E-4123-A783-823FF1EC26ED%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

increase query performance by adding more machines, shouldn't it be linear to # of machines?

2014-07-01 Thread Seungjin Lee 
Hi all,

I'm testing percolator performance, 50k/s is required condition with 3~4k
rules.

now I only have 1 simple rule, and 5 es vms with 1 shard and 4 replicas.

and using Java transport client like below

new TransportClient(settings)
.addTransportAddresses(transportAddressList.toArray(new
InetSocketTransportAddress[transportAddressList.size()]));


when I added just 1 address to transport client, percolator perf is about
10k/s

and when I added all 5 of vms to it, perf is about 15k/s

so it increases by about 1.5times only even though I added 4 more vm
addresses.

Is it supposed to be like this?

What I was thinking is, if it runs in for example round-robin fashion,
there should be about 5 times performance gain.

Could you comment anything on this?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL3_U412ecCDh0GONWJZy6S9zjY%2By8xvoVsPrt1849csfs-zUA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.