Re: Does more shards in core improve performance?

I've tried to run different shards from different machine, and there is a slight improvement in the performance (about 3 mins faster for 1GB worth of data, from 22 mins to 19 mins). Is this a normal scenario? Both of my machine are running on Intel i7 core. Regards, Edwin On 21 September 2015

Re: Does more shards in core improve performance?

I leave it to the default settings for now, which should be balancing 50-50 across both shards. Regards, Edwin On 23 September 2015 at 22:49, Alessandro Benedetti < benedetti.ale...@gmail.com> wrote: > Using a second machines , you will dispose of fresh memory, disk and CPUs. > So assuming you

Re: Does more shards in core improve performance?

Using a second machines , you will dispose of fresh memory, disk and CPUs. So assuming you succeeded in saturating the first machine indexing power, of course it is normal that you improve your indexing time giving an additional node serving the indexing process. Are you balancing 50:50 or with

Re: Does more shards in core improve performance?

On Mon, 2015-09-21 at 10:13 +0800, Zheng Lin Edwin Yeo wrote: > I didn't find any increase in indexing throughput by adding shards in the > same machine. > > However, I've managed to feed the index to Solr from more than one thread > at a time. It can take up to 3 threads without affecting the

Re: Does more shards in core improve performance?

I'm not sure if that is because currently my machine is a normal PC and not a server, but my CPU specification for each of the core is Intel(R) Core(TM) i7-4910MQ CPU @ 2.90GHz. It should probably be better when the real server which has a much better specification comes, and I should be able to

Re: Does more shards in core improve performance?

I didn't find any increase in indexing throughput by adding shards in the same machine. However, I've managed to feed the index to Solr from more than one thread at a time. It can take up to 3 threads without affecting the indexing speed. Anything more than that, the CPU will hit 100%, and the

Re: Does more shards in core improve performance?

If cpu is just 50% and adding a shard does increase indexing throughput then check for disk bottleneck. On Sep 17, 2015 18:19, "Zheng Lin Edwin Yeo" wrote: > Thank you everyone for your reply. > > > How many CPUs on that machine? How many other requests using the server? >

Does more shards in core improve performance?

Hi, Would like to check, does creating more shards for the core improve the overall performance? I'm using Solr 5.3.0. I tried the indexing for a core with 1 shard and another core with 2 shards, but both are taking the same amount of time to do the indexing. Currently, both my shards are in

Re: Does more shards in core improve performance?

On Thu, 2015-09-17 at 12:04 +0530, Shalin Shekhar Mangar wrote: > Yes, of course, the only reason to have more shards is so that they > can reside on different machines (or use different disks, assuming you > have enough CPU/memory etc) so that you can scale your indexing > throughput. For

Re: Does more shards in core improve performance?

Yes, of course, the only reason to have more shards is so that they can reside on different machines (or use different disks, assuming you have enough CPU/memory etc) so that you can scale your indexing throughput. Move one of them to a different machine and measure the performance. On Thu, Sep

Re: Does more shards in core improve performance?

On Thu, 2015-09-17 at 16:58 +0800, Zheng Lin Edwin Yeo wrote: > I was trying with 2 shards and 4 shards but all on the same machine, > and they have the same performance (no improvement in performance) as > the one with 1 shard. My machine has a 32GB RAM. As you are testing indexing speed,

Re: Does more shards in core improve performance?

How many CPUs on that machine? How many other requests using the server? On Thu, Sep 17, 2015, at 09:58 AM, Zheng Lin Edwin Yeo wrote: > Thanks for the information. > > I was trying with 2 shards and 4 shards but all on the same machine, and > they have the same performance (no improvement in

Re: Does more shards in core improve performance?

Thanks for the information. I was trying with 2 shards and 4 shards but all on the same machine, and they have the same performance (no improvement in performance) as the one with 1 shard. My machine has a 32GB RAM. Probably I should try one of the shard in different machine and see how it goes?

Re: Does more shards in core improve performance?

Thank you everyone for your reply. > How many CPUs on that machine? How many other requests using the server? A) There's 8 CPU on the machine, and there is no other requests that's using the server. Only the indexing script is running. > A simple metric is to look at CPU usage on the machine: