Hi Walter, For your suggestion to try out the time gunzip < solr-6.4.1.tgz > /dev/null, does it works on Windows system? I tried on Windows, and it give me the error "The syntax of the command is incorrect".
In my current setup, if running on one trip, I can index about 16000 lines in a CSV file per minute on my laptop, but I can only index less than 1600 lines per minute on the server, which is more than 10 times slower. Regards, Edwin On 14 February 2017 at 13:45, Zheng Lin Edwin Yeo <edwinye...@gmail.com> wrote: > Thanks for the info. > > Yes, I'm running Solr 6.4.1 on both hosts. > > Regards, > Edwin > > > On 14 February 2017 at 13:21, Walter Underwood <wun...@wunderwood.org> > wrote: > >> It is worth doing a basic CPU speed test. Once you have enough RAM, >> indexing is mostly CPU-bound. >> >> Try something like this. Run it once to get the tgz file cached in OS >> file buffers, then once to time it. >> >> time gunzip < solr-6.4.1.tgz > /dev/null >> >> I get 1.3 seconds on an Amazon c4.8xlarge and 0.8 seconds on my MacBook. >> A bigger file would be a better test, but that is the general idea. >> >> Also, are you running 6.4.1 on both hosts? The new metrics code caused >> some slowdowns from 6.3.0 to 6.4.0. >> >> On the other hand, I’m indexing about a million documents per minute into >> a 16 node cluster (4 shards, 4-way replication factor) built with the >> c4.8xlarge instances. I’m running 64 indexing threads and 1000 doc batches. >> It might go a bit faster after we switch the cloud driver in SolrJ. >> >> wunder >> Walter Underwood >> wun...@wunderwood.org >> http://observer.wunderwood.org/ (my blog) >> >> >> > On Feb 13, 2017, at 9:10 PM, Zheng Lin Edwin Yeo <edwinye...@gmail.com> >> wrote: >> > >> > No, currently the server is slower, and my laptop is faster. >> > >> > But shouldn't the server be faster, since it has a much better >> > specification, like more RAM, better processor and SSD drive. >> > >> > Regards, >> > Edwin >> > >> > >> > On 14 February 2017 at 12:26, Walter Underwood <wun...@wunderwood.org> >> > wrote: >> > >> >> Are you sure the server is faster? My MacBook Pro is a lot faster than >> >> many of our Amazon EC2 servers. >> >> >> >> wunder >> >> Walter Underwood >> >> wun...@wunderwood.org >> >> http://observer.wunderwood.org/ (my blog) >> >> >> >> >> >>> On Feb 13, 2017, at 8:12 PM, Zheng Lin Edwin Yeo < >> edwinye...@gmail.com> >> >> wrote: >> >>> >> >>> Hi, >> >>> >> >>> I'm facing the issue of the indexing speed is slower is slower on a >> >> server >> >>> with a much better specification with Solr running on SSD, as compared >> >> to a >> >>> laptop with a normal hard disk. >> >>> >> >>> Both the system has the exact same configurations. The configurations >> are >> >>> first setup on the laptop, before being replicate to the server. >> >>> >> >>> The setup is Solr 6.4.1, of 1 shard with 2 replica, using external >> >>> ZooKeeper 3.4.8. The only difference is that in my laptop, both the >> >> shards >> >>> and ZooKeeper are on the same hard disk, while a the server, the >> >> ZooKeeper >> >>> is running on it's own hard disk, and each of the shards are also >> running >> >>> on a separate hard disk. From what I know, this configuration should >> >> result >> >>> in improving the performance, instead of making it worse? >> >>> >> >>> What could be the other reasons that this could happen? >> >>> >> >>> I'm running on Solr 6.4.1 >> >>> >> >>> Regards, >> >>> Edwin >> >> >> >> >> >> >
