Does the number and type of disks affect the ideal q value?
On spinning disks, I might imagine a performance gain by putting each
shard on a separate spindle, in which case the ideal performance might
be something like q = # of cores = # of spindles. (Although I don't
believe it's trivial to separate shards in such a way.)
This may not need to figure into the discussion, since I believe the
goal is to find the ideal for low-end hardware, not every possible tweak
for high performance.
Jonathan
On 7/11/19 6:07 PM, Joan Touzet wrote:
Hi Tabeth,
This is a good start. I'll let others comment on the specific advice.
One point raised in this thread is ermouth's request to have this value
automatically set by the installer. Detecting the number of cores in a
cross-platform way is pretty challenging, especially when you consider
things like hyperthreading (which aren't full "cores") on physical machines.
I agree a table would be a good thing for the docs, but we really need a
single number for q (plus other config settings) that will be good
defaults for low-performing HW.
-Joan
On 2019-07-11 11:43, Tabeth Nkangoh wrote:
Hello all,
After reading the documentation
<https://docs.couchdb.org/en/2.2.0/cluster/sharding.html> and this thread I
propose the following formula to document officially for determining the Q and N
values.
To start, as currently documented, N should be set to the number of nodes. This
much is straightforward and is documented more or less correctly.
For Q, it should equal the amount of cores at a minimum. So, for the cheapest
Digital Ocean droplet, which 1GB of RAM and 1vCPU, Q should be set to 1 by the
user. In the case that each core is sufficiently beefy, e.g. Intel Xeon or
turbo clocked AMD EPYC 7000 2 should be added. 2 is an arbitrary value but
reflects the reality that from what I can tell a dramatically higher clock
speed benefits CouchDB performance AFAIK (I'm not sure if this benefit is a
constant or a multiplier, but I doubt it is a multiplier simply due to the
diminishing returns seen in increased clock speed).
To summarize the following table can be used (for example):
CPU Cores CPU Clock Speed Q Value
1 Low 1
2 Low 2
2 High 4
3 High 5
4 Low 4
Does anyone have any qualms with this as a general guide to give users in
determining the Q value they should set for their CouchDB instance? If not I
can submit a PR soon on this (have to figure out how to create a new reference
for one that's currently open first).
Tabeth
________________________________
From: Joan Touzet <woh...@apache.org>
Sent: Thursday, July 11, 2019 11:17 AM
To: dev@couchdb.apache.org
Subject: Re: CouchDB and future
On 2019-07-11 8:05, ermouth wrote:
to help with the documentation step
Probably. Please give me a hint what you need.
What default settings right now are wrong for a system with low RAM, low
CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
-Joan
