Re: PutAll Behavior Single vs Multiple Servers

[Victor]

Denis, i get that about when to scale. And with my test, these are fairly
large boxes with lots of head room. Hardly doing 10% or less CPU wise.

But why do i see a performance difference when i put data into 1 node vs 3
node with PRIMARY_SYNC turned on.

Shouldn't the behavior and workflow be the same?

1. Client sends the put request.
2. Server applies the change to the primary node. Does not wait for backup
to be updated.
3. Returns the response back to the client.

Backup change happens async.

ideally with PRIMARY_SYNC, this workflow should be the same irrespective of
the number of nodes and/or backups. Since the server does not wait for
backups to be updated.

Let me know if there is more to it that i am missing.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: PutAll Behavior Single vs Multiple Servers

[Denis Magda]

Each node process uses many threads, thread pools, and other resources to
process the app's requests as well as for internal needs. Thus once a
single node cluster reaches its maximum potential we need to scale. That's
for the 1 vs 3 nodes setup question.

-
Denis


On Thu, Dec 12, 2019 at 5:54 AM Victor  wrote:

> I am aware of those nuances around distributed systems.
>
> What i am trying to understand is, with Sync mode as PRIMARY_SYNC, where
> the
> response does not wait for updates to backups, so long as primary node is
> updated. With this setting why should 1 node vs 3 node matter?
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: PutAll Behavior Single vs Multiple Servers

[Victor]

I am aware of those nuances around distributed systems.

What i am trying to understand is, with Sync mode as PRIMARY_SYNC, where the
response does not wait for updates to backups, so long as primary node is
updated. With this setting why should 1 node vs 3 node matter?




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: PutAll Behavior Single vs Multiple Servers

[Stanislav Lukyanov]

This is a very common pitfall with distributed systems - comparing 1 node
vs 3 nodes. In short, this is not correct to compare them.

When you write to one node each write does the following:
1) client sends the request to the server
2) server updates data
3) server sends the response to the client

When you write to three nodes with backups=1 each write needs to
1) client sends the request to the primary server
2) primary server updates data
3) primary server sends the request to the backup
4) backup updates data
5) backup sends the response to the primary
6) primary sends the response to the client

So, when you have more than one node you can actually have backups, and
having backups you need to do much more for each write.

If you want to check scalability, compare 3 nodes vs 4 vs 5, but 1 vs 3 is
not a fair comparison.

Also, you shouldn't start more than one node per host. They will just
compete for the same resources.

Stan

On Mon, Dec 9, 2019 at 9:57 PM Victor  wrote:

> Any pointers to understand this behavior?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: PutAll Behavior Single vs Multiple Servers

[Victor]

Any pointers to understand this behavior?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

PutAll Behavior Single vs Multiple Servers

[Victor]

Hi Folks,

I am doing a putAll test with a simple Employee Pojo, stored as binary. The
cache is configured with,

Atomicity Mode = Transactional
Write Sync Mode = Full Sync
Backup Count - 1

Deployment config is, 2 large linux boxes,
Box 1 - 3 server nodes
Box 2 - 1 client node

500k load with batch size of 10 (is configurable, can be changed, but for
now 10 is what i am running the test with). And 10 threads processing the
load.

Test 1 (Default config - as above)
Throughput - ~4200/s

Test 2 (Atomicity = Atomic)
Throughput = ~4600/s

Test 3 (Atomicity = Atomic & write sync mode = Primary Sync)
Throughput = ~4600/s

Test 4 (Atomicity = Atomic & write sync mode = Async)
Throughput = ~71000/s

Reduced the server nodes to a single node.

Test 1 (Default config - as above)
Throughput - ~12000/s

Test 2 (Atomicity = Atomic)
Throughput = ~13500/s

Test 3 (Atomicity = Atomic & write sync mode = Primary Sync)
Throughput = ~13500/s

I am trying to understand this significant difference in throughput, with 1
server vs 3 server nodes, especially for the case, 3 servers
(Atomicity=Atomic, Write Sync Mode=Primary_Sync) vs 1 server
(Atomicity=Atomic, Write Sync Mode=Primary_Sync).

Technically, i feel they are the same. Since in both cases it updates the
primary owner node and returns.

Let me know if there is something obvious i am missing.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/