BK currently isn't wired to stream bytes to a ledger, so writing synchronously
large entries as you're doing is likely not to get the best its performance. A
couple of things you could try to get higher performance are to write
asynchronously and to have multiple clients writing.
-Flavio
On Wednesday, June 10, 2015 12:08 PM, Maciej Smoleński <[email protected]>
wrote:
Hello,
I'm testing BK performance when appending 100K entries synchronously from 1
thread (using one ledger).
The performance I get is 250 entries/s.
What performance should I expect ?
My setup:
Ledger:
Ensemble size: 3
Quorum size: 2
1 client machine and 3 server machines.
Network:
Each machine with bonding: 4 x 1000Mbps on each machine
manually tested between client and server: 400MB/s
Disk:
I tested two configurations:
dedicated disks with ext3 (different for zookeeper, journal, data, index, log)
dedicated ramfs partitions (different for zookeeper, journal, data, index, log)
In both configurations the performance is the same: 250 entries / s (25MB / s).
I confirmed this with measured network bandwidth:
- on client 50 MB/s
- on server 17 MB/s
I run java with profiler enabled on BK client and BK server but didn't find
anything unexpected (but I don't know bookkeeper internals).
I tested it with two BookKeeper versions:
- 4.3.0
- 4.2.2
The result were the same with both BookKeeper versions.
What should be changed/checked to get better performance ?
Kind regards,
Maciej
