Thanks for your help. After adding the two config items into bk_server.conf : journalFlushWhenQueueEmpty=true journalMaxGroupWaitMSec=0 The write speed rise up to 800-900 entries per second in one ledger, length of each entry is 50 bytes. But it's still lower than I expected（The sync write speed is 2941 entries/s based onbookkeepr-4.2.2）. Could you describe your test result? And Is there any way to improve write performance ? Thank you.
At 2014-10-17 20:13:07, "Ivan Kelly" <[email protected]> wrote: >Ah, I know what this is. Twitter added some changes that makes write >throughput much higher when there's a high write rate. In this case >the latency for a single writes goes go up though (as it's doing more >buffering on the journal). Try setting: > >journalFlushWhenQueueEmpty=true >journalMaxGroupWaitMSec=0 > >in bk_server.conf > > >Also, the journal and ledger directories are on the same disk. This >will have a detrimental affect once the write buffers start flushing. >We always recommend that you put the journal on its own independent >disk. > >The reads are fast because it doesn't need to hit disk, as there's not >much data that has been written. > >-Ivan > >On 17 October 2014 13:32, [email protected] <[email protected]> wrote: >> The write speed of bookkeper that version less than 4.30 is bigger than 1200 >> entries per second when i use the same test code >> >> ________________________________ >> [email protected] >> >> >> From: Jiannan Wang >> Date: 2014-10-17 18:31 >> To: [email protected] >> CC: [email protected] >> Subject: Re: hi, why write speed of bookkeeper-4.3.0 is very slow? >> Could you share more information: >> - What's the number of write_num? >> - What's the read throughput under your test? >> - Does the 5 bookie servers run in same machine? >> - Could you check whether StringUtil.div works well? >> - Could you also try async write API and tell us the output? >> >> Regards, >> Jiannan >> >> From: "[email protected]" <[email protected]> >> Reply-To: "[email protected]" >> <[email protected]> >> Date: Friday, October 17, 2014 at 3:38 PM >> To: bookkeeper-user <[email protected]> >> Subject: hi, why write speed of bookkeeper-4.3.0 is very slow? >> >> hi, >> I have tried bookkeeper-4.3.0 today. But its write speed is 4 entries >> per sec in my environment. >> There are 5 bookkeeper-server-4.3.0 in my env. >> Is there anything wrong? Test codes as follows: >> public void syncWriteLedger(long write_num) throws InterruptedException, >> BKException { >> byte[] pwd = "foobar".getBytes(); >> LedgerHandle lh = client.createLedger(3, 2, >> DigestType.MAC, pwd); >> >> long ledgerId = lh.getId(); >> System.out.println("ledgerId=" + ledgerId); >> System.out >> .println("data_len=" >> + ("Hello World!afdafdafdfxtyb457u8n9,7.,pbesgcerfqx" + write_num) >> .getBytes().length); >> byte[] data = ("Hello World!afdafdafdfxtyb457u8n9,7.,pbesgcerfqx" + >> write_num).getBytes(); >> long st = System.currentTimeMillis(); >> for (int i = 0; i < write_num; i++) { >> lh.addEntry(data); >> } >> long ut = System.currentTimeMillis() - st; >> lh.close(); >> double uts = StringUtil.div(ut, 1000, 3); >> double wspeed = StringUtil.div(write_num, uts, 2); >> System.out.println("ut=" + ut + " ms, write_num=" + write_num >> + ", write speed=" + wspeed + "/s"); >> >> >> LedgerHandle lh2 = client.openLedger(ledgerId, BookKeeper.DigestType.MAC, >> pwd); >> long lastEntry = lh2.getLastAddConfirmed(); >> st = System.currentTimeMillis(); >> Enumeration<LedgerEntry> entries = lh2.readEntries(0, lastEntry); >> while (entries.hasMoreElements()) { >> byte[] bytes = entries.nextElement().getEntry(); >> System.out.println(new String(bytes)); >> } >> ut = System.currentTimeMillis() - st; >> lh2.close(); >> uts = StringUtil.div(ut, 1000, 3); >> double rspeed = StringUtil.div(write_num, uts, 2); >> System.out.println("ut=" + ut + " ms, read_num=" + write_num >> + ", read speed=" + rspeed + "/s"); >> >> } >> ________________________________ >> [email protected]
