Ah, I know what this is. Twitter added some changes that makes write throughput much higher when there's a high write rate. In this case the latency for a single writes goes go up though (as it's doing more buffering on the journal). Try setting:
journalFlushWhenQueueEmpty=true journalMaxGroupWaitMSec=0 in bk_server.conf Also, the journal and ledger directories are on the same disk. This will have a detrimental affect once the write buffers start flushing. We always recommend that you put the journal on its own independent disk. The reads are fast because it doesn't need to hit disk, as there's not much data that has been written. -Ivan On 17 October 2014 13:32, [email protected] <[email protected]> wrote: > The write speed of bookkeper that version less than 4.30 is bigger than 1200 > entries per second when i use the same test code > > ________________________________ > [email protected] > > > From: Jiannan Wang > Date: 2014-10-17 18:31 > To: [email protected] > CC: [email protected] > Subject: Re: hi, why write speed of bookkeeper-4.3.0 is very slow? > Could you share more information: > - What's the number of write_num? > - What's the read throughput under your test? > - Does the 5 bookie servers run in same machine? > - Could you check whether StringUtil.div works well? > - Could you also try async write API and tell us the output? > > Regards, > Jiannan > > From: "[email protected]" <[email protected]> > Reply-To: "[email protected]" > <[email protected]> > Date: Friday, October 17, 2014 at 3:38 PM > To: bookkeeper-user <[email protected]> > Subject: hi, why write speed of bookkeeper-4.3.0 is very slow? > > hi, > I have tried bookkeeper-4.3.0 today. But its write speed is 4 entries > per sec in my environment. > There are 5 bookkeeper-server-4.3.0 in my env. > Is there anything wrong? Test codes as follows: > public void syncWriteLedger(long write_num) throws InterruptedException, > BKException { > byte[] pwd = "foobar".getBytes(); > LedgerHandle lh = client.createLedger(3, 2, > DigestType.MAC, pwd); > > long ledgerId = lh.getId(); > System.out.println("ledgerId=" + ledgerId); > System.out > .println("data_len=" > + ("Hello World!afdafdafdfxtyb457u8n9,7.,pbesgcerfqx" + write_num) > .getBytes().length); > byte[] data = ("Hello World!afdafdafdfxtyb457u8n9,7.,pbesgcerfqx" + > write_num).getBytes(); > long st = System.currentTimeMillis(); > for (int i = 0; i < write_num; i++) { > lh.addEntry(data); > } > long ut = System.currentTimeMillis() - st; > lh.close(); > double uts = StringUtil.div(ut, 1000, 3); > double wspeed = StringUtil.div(write_num, uts, 2); > System.out.println("ut=" + ut + " ms, write_num=" + write_num > + ", write speed=" + wspeed + "/s"); > > > LedgerHandle lh2 = client.openLedger(ledgerId, BookKeeper.DigestType.MAC, > pwd); > long lastEntry = lh2.getLastAddConfirmed(); > st = System.currentTimeMillis(); > Enumeration<LedgerEntry> entries = lh2.readEntries(0, lastEntry); > while (entries.hasMoreElements()) { > byte[] bytes = entries.nextElement().getEntry(); > System.out.println(new String(bytes)); > } > ut = System.currentTimeMillis() - st; > lh2.close(); > uts = StringUtil.div(ut, 1000, 3); > double rspeed = StringUtil.div(write_num, uts, 2); > System.out.println("ut=" + ut + " ms, read_num=" + write_num > + ", read speed=" + rspeed + "/s"); > > } > ________________________________ > [email protected]
