Re: [jira] [Updated] (BOOKKEEPER-1040) Use separate log for compaction

2017-04-19 Thread Yiming Zang
Sounds good. On Wed, Apr 19, 2017 at 10:54 PM, Venkateswara Rao Jujjuri < jujj...@gmail.com> wrote: > We have addressed this problem in slightly different way. I would like to > add it to the conversation tomorrow. > > On Wed, Apr 19, 2017 at 10:46 PM, Yiming Za

Re: why write to journal is happening after write to ledgerStorage?

2017-05-01 Thread Yiming Zang
Hi Andrey, That's a good point, and you're actually correct that if write to memTable got throttled somehow, the addEntry request latency will be affected a lot. This actually happens a few times in production cluster. Normally, the idea of using Journal is to write data to the write-ahead log and

Re: why write to journal is happening after write to ledgerStorage?

2017-05-01 Thread Yiming Zang
; correctness. > > It > > > > might end up a case - the lac is already advanced but the entry is > not > > > > found - it can happen in following sequence. > > > > > > > > - Client issue write entry N (lac = N-1) > > > > - Boo

Re: Bookie Http Endpoint

2017-07-18 Thread Yiming Zang
Hi all, We also have a working implementation in Twitter, we use Http Endpoint mostly for getting quorum loss information, underreplicated ledgers, manage bookie status etc. The HTTP server in Twitter is implemented in such a way that it has a generic skeleton which can easily embed with differen

Re: BP-11: New BookKeeper Website

2017-07-25 Thread Yiming Zang
This looks awesome! On Tue, Jul 25, 2017 at 12:09 PM, Sijie Guo wrote: > Hi all, > > Luc Perkins is helping on a new bookkeeper documentation site for BP-11. > Although it is still WIP, we'd like to first share the beta version with > the community to get some feedbacks. > > The new website: htt

RackawareEnsemblePlacementPolicy only ensure at least 2 racks?

2017-08-02 Thread Yiming Zang
I was looking into RackawareEnsemblePlacementPolicyImpl.java, and it turns out to me that the current implementation of RackawareEnsemblePlacementPolicy only enforces at least 2 different racks, it doesn't care about ensemble size, write quorum size or ack quorum size. For example, imagine we have

Re: RackawareEnsemblePlacementPolicy only ensure at least 2 racks?

2017-08-02 Thread Yiming Zang
rom same rack as the client. Or am I missing anything? On Wed, Aug 2, 2017 at 7:29 PM, Sijie Guo wrote: > On Aug 2, 2017 6:54 PM, "Yiming Zang" wrote: > > I was looking into RackawareEnsemblePlacementPolicyImpl.java, and it turns > out to me that the current implementation

Re: RackawareEnsemblePlacementPolicy only ensure at least 2 racks?

2017-08-03 Thread Yiming Zang
). Because in this case, num of racks will become 2, this will still potentially break the rack-diversity. So if we want to add such an option in the future, it should be min(num of racks, ensemble size)? On Thu, Aug 3, 2017 at 12:53 AM, Sijie Guo wrote: > On Wed, Aug 2, 2017 at 11:34 PM, Yimi

Re: [DISCUSS] BP-17: define BookKeeper public http endpoints

2017-09-12 Thread Yiming Zang
Sure, I think the current HTTP endpoints in Twitter are only designed for Twitter specific, such as check quorum loss, check rack/region diversity. So the endpoints convention in Twitter are not the same as in the proposal. I think it would be great to have an agreement on the API naming design, so

Re: [DISCUSS] BP-17: define BookKeeper public http endpoints

2017-09-19 Thread Yiming Zang
t; >> > >> >> > > >> >> > IMHO The 'create ledger' API is not useful, due to the design of > BK > >> you > >> >> > have to create a ledger and then write immediately to it, I think > >> that > >>

Re: [ANNOUNCE] New BookKeeper Committer: Yiming Zang

2017-12-18 Thread Yiming Zang
Thank you! I'm very happy to join the community! On Mon, Dec 18, 2017 at 12:18 PM, Enrico Olivelli wrote: > Congrats Yiming! > Enrico > > Il lun 18 dic 2017, 19:56 Sijie Guo ha scritto: > >> The Apache BookKeeper PMC recently extended committer karma to Yiming Z

Re: [VOTE] Release 4.6.1, release candidate #0

2018-01-28 Thread Yiming Zang
+1 - MD5 SHA signature looks good - Source code looks good - Binary looks good On Sun, Jan 28, 2018 at 6:59 AM, David Rusek wrote: > I'll take a look first thing tomorrow morning. > > On Sun, Jan 28, 2018 at 3:23 AM Enrico Olivelli > wrote: > > > Thank you all, > > We need another PMC +1 > > >

Re: BP-27: New BookKeeper CLI

2018-01-31 Thread Yiming Zang
+1 to improve the the tool. On Wed, Jan 31, 2018 at 5:52 PM, Jia Zhai wrote: > +1, It is great to break down this huge file, and make it more > structure-able. > The prototype PR seems more clear, and jcommander is nice. > > On Thu, Feb 1, 2018 at 7:22 AM, Sijie Guo wrote: > > > Hi all, > > > >

Re: Next Community Meeting?

2016-11-15 Thread Yiming Zang
SGTM +1 On Tue, Nov 15, 2016 at 10:34 AM, Venkateswara Rao Jujjuri < jujj...@gmail.com> wrote: > Sounds good. > > On Tue, Nov 15, 2016 at 1:41 AM Enrico Olivelli > wrote: > > > Ok for me. > > > > Enrico > > > > Il mar 15 nov 2016, 10:39 Sijie Guo ha scritto: > > > > > We will continue discussin

Re: Next Community Meeting?

2016-12-07 Thread Yiming Zang
t; > > > > > >>>>>> I see that this new release is full of new great features, > is > > it > > > > > > > >>> worth > > > > > > > >>>>>> to change to 5.0.0 ? > > > > > > > >>>>>> > > &g

Questions about compaction change

2017-03-23 Thread Yiming Zang
Hi JV, I noticed that you're doing a bunch of work to improve BK compaction in https://issues.apache.org/jira/browse/BOOKKEEPER-944 and https://issues.apache.org/jira/browse/BOOKKEEPER-949. Earlier, I mentioned that we were also doing some work to make compaction use a separate log file and add i

Re: Next community meeting and notes

2017-04-06 Thread Yiming Zang
Hi Bobby, here's the link to the meeting https://goo.gl/iyRA6G On Thu, Apr 6, 2017 at 8:04 AM, Bobby Evans wrote: > What is the URL. I can never find it > > > - Bobby > > On Thursday, April 6, 2017, 5:02:53 AM CDT, Enrico Olivelli < > eolive...@gmail.com> wrote:Anyone on the Hangout ? > > 2017-

[jira] [Updated] (BOOKKEEPER-1040) Use separate log for compaction

2017-04-19 Thread Yiming Zang (JIRA)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiming Zang updated BOOKKEEPER-1040: Description: 1. Bookkeeper is not able to reclaim disk space when it's full I

[jira] [Created] (BOOKKEEPER-1040) Use separate log for compaction

2017-04-19 Thread Yiming Zang (JIRA)
Yiming Zang created BOOKKEEPER-1040: --- Summary: Use separate log for compaction Key: BOOKKEEPER-1040 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1040 Project: Bookkeeper Issue

[jira] [Updated] (BOOKKEEPER-1040) Use separate log for compaction

2017-04-19 Thread Yiming Zang (JIRA)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiming Zang updated BOOKKEEPER-1040: Description: Bookkeeper is not able to reclaim disk space when it's full If all

[jira] [Created] (BOOKKEEPER-1100) Add Http Endpoint for Bookkeeper

2017-06-24 Thread Yiming Zang (JIRA)
Yiming Zang created BOOKKEEPER-1100: --- Summary: Add Http Endpoint for Bookkeeper Key: BOOKKEEPER-1100 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1100 Project: Bookkeeper Issue

[jira] [Created] (BOOKKEEPER-952) Fix RegionAwarePlacementPolicy

2016-10-13 Thread Yiming Zang (JIRA)
Yiming Zang created BOOKKEEPER-952: -- Summary: Fix RegionAwarePlacementPolicy Key: BOOKKEEPER-952 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-952 Project: Bookkeeper Issue Type

[jira] [Created] (BOOKKEEPER-960) Re-replicator: bookie checking should handle flapping bookie registration

2016-10-30 Thread Yiming Zang (JIRA)
Yiming Zang created BOOKKEEPER-960: -- Summary: Re-replicator: bookie checking should handle flapping bookie registration Key: BOOKKEEPER-960 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-960

[jira] [Commented] (BOOKKEEPER-979) Bookie recovery should skip replicating empty ledgers.

2016-11-28 Thread Yiming Zang (JIRA)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703353#comment-15703353 ] Yiming Zang commented on BOOKKEEPER-979: I think I have a similar fix in

[jira] [Created] (BOOKKEEPER-1014) Make BK Compaction use a separate log file

2017-03-23 Thread Yiming Zang (JIRA)
Yiming Zang created BOOKKEEPER-1014: --- Summary: Make BK Compaction use a separate log file Key: BOOKKEEPER-1014 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1014 Project: Bookkeeper

[jira] [Updated] (BOOKKEEPER-1014) Make BK Compaction use a separate log file

2017-03-23 Thread Yiming Zang (JIRA)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiming Zang updated BOOKKEEPER-1014: Description: There're a few issues brought by the current compaction: 1. BK

[jira] [Updated] (BOOKKEEPER-1014) User a separate log file for compaction

2017-04-03 Thread Yiming Zang (JIRA)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiming Zang updated BOOKKEEPER-1014: Summary: User a separate log file for compaction (was: Make BK Compaction use a