> I will also send soon a request for enhancements and a PR to have the list of bookies
List of bookies in the cluster? how the available and RO are listed or list of bookies in the metadata? On Sat, May 25, 2019 at 9:02 AM Enrico Olivelli <eolive...@gmail.com> wrote: > JV, > sorry for so late reply. > > > Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <jujj...@gmail.com> ha > scritto: > > > > we should take care of designing a better API for the placement policy > > +1 Our placement policy is super complex and confusing. > > > > > We could also take into account the ability of adding labels/tags to > > bookies. > > > > Can you add more context and color to your statement? > > In the k8s world we do have a way to add tags. Can you please elaborate > > what you are thinking? > > > > Most of my products have strong requirements about multi tenancy and smart > resource allocation. > I have always very small clusters (3-5 bookies) so rack/region awareness is > not my primary focus (I have very few cases of multi region clusters). > Currently if you want to drive the allocation of resources you have to > write your own placement policy, put some tenant binding info on 'custom > metadata' and use some out-of-band (in respect to BK) information about > bookies and which bookie can be used for the tenant who is requesting the > ledger placement. > > Having some sort of tags directly set on bookies zk directory will drop the > need for such out of bound information distribution. > > Some year ago we discussed about this topic but the discussion did not go > anywhere. > IIRC the reason was that generic labels may be too generic and make the > placement rules too complex. > > I see value on this fault domain id. > > I think we should have some information on zookeeper about which fault > domains are defined. > We can't discover new fault domains just by listing bookies metadata. > > I will also send soon a request for enhancements and a PR to have the list > of bookies (currently we only have APIs to discovery bookies that are up > and running). > One of my colleagues is working on management tools and we found the lack > of this information. > The script approach for dns to switch mapping also makes it impossible to > understand the topology of the cluster without testing every bookie > address. > > Enrico > > > > > > > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eolive...@gmail.com> > > wrote: > > > > > Il lun 20 mag 2019, 05:03 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > > > +1 from me. `Cookie` was designed for keeping the informations that > is > > > > associated with a bookie (e.g. disk layouts, bookie id and etc). > > > > > > > > I think it is making sense to have `FaultZoneId` stored as part of > the > > > > cookie. > > > > > > > > > > I agree. > > > > > > But we should take care of designing a better API for the placement > > policy. > > > We are changing signatures quite often, adding parameters, changing > > return > > > type.... > > > > > > We could also take into account the ability of adding labels/tags to > > > bookies. > > > > > > Enrico > > > > > > > > > > > > > > > > - Sijie > > > > > > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri < > > > > jujj...@gmail.com> > > > > wrote: > > > > > > > > > In the current code, bookie to faultDomain mapping is supplied > > through > > > > > different methods. Salesforce uses a script to read yaml file which > > > > > contains racks/machines mapping. But I am wondering why can't we > put > > > this > > > > > info in the Cookie? Assuming that these machines can never move > > across > > > > > fault zones. > > > > > > > > > > Currently cookies contain version, Host, JourlanDirs, ledgerDirs, > > and > > > > > instanceId. > > > > > If we add faultzoneId to it, it will be always available for > everyone > > > to > > > > > look into. > > > > > Is there any reason why it would be a bad idea? > > > > > > > > > > Thanks, > > > > > -- > > > > > Jvrao > > > > > --- > > > > > First they ignore you, then they laugh at you, then they fight you, > > > then > > > > > you win. - Mahatma Gandhi > > > > > > > > > > > > > > > > > > -- > > Jvrao > > --- > > First they ignore you, then they laugh at you, then they fight you, then > > you win. - Mahatma Gandhi > > > -- Jvrao --- First they ignore you, then they laugh at you, then they fight you, then you win. - Mahatma Gandhi
