Il lun 27 mag 2019, 03:13 Venkateswara Rao Jujjuri <jujj...@gmail.com> ha scritto:
> > I will also send soon a request for enhancements and a PR to have the > list of bookies > > List of bookies in the cluster? how the available and RO are listed If a bookie is down it is not present in any of the two lists. or list > of bookies in the metadata? > Actually we are listing /ledgers/cookies There is no API for this scan, you have to query ZK directly Enrico > On Sat, May 25, 2019 at 9:02 AM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > JV, > > sorry for so late reply. > > > > > > Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <jujj...@gmail.com> > ha > > scritto: > > > > > > we should take care of designing a better API for the placement > policy > > > +1 Our placement policy is super complex and confusing. > > > > > > > We could also take into account the ability of adding labels/tags to > > > bookies. > > > > > > Can you add more context and color to your statement? > > > In the k8s world we do have a way to add tags. Can you please elaborate > > > what you are thinking? > > > > > > > Most of my products have strong requirements about multi tenancy and > smart > > resource allocation. > > I have always very small clusters (3-5 bookies) so rack/region awareness > is > > not my primary focus (I have very few cases of multi region clusters). > > Currently if you want to drive the allocation of resources you have to > > write your own placement policy, put some tenant binding info on 'custom > > metadata' and use some out-of-band (in respect to BK) information about > > bookies and which bookie can be used for the tenant who is requesting the > > ledger placement. > > > > Having some sort of tags directly set on bookies zk directory will drop > the > > need for such out of bound information distribution. > > > > Some year ago we discussed about this topic but the discussion did not go > > anywhere. > > IIRC the reason was that generic labels may be too generic and make the > > placement rules too complex. > > > > I see value on this fault domain id. > > > > I think we should have some information on zookeeper about which fault > > domains are defined. > > We can't discover new fault domains just by listing bookies metadata. > > > > I will also send soon a request for enhancements and a PR to have the > list > > of bookies (currently we only have APIs to discovery bookies that are up > > and running). > > One of my colleagues is working on management tools and we found the > lack > > of this information. > > The script approach for dns to switch mapping also makes it impossible to > > understand the topology of the cluster without testing every bookie > > address. > > > > Enrico > > > > > > > > > > > > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eolive...@gmail.com> > > > wrote: > > > > > > > Il lun 20 mag 2019, 05:03 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > > > > > +1 from me. `Cookie` was designed for keeping the informations that > > is > > > > > associated with a bookie (e.g. disk layouts, bookie id and etc). > > > > > > > > > > I think it is making sense to have `FaultZoneId` stored as part of > > the > > > > > cookie. > > > > > > > > > > > > > I agree. > > > > > > > > But we should take care of designing a better API for the placement > > > policy. > > > > We are changing signatures quite often, adding parameters, changing > > > return > > > > type.... > > > > > > > > We could also take into account the ability of adding labels/tags to > > > > bookies. > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > - Sijie > > > > > > > > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri < > > > > > jujj...@gmail.com> > > > > > wrote: > > > > > > > > > > > In the current code, bookie to faultDomain mapping is supplied > > > through > > > > > > different methods. Salesforce uses a script to read yaml file > which > > > > > > contains racks/machines mapping. But I am wondering why can't we > > put > > > > this > > > > > > info in the Cookie? Assuming that these machines can never move > > > across > > > > > > fault zones. > > > > > > > > > > > > Currently cookies contain version, Host, JourlanDirs, > ledgerDirs, > > > and > > > > > > instanceId. > > > > > > If we add faultzoneId to it, it will be always available for > > everyone > > > > to > > > > > > look into. > > > > > > Is there any reason why it would be a bad idea? > > > > > > > > > > > > Thanks, > > > > > > -- > > > > > > Jvrao > > > > > > --- > > > > > > First they ignore you, then they laugh at you, then they fight > you, > > > > then > > > > > > you win. - Mahatma Gandhi > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Jvrao > > > --- > > > First they ignore you, then they laugh at you, then they fight you, > then > > > you win. - Mahatma Gandhi > > > > > > > > -- > Jvrao > --- > First they ignore you, then they laugh at you, then they fight you, then > you win. - Mahatma Gandhi >
