Thinking more about this topic....

What about retrieving this information with getBookieInfo?

We can add faultDomainId and we can add other info useful both for
placement.
For instance we could add info about the ability to accept secure
communications, the current bookkeeper version.....

An important thing is that this information is available to the placement
policy in a future proof way




Enrico

Il lun 27 mag 2019, 09:19 Enrico Olivelli <eolive...@gmail.com> ha scritto:

>
>
> Il lun 27 mag 2019, 03:13 Venkateswara Rao Jujjuri <jujj...@gmail.com> ha
> scritto:
>
>> > I will also send soon a request for enhancements and a PR to have the
>> list of bookies
>>
>> List of bookies in the cluster? how the available and RO are listed
>
>
> If a bookie is down it is not present in any of the two lists.
>
> or list
>> of bookies in the metadata?
>>
>
> Actually we are listing /ledgers/cookies
> There is no API for this scan, you have to query ZK directly
>
> Enrico
>
>
>> On Sat, May 25, 2019 at 9:02 AM Enrico Olivelli <eolive...@gmail.com>
>> wrote:
>>
>> > JV,
>> > sorry for so late reply.
>> >
>> >
>> > Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <jujj...@gmail.com>
>> ha
>> > scritto:
>> >
>> > > > we should take care of designing a better API for the placement
>> policy
>> > > +1 Our placement policy is super complex and confusing.
>> > >
>> > > > We could also take into account the ability of adding labels/tags to
>> > > bookies.
>> > >
>> > > Can you add more context and color to your statement?
>> > > In the k8s world we do have a way to add tags. Can you please
>> elaborate
>> > > what you are thinking?
>> > >
>> >
>> > Most of my products have strong requirements about multi tenancy and
>> smart
>> > resource allocation.
>> > I have always very small clusters (3-5 bookies) so rack/region
>> awareness is
>> > not my primary focus (I have very few cases of multi region clusters).
>> > Currently if you want to drive the allocation of resources you have to
>> > write your own placement policy, put some tenant binding info on 'custom
>> > metadata' and use some out-of-band (in respect to BK)  information about
>> > bookies and which bookie can be used for the tenant who is requesting
>> the
>> > ledger placement.
>> >
>> > Having some sort of tags directly set on bookies zk directory will drop
>> the
>> > need for such out of bound information distribution.
>> >
>> > Some year ago we discussed about this topic but the discussion did not
>> go
>> > anywhere.
>> > IIRC the reason was that generic labels may be too generic and make the
>> > placement rules too complex.
>> >
>> > I see value on this fault domain id.
>> >
>> > I think we should have some information on zookeeper about which fault
>> > domains are defined.
>> > We can't discover new fault domains just by listing bookies metadata.
>> >
>> > I will also send soon a request for enhancements and a PR to have the
>> list
>> > of bookies (currently we only have APIs to discovery bookies that are up
>> > and running).
>> >  One of my colleagues is working on management tools and we found the
>> lack
>> > of this information.
>> > The script approach for dns to switch mapping also makes it impossible
>> to
>> > understand the topology of the cluster without testing every bookie
>> > address.
>> >
>> > Enrico
>> >
>> >
>> >
>> > >
>> > > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eolive...@gmail.com
>> >
>> > > wrote:
>> > >
>> > > > Il lun 20 mag 2019, 05:03 Sijie Guo <guosi...@gmail.com> ha
>> scritto:
>> > > >
>> > > > > +1 from me. `Cookie` was designed for keeping the informations
>> that
>> > is
>> > > > > associated with a bookie (e.g. disk layouts, bookie id and etc).
>> > > > >
>> > > > > I think it is making sense to have `FaultZoneId` stored as part of
>> > the
>> > > > > cookie.
>> > > > >
>> > > >
>> > > > I agree.
>> > > >
>> > > > But we should take care of designing a better API for the placement
>> > > policy.
>> > > > We are changing signatures quite often, adding parameters, changing
>> > > return
>> > > > type....
>> > > >
>> > > > We could also take into account the ability of adding labels/tags to
>> > > > bookies.
>> > > >
>> > > > Enrico
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > > - Sijie
>> > > > >
>> > > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
>> > > > > jujj...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > In the current code, bookie to faultDomain mapping is supplied
>> > > through
>> > > > > > different methods. Salesforce uses a script to read yaml file
>> which
>> > > > > > contains racks/machines mapping. But I am wondering why can't we
>> > put
>> > > > this
>> > > > > > info in the Cookie? Assuming that these machines can never move
>> > > across
>> > > > > > fault zones.
>> > > > > >
>> > > > > > Currently cookies contain  version, Host, JourlanDirs,
>> ledgerDirs,
>> > > and
>> > > > > > instanceId.
>> > > > > > If we add faultzoneId to it, it will be always available for
>> > everyone
>> > > > to
>> > > > > > look into.
>> > > > > > Is there any reason why it would be a bad idea?
>> > > > > >
>> > > > > > Thanks,
>> > > > > > --
>> > > > > > Jvrao
>> > > > > > ---
>> > > > > > First they ignore you, then they laugh at you, then they fight
>> you,
>> > > > then
>> > > > > > you win. - Mahatma Gandhi
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Jvrao
>> > > ---
>> > > First they ignore you, then they laugh at you, then they fight you,
>> then
>> > > you win. - Mahatma Gandhi
>> > >
>> >
>>
>>
>> --
>> Jvrao
>> ---
>> First they ignore you, then they laugh at you, then they fight you, then
>> you win. - Mahatma Gandhi
>>
>

Reply via email to