Dustin, Enrico,
If you guys have time, please take a look at Yiming's new pull request :
https://github.com/apache/bookkeeper/pull/278
- Sijie
On Wed, Jul 19, 2017 at 9:46 PM, Enrico Olivelli <eolive...@gmail.com>
wrote:
> Il gio 20 lug 2017, 00:37 Sijie Guo <guosi...@gmail.com> ha scritto:
>
> > On Wed, Jul 19, 2017 at 10:02 AM, Dustin Castor <
> > dustincas...@yahoo.com.invalid> wrote:
> >
> > > I do agree that we need a contractual way of managing the server
> > lifecycle
> > > via an interface -- identical to how we do it with stat providers.
> >
> >
> >
> >
> > > Personally I don't think we can really have contracts for each and
> every
> > > method we all want from an interface, so not so sure how far the
> > interface
> > > can go in offering uniform methods for what we all want an endpoint to
> > do.
> > > Sure, we all want to get configurations, but what if we wanted to to do
> > > something else (e.g., run-time config changes of certain parameters)
> and
> > > someone else didn't?
> > >
> >
> > I think if the methods are contributed back to the open source community,
> > we can form the superset of methods that work for most of the people.
> Does
> > this work for you?
> >
> >
> > >
> > > Regarding paths, I did like the simplicity of the Jersey
> implementation.
> > > E.g., put this right above class to denote a path:
> > > @Path("/resources")public class Rest{
> > > And this above a method:@GET
> > > @Path("/v1/configurations")
> > > @Produces(MediaType.APPLICATION_JSON)
> > > public String getConfiguration(@QueryParam("config") String config) {
> }
> > > This could be hit via something like [host]:[port]/resources/v1/
> > > configurations?config=BookiePort
> > >
> >
> > I think this can also fit in a simple server lifecycle, right?
> >
> >
> > >
> > > Agreed that ultimately so long as they work, I don't really care which
> > > library is used.
> > >
> >
> >
> > If that's the case, can we see if it can be based on Yiming's patch?
> >
> > - Keep the simple server lifecycle.
> > - Move the TwitterHttpServer and VexServer as plugins.
> >
> > Does it look like a base that can meet the minimal requirement here?
> >
>
> Yes for me
>
> Thoughts?
> >
>
> I will put all these things together in a wiki page
>
> Enrico
>
>
> > - Sijie
> >
> >
> > >
> > >
> > >
> > >
> > > On Wednesday, July 19, 2017, 2:52:38 AM PDT, Sijie Guo <
> > guosi...@gmail.com>
> > > wrote:
> > >
> > > On Wed, Jul 19, 2017 at 2:55 PM, Enrico Olivelli <eolive...@gmail.com>
> > > wrote:
> > >
> > > > 2017-07-19 3:27 GMT+02:00 Yiming Zang <yz...@twitter.com.invalid>:
> > > >
> > > > > Hi all,
> > > > >
> > > > > We also have a working implementation in Twitter, we use Http
> > Endpoint
> > > > > mostly for getting quorum loss information, underreplicated
> ledgers,
> > > > manage
> > > > > bookie status etc.
> > > > >
> > > > > The HTTP server in Twitter is implemented in such a way that it
> has a
> > > > > generic skeleton which can easily embed with different HTTP
> > frameworks
> > > > > (Vertx, Undertow, TwitterServer, etc), because all these frameworks
> > > have
> > > > > something in common, they all have Http Handler, we only need to
> > > > implement
> > > > > the functions for the Handler, and then bind the Handler to a
> > specific
> > > > > endpoint in the router. We intend to keep the code simple and neat,
> > > easy
> > > > to
> > > > > extend, and keep the implementation flexible. There's no limitation
> > > here.
> > > > >
> > > > > The pull request https://github.com/apache/bookkeeper/pull/210 is
> > > > > basically
> > > > > the skeleton of the HTTP server in Twitter. If this is what's
> needed
> > in
> > > > > OSS, I'm happy to keep working on the pull request and push the
> > changes
> > > > to
> > > > > master.
> > > > >
> > > >
> > > >
> > > > I think that the main point is that we need to draft a "standard"
> HTTP
> > > API
> > > > for the Bookie, then we can make several implementations of such API.
> > > > Once we have a common API it will be really easy to create tools and
> > > > integrate BK with other systems, like we are already doing with the
> > Stats
> > > > API.
> > > >
> > > > For me a great thing would be to have a ready to use HttpServlet,
> which
> > > is
> > > > the "standard" in the Java Web would and can be deployed on every
> Java
> > > Web
> > > > container.
> > > > A JAX-RS resource could be good too, but it needs more support from
> > the
> > > > container.
> > > >
> > > > In the near future we (at Diennea) wnat to start developing a global
> > > WebUI
> > > > for BookKeeper, which will show the status of the cluster, and having
> > > HTTP
> > > > endpoints in Bookie will ease this work
> > > >
> > > > Does anyone want to start a Wiki page ?
> > > >
> > >
> > > Enrico, can we hold on starting a wiki page?
> > >
> > > Currently I see three different approaches on this topic. An email
> > > discussion or a google doc might be better at this point.
> > >
> > > I think people have different opinions on what frameworks to use for
> > > implementing a http service. Can we do something like what we did for
> > stats
> > > provider. The contract that we need to make for the http service is:
> > >
> > > 1. what are the uri endpoints for different functions?
> > > 2. for each endpoint, what is request method (put/get/post), what is
> the
> > > request (json) and what is the response (json)?
> > >
> > > If all the implementations meet the requirements, it doesn't matter it
> is
> > > implemented in which frameworks. This is also how web services work for
> > > 3rd-party developers.
> > >
> > > If using this rule, I don't know see any major problems from Yiming's
> > > approach. It is actually quite clean an approach.
> > >
> > > - all the endpoints, requests and response spec go to the HttpServer
> > > interface (
> > > https://github.com/apache/bookkeeper/pull/210/files#diff-
> > > 3d2be384942070a58132c5392fc105c3).
> > > It is the contract between the bookie http server and the 3rd party
> > > developers.
> > > - all the implementations are just implementing the contract. if there
> > is a
> > > requirement to have a servlet implementation, it should be also easy to
> > > implement. same applies for jetty + JAX-RS.
> > >
> > >
> > > So can we agree on the contract and defer the implementation to
> different
> > > organizations (like what we did for stats providers)?
> > >
> > > ^^ Yiming, Dustin, Enrico
> > >
> > >
> > > - Sijie
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > -- Enrico
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Best,
> > > > >
> > > > > Yiming
> > > > >
> > > > > On Mon, Jul 17, 2017 at 12:52 PM, Dustin Castor <
> > > dustincas...@yahoo.com>
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > So I did define a little endpoint that can be used for multiple
> > > things
> > > > on
> > > > > > our end. Basically it spins up a jetty server, attaches a
> handler,
> > > and
> > > > > then
> > > > > > maps that handler to a class. Within that class, you can add
> > methods
> > > > and
> > > > > > map them using the Jersey API where you'd "decorate" methods with
> > > > things
> > > > > > like @PATH, @PARAM, etc. Currently we use it to return configs
> > (all,
> > > or
> > > > > one
> > > > > > that you specify in the URL). Realistically, it can be used for
> > > > whatever
> > > > > > you want if you define a method to handle the endpoint.
> > > > > >
> > > > > > The only limitation that I've encountered is that the Jersey API
> > > cannot
> > > > > > instantiate objects without a no arg constructor, so basically if
> > you
> > > > > want
> > > > > > to have application context, it needs to be something static and
> > > passed
> > > > > in
> > > > > > to this class as an object.
> > > > > >
> > > > > > I'd be happy to consolidate or lend a hand here. If this sounds
> > like
> > > > > > something that isn't too limited (as per what I described) for
> > > general
> > > > > use,
> > > > > > then I'd be happy to work on introducing it generally.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Dustin
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Monday, July 17, 2017, 12:45:51 PM PDT, Venkateswara Rao
> > Jujjuri <
> > > > > > jujj...@gmail.com> wrote:
> > > > > >
> > > > > >
> > > > > > + Dustin
> > > > > >
> > > > > > On Mon, Jul 17, 2017 at 12:30 PM, Sijie Guo <guosi...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > + Yiming
> > > > > > >
> > > > > > > I would suggest the people who already started the
> > implementations
> > > > > > (either
> > > > > > > from Twitter or Salesforce) taking the lead. because they
> either
> > > > > already
> > > > > > > had the implementation or are working on the implementation. I
> > > think
> > > > > the
> > > > > > > goal is to consolidate existing implementations to see if we
> can
> > > > have a
> > > > > > > unified effort on this.
> > > > > > >
> > > > > > > - Sijie
> > > > > > >
> > > > > > > On Mon, Jul 17, 2017 at 5:39 PM, Enrico Olivelli <
> > > > eolive...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > > A discussion has been started about introducing an HTTP
> > endpoint
> > > to
> > > > > the
> > > > > > > > bookie.
> > > > > > > > There has been a proposal from Twitter and this is the patch
> > > > > > > > https://github.com/apache/bookkeeper/pull/210
> > > > > > > > On Salesforce there is an ongoing implementation too.
> > > > > > > > I have added that we should provide a Servlet based approach
> or
> > > at
> > > > > > least
> > > > > > > > define a public API.
> > > > > > > > We should start a discussion and maybe a BP.
> > > > > > > > We need a leader for the discussion
> > > > > > > >
> > > > > > > > Any volunteer?
> > > > > > > > Enrico
> > > > > > > > --
> > > > > > > >
> > > > > > > >
> > > > > > > > -- Enrico Olivelli
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Jvrao
> > > > > > ---
> > > > > > First they ignore you, then they laugh at you, then they fight
> you,
> > > > then
> > > > > > you win. - Mahatma Gandhi
> > > > > >
> > > > >
> > > >
> > >
> >
> --
>
>
> -- Enrico Olivelli
>
