https://issues.apache.org/jira/browse/ZOOKEEPER-2169
That's for the TTL work On Mon, Apr 13, 2015 at 6:51 PM, Michi Mutsuzaki <mutsuz...@gmail.com> wrote: > We should move the discussion to JIRA so that we don't lose all the > inputs. Camille, could you open 2 JIRAs when you get a chance? > > 1. Node TTLs > 2. Pluggable RPC endpoints, starting with HTTP > > > On Mon, Apr 13, 2015 at 3:00 PM, Patrick Hunt <ph...@apache.org> wrote: > > That's really the easy part. Coming up with something that will work in > our > > context is the harder bit IMO. For example, if you have multiple REST > > service endpoints how will that work, one per zk server? Are you going to > > tie a particular client session to a particular server, how will sessions > > be maintained. Would you want to put a http load balancer in between? > > Typically you want to do this otw you need more logic on the client (like > > we have today in the java client). Seems like there's a lot of these > types > > of issues that would need to be worked through. Same (but different) for > > thrift support. > > > > Patrick > > > > On Mon, Apr 13, 2015 at 2:19 PM, Flavio Junqueira < > > fpjunque...@yahoo.com.invalid> wrote: > > > >> I also like the plugin idea, it's a good one. > >> > >> -Flavio > >> > >> > On 13 Apr 2015, at 18:58, Patrick Hunt <ph...@apache.org> wrote: > >> > > >> > Once something is generally available, embedded and easy to enable, it > >> > probably won't take much for someone to extend. Once they see the > >> benefits. > >> > Although there are already folks interested in what we have today. > >> > > >> > We could do the same thing re thrift, or any other "plugin" for that > >> > matter, (enable dynamic config of embedded endpoint) if there's > interest. > >> > > >> > Patrick > >> > > >> > On Mon, Apr 13, 2015 at 10:48 AM, Camille Fournier < > cami...@apache.org> > >> > wrote: > >> > > >> >> Yes ok agreed. I do personally think fully supporting TTL nodes (and > >> >> long-poll based watches) for purposes of enabling the same types of > >> >> high-level behaviors as etcd/consul would probably be useful, as per > the > >> >> original start of this epic thread. But even starting with an > embedded > >> >> proxy is a good start. > >> >> > >> >> C > >> >> > >> >> On Mon, Apr 13, 2015 at 1:42 PM, Patrick Hunt <ph...@apache.org> > wrote: > >> >> > >> >>> We already have a proxy. My thought was to take advantage of jetty > now > >> >>> being embedded in ZK (3.5+) so that a proxy wouldn't be necessary. > >> >>> > >> >>> Patrick > >> >>> > >> >>> On Mon, Apr 13, 2015 at 10:39 AM, Camille Fournier < > cami...@apache.org > >> > > >> >>> wrote: > >> >>> > >> >>>> I've lost the thread now. Are we talking about inside of the ZK > >> servers > >> >>>> themselves, or as a proxy in front of them? > >> >>>> > >> >>>> On Mon, Apr 13, 2015 at 1:35 PM, Jordan Zimmerman < > >> >>>> jor...@jordanzimmerman.com> wrote: > >> >>>> > >> >>>>> I agree that get/set kinds of things is useful - especially for > OPs > >> >>>>> groups. What if ZK had some sort of pluggable server support? Ops > >> folks > >> >>>>> could plug in a simple REST server, others could plug in a Thrift > >> server, > >> >>>>> etc. > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> On April 13, 2015 at 12:31:51 PM, Patrick Hunt (ph...@apache.org) > >> >>>>> wrote: > >> >>>>> > >> >>>>> I think it depends on the use case. I've had a number of folks > asking > >> >>>>> for > >> >>>>> the REST support, in those cases I think they are mainly > interested > >> in > >> >>>>> active participation (get/set rather than watches/ephemerals). > >> >>>>> > >> >>>>> The SPEC page has all the details on what the REST proxy currently > >> >>>>> supports. > >> >>>>> > >> >>>>> Patrick > >> >>>>> > >> >>>>> On Mon, Apr 13, 2015 at 8:35 AM, Jordan Zimmerman < > >> >>>>> jor...@jordanzimmerman.com> wrote: > >> >>>>> > >> >>>>>> How does the current REST code handle watches and background > >> >>>>>> notifications? Are clients required to poll periodically? This is > >> >>>>> really > >> >>>>>> RPC and REST is not ideal for RPC. So, as I mentioned before, I > >> think > >> >>>>> the > >> >>>>>> ZK team should consider a real RPC system like Thrift instead of > >> REST. > >> >>>>>> > >> >>>>>> -Jordan > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> On April 11, 2015 at 1:33:03 PM, Patrick Hunt (ph...@apache.org) > >> >>>>> wrote: > >> >>>>>> > >> >>>>>> We pack quite a bit of functionality in our client side > libraries. > >> >>>>> That's > >> >>>>>> probably one of the main things you'll noticed if you try to use > >> >>>>> REST. But > >> >>>>>> then if your use case is primarily configuration, or something > that > >> >>>>> doesn't > >> >>>>>> require sessions then the current REST support should be more > than > >> >>>>>> sufficient. What are the other systems doing that we are > currently > >> not > >> >>>>>> doing wrt API support? > >> >>>>>> > >> >>>>>> I'd think that taking our current rest contrib, updating the > >> >>>>> dependencies, > >> >>>>>> and deploying as part of the embedded jetty would work well with > >> >>>>> minimal > >> >>>>>> effort. > >> >>>>>> > >> >>>>>> Patrick > >> >>>>>> > >> >>>>>> On Sat, Apr 11, 2015 at 9:22 AM, Camille Fournier < > >> cami...@apache.org > >> >>>>>> > >> >>>>>> wrote: > >> >>>>>> > >> >>>>>>> I agree that they have to write clients that do work, but there > is > >> >>>>>> clearly > >> >>>>>>> a desire and willingness out there to do that work in exchange > for > >> >>>>> a more > >> >>>>>>> "obvious" way of interacting. So supporting it should be > something > >> >>>>> that > >> >>>>>> we > >> >>>>>>> consider if the users of such systems want the option. > >> >>>>>>> > >> >>>>>>> On Sat, Apr 11, 2015 at 12:07 PM, Jordan Zimmerman < > >> >>>>>>> jor...@jordanzimmerman.com> wrote: > >> >>>>>>> > >> >>>>>>>> REST has issues for end users. They will still have to write > >> >>>>> clients > >> >>>>>> and > >> >>>>>>>> do a lot of extra work. REST support is good in most languages > but > >> >>>>>> having > >> >>>>>>>> native support is superior. That's why I choose Thrift for the > >> >>>>>> CuratorRPC > >> >>>>>>>> Proxy. > >> >>>>>>>> > >> >>>>>>>> -Jordan > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> On April 10, 2015 at 9:32:47 PM, Michi Mutsuzaki ( > >> >>>>>> mi...@cs.stanford.edu) > >> >>>>>>>> wrote: > >> >>>>>>>> > >> >>>>>>>> Yeah it's quite painful to manage another set of processes just > >> >>>>> for > >> >>>>>>>> proxying requests. I'd definitely use this if I can run it > >> >>>>> embedded in > >> >>>>>>>> the ZooKeeper process. I'm very excited about the idea of being > >> >>>>> able > >> >>>>>>>> to use curl to look at what's in ZooKeeper :) > >> >>>>>>>> > >> >>>>>>>> On Fri, Apr 10, 2015 at 6:03 PM, Patrick Hunt < > ph...@apache.org> > >> >>>>>> wrote: > >> >>>>>>>>> Here's the spec and readme from contrib/rest: > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> > http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/SPEC.txt?view=markup > >> >>>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> > http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/README.txt?view=markup > >> >>>>>>>>> > >> >>>>>>>>> The current implementation is a standalone proxy. It's not > >> >>>>> embedded > >> >>>>>> in > >> >>>>>>> zk > >> >>>>>>>>> itself. That might be part of the reason. > >> >>>>>>>>> > >> >>>>>>>>> Patrick > >> >>>>>>>>> > >> >>>>>>>>> On Fri, Apr 10, 2015 at 4:43 PM, Camille Fournier < > >> >>>>>> cami...@apache.org> > >> >>>>>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>>> Forgive me for not reading the code, can you share in more > >> >>>>> detail > >> >>>>>> what > >> >>>>>>>> the > >> >>>>>>>>>> existing REST proxy provides? I'm curious why people are > >> >>>>> jumping to > >> >>>>>>> use > >> >>>>>>>>>> etcd because of ease of use w/http access if we already have > >> >>>>>> something > >> >>>>>>>> that > >> >>>>>>>>>> works? > >> >>>>>>>>>> > >> >>>>>>>>>> On Fri, Apr 10, 2015 at 7:28 PM, Patrick Hunt < > >> >>>>> ph...@apache.org> > >> >>>>>>> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>>> There's the REST work in contrib. Both Andrei and I worked > >> >>>>> on that > >> >>>>>>> - I > >> >>>>>>>>>> did > >> >>>>>>>>>>> the basic support and Andrei added sessions and heartbeating > >> >>>>> among > >> >>>>>>>> other > >> >>>>>>>>>>> improvements. > >> >>>>>>>>>>> > >> >>>>>>>>>>> Now that 3.5 has embedded Jetty it should be much simpler to > >> >>>>> run > >> >>>>>>> REST > >> >>>>>>>> as > >> >>>>>>>>>>> part of the ZK service itself. When the original proxy work > >> >>>>> was > >> >>>>>> done > >> >>>>>>>>>> Jetty > >> >>>>>>>>>>> was not yet part of ZK. > >> >>>>>>>>>>> > >> >>>>>>>>>>> Patrick > >> >>>>>>>>>>> > >> >>>>>>>>>>> On Thu, Apr 9, 2015 at 10:20 AM, Jordan Zimmerman < > >> >>>>>>>>>>> jor...@jordanzimmerman.com> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>>> Since Curator is now Apache and I'm no longer at Netflix, I > >> >>>>> don't > >> >>>>>>>> follow > >> >>>>>>>>>>>> Netflix messages very much. Sorry about that. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> -Jordan > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> On April 9, 2015 at 12:15:02 PM, Camille Fournier ( > >> >>>>>>>> cami...@apache.org) > >> >>>>>>>>>>>> wrote: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Thanks Jordan! I actually asked on Twitter whether Netflix > >> >>>>> had > >> >>>>>>>> anything > >> >>>>>>>>>>>> but > >> >>>>>>>>>>>> didn't get a clear answer. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> On Thu, Apr 9, 2015 at 11:22 AM, Jordan Zimmerman < > >> >>>>>>>>>>>> jor...@jordanzimmerman.com> wrote: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>>> FYI > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Curator now has a Thrift-based proxy that has all the ZK > >> >>>>> APIs > >> >>>>>>>> exposed > >> >>>>>>>>>> as > >> >>>>>>>>>>>>> well as Curator's added APIs and recipes: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> http://curator.apache.org/curator-x-rpc/index.html > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> -Jordan > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> On April 8, 2015 at 2:09:33 PM, Camille Fournier ( > >> >>>>>>>> cami...@apache.org) > >> >>>>>>>>>>>>> wrote: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> All, > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> I've been doing a bit of research on etcd as part of work > >> >>>>> for > >> >>>>>> an > >> >>>>>>>>>>>> upcoming > >> >>>>>>>>>>>>> talk, and it has gotten me thinking about what it would > >> >>>>> take to > >> >>>>>>>> create > >> >>>>>>>>>>>> an > >> >>>>>>>>>>>>> http version of ZK for certain operations. For many > >> >>>>> operations > >> >>>>>>> you > >> >>>>>>>>>> could > >> >>>>>>>>>>>>> put an http proxy in front of ZK to translate, even > >> >>>>>> implementing > >> >>>>>>>> the > >> >>>>>>>>>>>>> "long-poll-style" watch operation to some extent. But it > >> >>>>> would > >> >>>>>> be > >> >>>>>>>> very > >> >>>>>>>>>>>>> hard > >> >>>>>>>>>>>>> to do a temporary node via a proxy without a lot of proxy > >> >>>>>>> failover > >> >>>>>>>>>>>>> complexity. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> As a bit of background, if you want to do an "ephemeral" > >> >>>>> node > >> >>>>>> in > >> >>>>>>>> etcd, > >> >>>>>>>>>>>> you > >> >>>>>>>>>>>>> basically create a key with a TTL. Unless the key is > >> >>>>> updated > >> >>>>>>> with a > >> >>>>>>>>>> new > >> >>>>>>>>>>>>> TTL, the key will auto-expire when the TTL is reached. > >> >>>>> Now, I > >> >>>>>>> have > >> >>>>>>>> a > >> >>>>>>>>>> lot > >> >>>>>>>>>>>>> of > >> >>>>>>>>>>>>> thoughts about this (seems like you have to implement > >> >>>>>> heartbeats > >> >>>>>>>> via > >> >>>>>>>>>>>> http > >> >>>>>>>>>>>>> to truly mimic ephemeral nodes which may not be as simple > >> >>>>> as > >> >>>>>> all > >> >>>>>>>> this > >> >>>>>>>>>>>> http > >> >>>>>>>>>>>>> sounds), but I do think that if there is appetite for > >> >>>>> easy http > >> >>>>>>>> access > >> >>>>>>>>>>>> for > >> >>>>>>>>>>>>> consensus systems we should at least take the time to > >> >>>>> think > >> >>>>>> about > >> >>>>>>>> what > >> >>>>>>>>>>>> it > >> >>>>>>>>>>>>> would take for us to provide this. In particular, I think > >> >>>>> we'd > >> >>>>>>>> have to > >> >>>>>>>>>>>>> make > >> >>>>>>>>>>>>> it possible to create a node with a TTL that is not tied > >> >>>>> to a > >> >>>>>>>>>> particular > >> >>>>>>>>>>>>> session. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Curious to see if anyone has any thoughts on this. It > >> >>>>> seems > >> >>>>>> like > >> >>>>>>> a > >> >>>>>>>> bit > >> >>>>>>>>>>>> of > >> >>>>>>>>>>>>> a > >> >>>>>>>>>>>>> shame that ZK, which is a good battle-tested system, is > >> >>>>>>> frequently > >> >>>>>>>>>> being > >> >>>>>>>>>>>>> passed-over these days because of the complexity of > >> >>>>> clients, > >> >>>>>> and > >> >>>>>>>> the > >> >>>>>>>>>>>> fact > >> >>>>>>>>>>>>> that it is really pretty damn hard to do a client impl in > >> >>>>>> certain > >> >>>>>>>>>>>>> languages > >> >>>>>>>>>>>>> (Ruby is the notable one I've heard). > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Best, > >> >>>>>>>>>>>>> C > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> >>>> > >> >>>> > >> >>> > >> >> > >> > >> >
