Yes ok agreed. I do personally think fully supporting TTL nodes (and long-poll based watches) for purposes of enabling the same types of high-level behaviors as etcd/consul would probably be useful, as per the original start of this epic thread. But even starting with an embedded proxy is a good start.
C On Mon, Apr 13, 2015 at 1:42 PM, Patrick Hunt <ph...@apache.org> wrote: > We already have a proxy. My thought was to take advantage of jetty now > being embedded in ZK (3.5+) so that a proxy wouldn't be necessary. > > Patrick > > On Mon, Apr 13, 2015 at 10:39 AM, Camille Fournier <cami...@apache.org> > wrote: > >> I've lost the thread now. Are we talking about inside of the ZK servers >> themselves, or as a proxy in front of them? >> >> On Mon, Apr 13, 2015 at 1:35 PM, Jordan Zimmerman < >> jor...@jordanzimmerman.com> wrote: >> >>> I agree that get/set kinds of things is useful - especially for OPs >>> groups. What if ZK had some sort of pluggable server support? Ops folks >>> could plug in a simple REST server, others could plug in a Thrift server, >>> etc. >>> >>> >>> >>> On April 13, 2015 at 12:31:51 PM, Patrick Hunt (ph...@apache.org) wrote: >>> >>> I think it depends on the use case. I've had a number of folks asking for >>> the REST support, in those cases I think they are mainly interested in >>> active participation (get/set rather than watches/ephemerals). >>> >>> The SPEC page has all the details on what the REST proxy currently >>> supports. >>> >>> Patrick >>> >>> On Mon, Apr 13, 2015 at 8:35 AM, Jordan Zimmerman < >>> jor...@jordanzimmerman.com> wrote: >>> >>> > How does the current REST code handle watches and background >>> > notifications? Are clients required to poll periodically? This is >>> really >>> > RPC and REST is not ideal for RPC. So, as I mentioned before, I think >>> the >>> > ZK team should consider a real RPC system like Thrift instead of REST. >>> > >>> > -Jordan >>> > >>> > >>> > >>> > On April 11, 2015 at 1:33:03 PM, Patrick Hunt (ph...@apache.org) >>> wrote: >>> > >>> > We pack quite a bit of functionality in our client side libraries. >>> That's >>> > probably one of the main things you'll noticed if you try to use REST. >>> But >>> > then if your use case is primarily configuration, or something that >>> doesn't >>> > require sessions then the current REST support should be more than >>> > sufficient. What are the other systems doing that we are currently not >>> > doing wrt API support? >>> > >>> > I'd think that taking our current rest contrib, updating the >>> dependencies, >>> > and deploying as part of the embedded jetty would work well with >>> minimal >>> > effort. >>> > >>> > Patrick >>> > >>> > On Sat, Apr 11, 2015 at 9:22 AM, Camille Fournier <cami...@apache.org> >>> > wrote: >>> > >>> > > I agree that they have to write clients that do work, but there is >>> > clearly >>> > > a desire and willingness out there to do that work in exchange for a >>> more >>> > > "obvious" way of interacting. So supporting it should be something >>> that >>> > we >>> > > consider if the users of such systems want the option. >>> > > >>> > > On Sat, Apr 11, 2015 at 12:07 PM, Jordan Zimmerman < >>> > > jor...@jordanzimmerman.com> wrote: >>> > > >>> > > > REST has issues for end users. They will still have to write >>> clients >>> > and >>> > > > do a lot of extra work. REST support is good in most languages but >>> > having >>> > > > native support is superior. That’s why I choose Thrift for the >>> > CuratorRPC >>> > > > Proxy. >>> > > > >>> > > > -Jordan >>> > > > >>> > > > >>> > > > On April 10, 2015 at 9:32:47 PM, Michi Mutsuzaki ( >>> > mi...@cs.stanford.edu) >>> > > > wrote: >>> > > > >>> > > > Yeah it's quite painful to manage another set of processes just for >>> > > > proxying requests. I'd definitely use this if I can run it >>> embedded in >>> > > > the ZooKeeper process. I'm very excited about the idea of being >>> able >>> > > > to use curl to look at what's in ZooKeeper :) >>> > > > >>> > > > On Fri, Apr 10, 2015 at 6:03 PM, Patrick Hunt <ph...@apache.org> >>> > wrote: >>> > > > > Here's the spec and readme from contrib/rest: >>> > > > > >>> > > > > >>> > > > >>> > > >>> > >>> http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/SPEC.txt?view=markup >>> > > > > >>> > > > >>> > > >>> > >>> http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/README.txt?view=markup >>> > > > > >>> > > > > The current implementation is a standalone proxy. It's not >>> embedded >>> > in >>> > > zk >>> > > > > itself. That might be part of the reason. >>> > > > > >>> > > > > Patrick >>> > > > > >>> > > > > On Fri, Apr 10, 2015 at 4:43 PM, Camille Fournier < >>> > cami...@apache.org> >>> > > > > wrote: >>> > > > > >>> > > > >> Forgive me for not reading the code, can you share in more >>> detail >>> > what >>> > > > the >>> > > > >> existing REST proxy provides? I'm curious why people are >>> jumping to >>> > > use >>> > > > >> etcd because of ease of use w/http access if we already have >>> > something >>> > > > that >>> > > > >> works? >>> > > > >> >>> > > > >> On Fri, Apr 10, 2015 at 7:28 PM, Patrick Hunt <ph...@apache.org >>> > >>> > > wrote: >>> > > > >> >>> > > > >> > There's the REST work in contrib. Both Andrei and I worked on >>> that >>> > > - I >>> > > > >> did >>> > > > >> > the basic support and Andrei added sessions and heartbeating >>> among >>> > > > other >>> > > > >> > improvements. >>> > > > >> > >>> > > > >> > Now that 3.5 has embedded Jetty it should be much simpler to >>> run >>> > > REST >>> > > > as >>> > > > >> > part of the ZK service itself. When the original proxy work >>> was >>> > done >>> > > > >> Jetty >>> > > > >> > was not yet part of ZK. >>> > > > >> > >>> > > > >> > Patrick >>> > > > >> > >>> > > > >> > On Thu, Apr 9, 2015 at 10:20 AM, Jordan Zimmerman < >>> > > > >> > jor...@jordanzimmerman.com> wrote: >>> > > > >> > >>> > > > >> >> Since Curator is now Apache and I'm no longer at Netflix, I >>> don't >>> > > > follow >>> > > > >> >> Netflix messages very much. Sorry about that. >>> > > > >> >> >>> > > > >> >> -Jordan >>> > > > >> >> >>> > > > >> >> >>> > > > >> >> >>> > > > >> >> On April 9, 2015 at 12:15:02 PM, Camille Fournier ( >>> > > > cami...@apache.org) >>> > > > >> >> wrote: >>> > > > >> >> >>> > > > >> >> Thanks Jordan! I actually asked on Twitter whether Netflix >>> had >>> > > > anything >>> > > > >> >> but >>> > > > >> >> didn't get a clear answer. >>> > > > >> >> >>> > > > >> >> On Thu, Apr 9, 2015 at 11:22 AM, Jordan Zimmerman < >>> > > > >> >> jor...@jordanzimmerman.com> wrote: >>> > > > >> >> >>> > > > >> >> > FYI >>> > > > >> >> > >>> > > > >> >> > Curator now has a Thrift-based proxy that has all the ZK >>> APIs >>> > > > exposed >>> > > > >> as >>> > > > >> >> > well as Curator's added APIs and recipes: >>> > > > >> >> > >>> > > > >> >> > http://curator.apache.org/curator-x-rpc/index.html >>> > > > >> >> > >>> > > > >> >> > -Jordan >>> > > > >> >> > >>> > > > >> >> > On April 8, 2015 at 2:09:33 PM, Camille Fournier ( >>> > > > cami...@apache.org) >>> > > > >> >> > wrote: >>> > > > >> >> > >>> > > > >> >> > All, >>> > > > >> >> > >>> > > > >> >> > I've been doing a bit of research on etcd as part of work >>> for >>> > an >>> > > > >> >> upcoming >>> > > > >> >> > talk, and it has gotten me thinking about what it would >>> take to >>> > > > create >>> > > > >> >> an >>> > > > >> >> > http version of ZK for certain operations. For many >>> operations >>> > > you >>> > > > >> could >>> > > > >> >> > put an http proxy in front of ZK to translate, even >>> > implementing >>> > > > the >>> > > > >> >> > "long-poll-style" watch operation to some extent. But it >>> would >>> > be >>> > > > very >>> > > > >> >> > hard >>> > > > >> >> > to do a temporary node via a proxy without a lot of proxy >>> > > failover >>> > > > >> >> > complexity. >>> > > > >> >> > >>> > > > >> >> > As a bit of background, if you want to do an "ephemeral" >>> node >>> > in >>> > > > etcd, >>> > > > >> >> you >>> > > > >> >> > basically create a key with a TTL. Unless the key is >>> updated >>> > > with a >>> > > > >> new >>> > > > >> >> > TTL, the key will auto-expire when the TTL is reached. >>> Now, I >>> > > have >>> > > > a >>> > > > >> lot >>> > > > >> >> > of >>> > > > >> >> > thoughts about this (seems like you have to implement >>> > heartbeats >>> > > > via >>> > > > >> >> http >>> > > > >> >> > to truly mimic ephemeral nodes which may not be as simple >>> as >>> > all >>> > > > this >>> > > > >> >> http >>> > > > >> >> > sounds), but I do think that if there is appetite for easy >>> http >>> > > > access >>> > > > >> >> for >>> > > > >> >> > consensus systems we should at least take the time to think >>> > about >>> > > > what >>> > > > >> >> it >>> > > > >> >> > would take for us to provide this. In particular, I think >>> we'd >>> > > > have to >>> > > > >> >> > make >>> > > > >> >> > it possible to create a node with a TTL that is not tied >>> to a >>> > > > >> particular >>> > > > >> >> > session. >>> > > > >> >> > >>> > > > >> >> > Curious to see if anyone has any thoughts on this. It seems >>> > like >>> > > a >>> > > > bit >>> > > > >> >> of >>> > > > >> >> > a >>> > > > >> >> > shame that ZK, which is a good battle-tested system, is >>> > > frequently >>> > > > >> being >>> > > > >> >> > passed-over these days because of the complexity of >>> clients, >>> > and >>> > > > the >>> > > > >> >> fact >>> > > > >> >> > that it is really pretty damn hard to do a client impl in >>> > certain >>> > > > >> >> > languages >>> > > > >> >> > (Ruby is the notable one I've heard). >>> > > > >> >> > >>> > > > >> >> > Best, >>> > > > >> >> > C >>> > > > >> >> > >>> > > > >> >> > >>> > > > >> >> >>> > > > >> > >>> > > > >> > >>> > > > >> >>> > > > >>> > > >>> > >>> >> >> >
