Re: [PROPOSAL] Container nodes

2015-04-14 Thread Jordan Zimmerman
I now have a first pass production-level implementation. It still needs more testing and, more importantly, feedback from those who know ZK internals better than me. https://issues.apache.org/jira/browse/ZOOKEEPER-2163 -Jordan On April 14, 2015 at 1:52:38 PM, kishore g (g.kish...@gmail.com)

Re: [PROPOSAL] Container nodes

2015-04-14 Thread Jordan Zimmerman
How would one go about using this feature? Perhaps a pseudo api and client  code will help me understand.  Here’s my current API: zk.createContainer(path, data, acls); // and standard variations for async I opted to create a new API because I didn’t want to add a new CreateMode. Also, sequenti

Re: [PROPOSAL] Container nodes

2015-04-14 Thread kishore g
Hi Jordon, I like this feature and always thought it would be useful to have something like this for Apache Helix as well. We do have a clean up thread that deletes the znodes. But I felt it was tied to Helix. Here are some of the questions that made me think its better to have the user of zookee

Re: [PROPOSAL] Container nodes

2015-04-14 Thread Jordan Zimmerman
Thanks. How about the first question? -Jordan On April 14, 2015 at 12:49:52 PM, Camille Fournier (cami...@apache.org) wrote: Look at the session managers, they track what sessions are alive and clean up when they aren't. On Tue, Apr 14, 2015 at 1:49 PM, Camille Fournier wrote: > Loo

Re: [PROPOSAL] Container nodes

2015-04-14 Thread Camille Fournier
Look at the session managers, they track what sessions are alive and clean up when they aren't. C On Tue, Apr 14, 2015 at 1:36 PM, Jordan Zimmerman < jor...@jordanzimmerman.com> wrote: > Another question… > > So, my two current questions are: > > * noting that a ZNode is a container, would you s

Re: [PROPOSAL] Container nodes

2015-04-14 Thread Camille Fournier
Look at the session managers, they track what sessions are alive and clean up when they aren't. On Tue, Apr 14, 2015 at 1:49 PM, Camille Fournier wrote: > Look at the session managers, they track what sessions are alive and clean > up when they aren't. > > C > > On Tue, Apr 14, 2015 at 1:36 PM,

Re: [PROPOSAL] Container nodes

2015-04-14 Thread Jordan Zimmerman
Another question…  So, my two current questions are: * noting that a ZNode is a container, would you suggest the hack of a special ephemeralOwner value or would you add a new field to Stat? * is there a current mechanism in the ZK server code (for the leader in particular) to handle periodic h

Re: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
As for noting that a ZNode is a container, would you suggest the hack of a special ephemeralOwner value or would you add a new field to Stat? -Jordan On April 10, 2015 at 6:40:23 PM, Patrick Hunt (ph...@apache.org) wrote: Adding is typically good from a b/w compact perspective. If you use the

RE: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
t; To: ph...@apache.org; dev@zookeeper.apache.org > CC: mi...@cs.stanford.edu > Subject: Re: [PROPOSAL] Container nodes > > How are things such as Create2Request et al generated? I see the comment that > it’s the Hadoop compiler but I don’t see the source files anywhere. Is it OK > to man

RE: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
://github.com/apache/zookeeper/blob/trunk/src/zookeeper.jute - Hongchao Deng > Date: Mon, 13 Apr 2015 13:47:09 -0500 > From: jor...@jordanzimmerman.com > To: ph...@apache.org; dev@zookeeper.apache.org > CC: mi...@cs.stanford.edu > Subject: Re: [PROPOSAL] Container nodes

RE: [PROPOSAL] Container nodes

2015-04-13 Thread Hongchao Deng
stanford.edu > Subject: Re: [PROPOSAL] Container nodes > > How are things such as Create2Request et al generated? I see the comment that > it’s the Hadoop compiler but I don’t see the source files anywhere. Is it OK > to manually create these (for new classes) or am I missing some source?

Re: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
How are things such as Create2Request et al generated? I see the comment that it’s the Hadoop compiler but I don’t see the source files anywhere. Is it OK to manually create these (for new classes) or am I missing some source? -Jordan On April 10, 2015 at 6:40:23 PM, Patrick Hunt (ph...@apach

Re: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
FYI https://issues.apache.org/jira/browse/ZOOKEEPER-2163 On Fri, Apr 10, 2015 at 6:39 PM, Patrick Hunt wrote: > Adding is typically good from a b/w compact perspective. If you use the new > feature (at runtime) it generally precludes rollback though. > > See CreateTxn and CreateTxnV0 > > A bit o

Re: [PROPOSAL] Container nodes

2015-04-10 Thread Patrick Hunt
Adding is typically good from a b/w compact perspective. If you use the new feature (at runtime) it generally precludes rollback though. See CreateTxn and CreateTxnV0 A bit of background on convenience vs availability: Originally in ZK's life we explicitly stayed away from such operations at the

Re: [PROPOSAL] Container nodes

2015-04-09 Thread Jordan Zimmerman
You don’t even need to look at cversion. If the parent node is a container and has no children (i.e. the node being deleted is the last child), it gets deleted. The trouble I’m currently having, though, is that I don’t want to modify the  CreateTxn record. I can’t find a place to mark that the n

Re: [PROPOSAL] Container nodes

2015-04-09 Thread Michi Mutsuzaki
I see, so the container znode is a znode that gets deleted if it's empty and it ever had a child (cversion is greater than zero). It sounds good to me. Let's see what other people say. Thanks Jordan! On Thu, Apr 9, 2015 at 10:20 AM, Jordan Zimmerman wrote: > This sounds great to me, but it sound

Re: [PROPOSAL] Container nodes

2015-04-09 Thread Jordan Zimmerman
This sounds great to me, but it sounds a lot like ZOOKEEPER-723.  The problem with both ZOOKEEPER-723 and ZOOKEEPER-834 is that it overloads the concept of EPHEMERAL. EPHEMERALs are tied to sessions. In the use cases that I see, the parent node is always PERSISTENT - i.e. not tied to a session.

Re: [PROPOSAL] Container nodes

2015-04-09 Thread Michi Mutsuzaki
Hi Jordan. This sounds great to me, but it sounds a lot like ZOOKEEPER-723. Different people had different ideas there, but the original description was: "rather than changing the semantics of ephemeral nodes, i propose ephemeral parents: znodes that disappear when they have no more children. thi

[PROPOSAL] Container nodes

2015-04-09 Thread Jordan Zimmerman
BACKGROUND A recurring problem for ZooKeeper users is garbage collection of parent nodes. Many recipes (e.g. locks, leaders, etc.) call for the creation of a parent node under which participants create sequential nodes. When the participant is done, it deletes its node. In practice,