How to do hot standby with consistent store

2015-04-13 Thread Hongchao Deng
@Camille asked me how hot standby works in etcd. It's too much to answer over the weekend. I've been organizing information and try my best to answer now. It's better late than nothing. The goal is to make physically different identities behave logically the same. In order to do so, you need two t

[jira] [Commented] (ZOOKEEPER-2156) If JAVA_HOME is not set zk startup and fetching status command execution result misleads user.

2015-04-13 Thread J.Andreina (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493531#comment-14493531 ] J.Andreina commented on ZOOKEEPER-2156: --- Testcase failures are not related to th

Re: design thoughts: node TTLs

2015-04-13 Thread Michi Mutsuzaki
We should move the discussion to JIRA so that we don't lose all the inputs. Camille, could you open 2 JIRAs when you get a chance? 1. Node TTLs 2. Pluggable RPC endpoints, starting with HTTP On Mon, Apr 13, 2015 at 3:00 PM, Patrick Hunt wrote: > That's really the easy part. Coming up with somet

Re: design thoughts: node TTLs

2015-04-13 Thread Patrick Hunt
That's really the easy part. Coming up with something that will work in our context is the harder bit IMO. For example, if you have multiple REST service endpoints how will that work, one per zk server? Are you going to tie a particular client session to a particular server, how will sessions be ma

Re: design thoughts: node TTLs

2015-04-13 Thread Flavio Junqueira
I also like the plugin idea, it's a good one. -Flavio > On 13 Apr 2015, at 18:58, Patrick Hunt wrote: > > Once something is generally available, embedded and easy to enable, it > probably won't take much for someone to extend. Once they see the benefits. > Although there are already folks inter

Re: design thoughts: node TTLs

2015-04-13 Thread Flavio Junqueira
An even easier way to start is to put an HTTP server in front of a ZK client. There is clearly a performance overhead, but is high performance a requirement for such use cases? -Flavio > On 13 Apr 2015, at 18:48, Camille Fournier wrote: > > Yes ok agreed. I do personally think fully supporti

ZooKeeper-trunk-openjdk7 - Build # 779 - Failure

2015-04-13 Thread Apache Jenkins Server
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/779/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 107 lines...] [mkdir] Created dir: /x1/jenkins

Re: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
As for noting that a ZNode is a container, would you suggest the hack of a special ephemeralOwner value or would you add a new field to Stat? -Jordan On April 10, 2015 at 6:40:23 PM, Patrick Hunt (ph...@apache.org) wrote: Adding is typically good from a b/w compact perspective. If you use the

RE: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
Never mind - found it (ant build-generated) On April 13, 2015 at 2:14:24 PM, Jordan Zimmerman (jor...@jordanzimmerman.com) wrote: OK - I found it. How do I build it so that the generated files get correctly placed in the source tree? I don’t see an ant task for it. -Jordan On April 13, 20

RE: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
OK - I found it. How do I build it so that the generated files get correctly placed in the source tree? I don’t see an ant task for it. -Jordan On April 13, 2015 at 1:53:08 PM, Hongchao Deng (fengjingc...@hotmail.com) wrote: Those are generated by the jute compiler. Source file: https://git

RE: [PROPOSAL] Container nodes

2015-04-13 Thread Hongchao Deng
Those are generated by the jute compiler. Source file: https://github.com/apache/zookeeper/blob/trunk/src/zookeeper.jute - Hongchao Deng > Date: Mon, 13 Apr 2015 13:47:09 -0500 > From: jor...@jordanzimmerman.com > To: ph...@apache.org; dev@zookeeper.apache.org > CC: mi...@cs.stanford.edu > Subjec

Re: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
How are things such as Create2Request et al generated? I see the comment that it’s the Hadoop compiler but I don’t see the source files anywhere. Is it OK to manually create these (for new classes) or am I missing some source? -Jordan On April 10, 2015 at 6:40:23 PM, Patrick Hunt (ph...@apach

Re: [PROPOSAL] Container nodes

2015-04-13 Thread Jordan Zimmerman
FYI https://issues.apache.org/jira/browse/ZOOKEEPER-2163 On Fri, Apr 10, 2015 at 6:39 PM, Patrick Hunt wrote: > Adding is typically good from a b/w compact perspective. If you use the new > feature (at runtime) it generally precludes rollback though. > > See CreateTxn and CreateTxnV0 > > A bit o

[jira] [Created] (ZOOKEEPER-2163) Introduce new ZNode type: container

2015-04-13 Thread Jordan Zimmerman (JIRA)
Jordan Zimmerman created ZOOKEEPER-2163: --- Summary: Introduce new ZNode type: container Key: ZOOKEEPER-2163 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2163 Project: ZooKeeper I

Re: design thoughts: node TTLs

2015-04-13 Thread Patrick Hunt
Once something is generally available, embedded and easy to enable, it probably won't take much for someone to extend. Once they see the benefits. Although there are already folks interested in what we have today. We could do the same thing re thrift, or any other "plugin" for that matter, (enable

[jira] [Updated] (ZOOKEEPER-2147) Remove ZOO_NOTWATCHING_EVENT

2015-04-13 Thread Tom Distler (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Distler updated ZOOKEEPER-2147: --- Attachment: ZOOKEEPER-2147.patch Granted: ASF license consistent with current ZooKeeper li

Re: design thoughts: node TTLs

2015-04-13 Thread Camille Fournier
Yes ok agreed. I do personally think fully supporting TTL nodes (and long-poll based watches) for purposes of enabling the same types of high-level behaviors as etcd/consul would probably be useful, as per the original start of this epic thread. But even starting with an embedded proxy is a good st

[jira] [Commented] (ZOOKEEPER-1626) Zookeeper C client should be tolerant of clock adjustments

2015-04-13 Thread ram yadav (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492716#comment-14492716 ] ram yadav commented on ZOOKEEPER-1626: -- We are running zookeeper C client along w

Re: design thoughts: node TTLs

2015-04-13 Thread Patrick Hunt
We already have a proxy. My thought was to take advantage of jetty now being embedded in ZK (3.5+) so that a proxy wouldn't be necessary. Patrick On Mon, Apr 13, 2015 at 10:39 AM, Camille Fournier wrote: > I've lost the thread now. Are we talking about inside of the ZK servers > themselves, or

Re: design thoughts: node TTLs

2015-04-13 Thread Camille Fournier
I've lost the thread now. Are we talking about inside of the ZK servers themselves, or as a proxy in front of them? On Mon, Apr 13, 2015 at 1:35 PM, Jordan Zimmerman < jor...@jordanzimmerman.com> wrote: > I agree that get/set kinds of things is useful - especially for OPs > groups. What if ZK had

Re: design thoughts: node TTLs

2015-04-13 Thread Patrick Hunt
Yes, pluggable is what I had in mind. My thought was that now that jetty is embedded it shouldn't be too hard to allow the REST to be registered dynamically via config. Patrick On Mon, Apr 13, 2015 at 10:35 AM, Jordan Zimmerman < jor...@jordanzimmerman.com> wrote: > I agree that get/set kinds of

Re: design thoughts: node TTLs

2015-04-13 Thread Jordan Zimmerman
I agree that get/set kinds of things is useful - especially for OPs groups. What if ZK had some sort of pluggable server support? Ops folks could plug in a simple REST server, others could plug in a Thrift server, etc. On April 13, 2015 at 12:31:51 PM, Patrick Hunt (ph...@apache.org) wrote: I

Re: design thoughts: node TTLs

2015-04-13 Thread Patrick Hunt
I think it depends on the use case. I've had a number of folks asking for the REST support, in those cases I think they are mainly interested in active participation (get/set rather than watches/ephemerals). The SPEC page has all the details on what the REST proxy currently supports. Patrick On

RE: design thoughts: node TTLs

2015-04-13 Thread Hongchao Deng
Awesome! Is there any design docs and related code to read? I'm not very sure. The proxy is more like a client to ZooKeeper. I'm thinking in terms of ZK server RPC api. - Hongchao Deng > Date: Mon, 13 Apr 2015 11:57:45 -0500 > From: jor...@jordanzimmerman.com > To: fengjingc...@hotmail.com; dev@

RE: design thoughts: node TTLs

2015-04-13 Thread Jordan Zimmerman
Hongchao, as I mentioned before I wrote an RPC proxy for Curator that exposes all the ZooKeeper APIs and all the Curator APIs and recipes. This proxy uses Thrift and writing clients is actually very easy and natural as Thrift generates native APIs. http://curator.apache.org/curator-x-rpc/index.

RE: design thoughts: node TTLs

2015-04-13 Thread Hongchao Deng
Hi Jordan, I can share some thoughts on this. I think it's not easy to write client in other languages because the client connection is stateful. It contains information of watches, session, etc. Even if the server exposes RPC, REST, it's still a lot of work to write a client. I think making cli

RE: design thoughts: node TTLs

2015-04-13 Thread Hongchao Deng
@Flavio, Cool, thanks for the info. I will take a look at the JIRA. - Hongchao Deng > Date: Mon, 13 Apr 2015 10:33:40 + > From: fpjunque...@yahoo.com.INVALID > To: dev@zookeeper.apache.org > Subject: Re: design thoughts: node TTLs > > Yeah, I agree with Camille, there seems to be some sensiti

Re: design thoughts: node TTLs

2015-04-13 Thread Jordan Zimmerman
How does the current REST code handle watches and background notifications? Are clients required to poll periodically? This is really RPC and REST is not ideal for RPC. So, as I mentioned before, I think the ZK team should consider a real RPC system like Thrift instead of REST. -Jordan On Ap

[jira] [Commented] (ZOOKEEPER-2156) If JAVA_HOME is not set zk startup and fetching status command execution result misleads user.

2015-04-13 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492399#comment-14492399 ] Hadoop QA commented on ZOOKEEPER-2156: -- -1 overall. Here are the results of test

Failed: ZOOKEEPER-2156 PreCommit Build #2632

2015-04-13 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2156 Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2632/ ### ## LAST 60 LINES OF THE CONSOLE ### [

[jira] [Updated] (ZOOKEEPER-2156) If JAVA_HOME is not set zk startup and fetching status command execution result misleads user.

2015-04-13 Thread J.Andreina (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated ZOOKEEPER-2156: -- Attachment: ZOOKEEPER-2156.4.patch > If JAVA_HOME is not set zk startup and fetching status

[jira] [Commented] (ZOOKEEPER-2156) If JAVA_HOME is not set zk startup and fetching status command execution result misleads user.

2015-04-13 Thread J.Andreina (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492351#comment-14492351 ] J.Andreina commented on ZOOKEEPER-2156: --- Thanks [~rakeshr] for reviewing the pat

ZooKeeper-trunk - Build # 2659 - Failure

2015-04-13 Thread Apache Jenkins Server
See https://builds.apache.org/job/ZooKeeper-trunk/2659/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 363760 lines...] [exec] ZooKeeper JMX enabled by defau

ZooKeeper-trunk-jdk8 - Build # 351 - Still Failing

2015-04-13 Thread Apache Jenkins Server
See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/351/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 359884 lines...] [junit] 2015-04-13 11:44:20,794 [m

ZooKeeper_branch35_jdk7 - Build # 263 - Still Failing

2015-04-13 Thread Apache Jenkins Server
See https://builds.apache.org/job/ZooKeeper_branch35_jdk7/263/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 363127 lines...] [junit] 2015-04-13 10:57:46,378

Re: design thoughts: node TTLs

2015-04-13 Thread Flavio Junqueira
Yeah, I agree with Camille, there seems to be some sensitiveness around the topic, even though this thread seems to be a simple exercise of understanding what is out there.  Performing periodic verifications of the txn log CRCs enables one to detect latent errors earlier, but I'm not sure what y

ZooKeeper-trunk-solaris - Build # 1000 - Still Failing

2015-04-13 Thread Apache Jenkins Server
See https://builds.apache.org/job/ZooKeeper-trunk-solaris/1000/ ### ## LAST 60 LINES OF THE CONSOLE ### Started by timer Building remotely on solaris1 (Solaris) in works

ZooKeeper-trunk-WinVS2008 - Build # 1752 - Still Failing

2015-04-13 Thread Apache Jenkins Server
See https://builds.apache.org/job/ZooKeeper-trunk-WinVS2008/1752/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 108 lines...] [javacc] File "SimpleCharStream.