Re: are ephemeral nodes removed when client receives session expiration

2017-02-09 Thread Michael Han
Hi Ryan, Learner will ask leader to validate the session when reopen a session, you can check LearnerZooKeeperServer->revalidateSession and also the Leader.REVALIDATE case handling in LearnerHandler. When a session is in closing state, the request from learner to leader on validating the session w

Re: are ephemeral nodes removed when client receives session expiration

2017-02-08 Thread Ryan Zhang
Sorry, I was wrong on the session not checked in the learner and mark the session closing only run on the leader part. However, it looks like the session on the learner (the second server) will not expire as it was a reopened session and as far as I see the session time out is partly set by the

Re: are ephemeral nodes removed when client receives session expiration

2017-02-08 Thread Ryan Zhang
I am sorry that I still didn’t get it. Let me post my scenarios again 1. Client A connected to Leader with session S 2. Session S expired on Leader, which according to the code below it will set the session as closing 3. Leader then send out a close session request to its own first processor

Re: are ephemeral nodes removed when client receives session expiration

2017-02-08 Thread Michael Han
Hi Ryan, >> From the code I saw, there is no quorum operation when a ZK reconnect with a session ID The "createSession(cnxn, passwd, sessionTimeout)" will submit a session create request that will go through processor pipeline, and that is where the quorum operation is involved. Also note that w

Re: are ephemeral nodes removed when client receives session expiration

2017-02-08 Thread Ryan Zhang
Hi, Michael, I am sorry that I spoke too soon. From the code I saw, there is no quorum operation when a ZK reconnect with a session ID (pasted the code below). Did I miss something again? long sessionId = connReq.getSessionId(); if (sessionId == 0) { LOG.info("Client attempting to establi

Re: are ephemeral nodes removed when client receives session expiration

2017-02-07 Thread Ryan Zhang
Thank you very much Michael, I am now clear. On Feb 7, 2017, at 3:21 PM, Michael Han mailto:h...@cloudera.com>> wrote: Hi Ryan, Thanks for the elaborations. I see why you are confused. Your case is valid, except in step 5: >> Client now knows its session expired but for some reason it tries

Re: are ephemeral nodes removed when client receives session expiration

2017-02-07 Thread Michael Han
Hi Ryan, Thanks for the elaborations. I see why you are confused. Your case is valid, except in step 5: >> Client now knows its session expired but for some reason it tries to connect to another server B and issues a read. At this point of time, there are two possible cases: * Client A's sessio

Re: are ephemeral nodes removed when client receives session expiration

2017-02-07 Thread Ryan Zhang
Hi, Michael, Thank you for your clarification. What you said seems to be consistent with what I saw in the code. However, I am still confused by your conclusion. My conclusion from the code is that it is totally possible for a client to get session expired and reconnect to another sever to still

Re: are ephemeral nodes removed when client receives session expiration

2017-02-07 Thread Michael Han
Hi Ryan, I am not sure what you were confused about regarding session cleaning up code. Here is my understanding, hope it helps. * Session clean up is started from marking the state of a session as closed, as you noticed. This is because each session clean up will take a while so we need make sur

Re: are ephemeral nodes removed when client receives session expiration

2017-01-26 Thread Michael Han
>> However, saying that ZK does not guarantee a consistent view isn't correct, the view of clients is always consistent (we guarantee sequential consistency), but they aren't necessarily the same and they don't necessarily reflect the latest committed state. Agree, thanks for clarification, Flavio

Re: are ephemeral nodes removed when client receives session expiration

2017-01-26 Thread Jun Rao
Flavio and others, Thanks for the reply. That's what I wanted to confirm. So, if a client receives a session expiration event and then establishes a new session, it shouldn't see any ephemeral nodes created by the expired session. Jun On Thu, Jan 26, 2017 at 1:39 AM, Flavio Junqueira wrote: >

Re: are ephemeral nodes removed when client receives session expiration

2017-01-26 Thread Flavio Junqueira
I actually meant to say "createSession ordered after closeSession" rather than "before". Sorry about the confusion. > because ZooKeeper does not necessarily guarantee a consistent view of the > data tree across all clients at any given time. I know what you mean when you say this, clients do

Re: are ephemeral nodes removed when client receives session expiration

2017-01-25 Thread Michael Han
>> If you ask whether the client will see its ephemerals upon creating a new session, then the answer is that it shouldn't because the createSession txn will be ordered necessarily before the closeSession txn, which implies that the client should not see the ephemerals. Second this - so *for the s

Re: are ephemeral nodes removed when client receives session expiration

2017-01-25 Thread Flavio Junqueira
Hey Jun, The way you're phrasing the question is problematic. Receiving a session expiration means that a closeSession has been committed, so a quorum has seen the closeSession event. However, a minority might still think the session is up, so if you ask whether the ephemerals are still there a

Re: are ephemeral nodes removed when client receives session expiration

2017-01-25 Thread Ryan Zhang
I am a bit confused by the code On Jan 25, 2017, at 1:33 PM, Michael Han mailto:h...@cloudera.com>> wrote: Does ZK guarantee that ephemeral nodes from a client are removed on the sever by the time the client receives a session expiration event? "the server" is a vague definition, as ZooKeeper e

Re: are ephemeral nodes removed when client receives session expiration

2017-01-25 Thread Michael Han
>> Does ZK guarantee that ephemeral nodes from a client are removed on the sever by the time the client receives a session expiration event? "the server" is a vague definition, as ZooKeeper ensemble is composed of multiple servers :). >> Therefore, it seems to be possible for a client to connect

Re: are ephemeral nodes removed when client receives session expiration

2017-01-25 Thread Ryan Zhang
Good question, AFAIK, it’s not the case. The server will throw an SessionExpiredException during checkSession call as soon as the session is marked as isClosing. However, session expiration actually requires a transaction (of type OpCode.closeSession) which will be send to the leader to go thro

are ephemeral nodes removed when client receives session expiration

2017-01-25 Thread Jun Rao
Hi, Does ZK guarantee that ephemeral nodes from a client are removed on the sever by the time the client receives a session expiration event? I am getting conflicting info on this ( https://issues.apache.org/jira/browse/KAFKA-4277). Could someone clarify? Thanks, Jun