Re: Write retry issues with ZooKeeperClient

2017-06-28 Thread Venkateswara Rao Jujjuri
Current retry logic is transparent to the type of the operation and is down the stack which has no understanding of the operation type. If we want to differentiate between idempotent and non-idempotent, we may need to insert that logic into the retry logic. Other option is to handle retry at higher

Re: Write retry issues with ZooKeeperClient

2017-06-28 Thread Sijie Guo
On Tue, Jun 27, 2017 at 4:27 PM, Sam Just wrote: > Should EXPIRED be considered a recoverable error for retry purposes? > Retrying in that case would mean that operations which might have been > submitted under the assumption that ephemeral nodes were still present > would be retried after the ep

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sam Just
Should EXPIRED be considered a recoverable error for retry purposes? Retrying in that case would mean that operations which might have been submitted under the assumption that ephemeral nodes were still present would be retried after the ephemeral nodes disappeared. Don't all users have special ha

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sijie Guo
On Tue, Jun 27, 2017 at 2:36 PM, Sam Just wrote: > On Tue, Jun 27, 2017 at 2:29 PM, Sijie Guo wrote: > > > On Tue, Jun 27, 2017 at 10:18 AM, Sam Just wrote: > > > > > JV: What do you mean by "May not be perfect for negative testing"? > > > > > > I don't think there's anything inevitable about t

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sam Just
On Tue, Jun 27, 2017 at 2:29 PM, Sijie Guo wrote: > On Tue, Jun 27, 2017 at 10:18 AM, Sam Just wrote: > > > JV: What do you mean by "May not be perfect for negative testing"? > > > > I don't think there's anything inevitable about this particular class of > > behavior. ZK could have chosen to a

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sijie Guo
On Tue, Jun 27, 2017 at 10:18 AM, Sam Just wrote: > JV: What do you mean by "May not be perfect for negative testing"? > > I don't think there's anything inevitable about this particular class of > behavior. ZK could have chosen to avoid this problem entirely by doing > duplicate op detection se

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sam Just
JV: What do you mean by "May not be perfect for negative testing"? I don't think there's anything inevitable about this particular class of behavior. ZK could have chosen to avoid this problem entirely by doing duplicate op detection server-side with a per-session transaction log. Since it doesn

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sijie Guo
I agree in current callback model, we only propagate error code to the outer client, so we lose the information about the detail cause. But I think we also tried to map some zk error codes to bk error codes. NoNode -> NoLedger NodeExists -> LedgerExists ยทยทยท (other codes) Also we tried to hide zk

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Sijie Guo
I agree. For create ledger, it is easy to handle - the zk ledger manager can catch such exception and recreate a new one. For create ledger adv, the ledger id is supplied by application, so the application is responsible for handle this ledgerexists exception. Thoughts? On Jun 27, 2017 6:45 AM, "

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Enrico Olivelli - Diennea
Il giorno mar, 27/06/2017 alle 13.45 +, Venkateswara Rao Jujjuri ha scritto: This is nothing different in any network based system. Like nfs. So we need to have some kind of logic on the client side to make reasonable assumption. May not be perfect for negative testing. Many times I wanted t

Re: Write retry issues with ZooKeeperClient

2017-06-27 Thread Venkateswara Rao Jujjuri
This is nothing different in any network based system. Like nfs. So we need to have some kind of logic on the client side to make reasonable assumption. May not be perfect for negative testing. JV On Mon, Jun 26, 2017 at 11:19 PM Sijie Guo wrote: > Hi Sam, > > Let's assume there is no such retr

Re: Write retry issues with ZooKeeperClient

2017-06-26 Thread Sijie Guo
Hi Sam, Let's assume there is no such retry logic. How do you expect to handle this situation? Can your application try to create a new ledger or catch NodeExists exception? - Sijie On Mon, Jun 26, 2017 at 5:49 PM, Sam Just wrote: > Hmm, curator seems to have essentially the same problem: > h

Re: Write retry issues with ZooKeeperClient

2017-06-26 Thread Sam Just
Hmm, curator seems to have essentially the same problem: https://issues.apache.org/jira/browse/CURATOR-268 I'm not sure there's a good way to solve this transparently -- the right answer is probably to plumb the ConnectionLoss event through ZooKeeperClient for interested callers, audit for metadata

Write retry issues with ZooKeeperClient

2017-06-26 Thread Sam Just
BookKeeper has a wrapper class for the ZooKeeper client called ZooKeeperClient. Its purpose appears to be to transparently perform retries in the case that ZooKeeper returns ConnectionLoss on an operation due to a Disconnect event. The trouble is that it's possible that a write which received a Co