[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver11-1.patch Minor bug fix for count_hosts in case an empty string is passed in. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10-4.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver11-1.patch, > ZOOKEEPER-1355-ver11.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: (was: ZOOKEEPER-1355-ver11-1.patch) > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10-4.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver11-1.patch, > ZOOKEEPER-1355-ver11.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver11-1.patch Minor bugfix in count_hosts in case an empty string is passed in. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10-4.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver11-1.patch, > ZOOKEEPER-1355-ver11.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver11.patch This version fixes the core unit tests that were failing. All tests pass locally for me now. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10-4.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver11.patch, > ZOOKEEPER-1355-ver2.patch, ZOOKEEPER-1355-ver4.patch, > ZOOKEEPER-1355-ver5.patch, ZOOKEEPER-1355-ver6.patch, > ZOOKEEPER-1355-ver7.patch, ZOOKEEPER-1355-ver8.patch, > ZOOKEEPER-1355-ver9-1.patch, ZOOKEEPER-1355-ver9.patch, > ZOOKEEPER=1355-ver3.patch, ZOOOKEEPER-1355-test.patch, > ZOOOKEEPER-1355-ver1.patch, ZOOOKEEPER-1355.patch, > loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10-4.patch Uploading a new version that fixes the C code to properly force a disconnect if the list of servers is modified and the host we were connected to is not in the list anymore. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10-4.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10-4.patch Fixing mismerge on the last patch. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10.patch, > ZOOKEEPER-1355-ver2.patch, ZOOKEEPER-1355-ver4.patch, > ZOOKEEPER-1355-ver5.patch, ZOOKEEPER-1355-ver6.patch, > ZOOKEEPER-1355-ver7.patch, ZOOKEEPER-1355-ver8.patch, > ZOOKEEPER-1355-ver9-1.patch, ZOOKEEPER-1355-ver9.patch, > ZOOKEEPER=1355-ver3.patch, ZOOOKEEPER-1355-test.patch, > ZOOOKEEPER-1355-ver1.patch, ZOOOKEEPER-1355.patch, > loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10-3.patch Disabling the forced disconnect on server reconfig as it's causing grief on first connect and causing other tests to fail. Need to handle first connect more intelligently but don't have time to get to this today. Uploading a fixed version so unit tests will pass. Will upload an improved version tomorrow that deals with first connection properly. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10-2.patch Bug fix in C code. Need to update_addrs before checking if *fd == -1 else won't seek new connection after being forcibly disconnected due to server reconfig. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10.patch, > ZOOKEEPER-1355-ver2.patch, ZOOKEEPER-1355-ver4.patch, > ZOOKEEPER-1355-ver5.patch, ZOOKEEPER-1355-ver6.patch, > ZOOKEEPER-1355-ver7.patch, ZOOKEEPER-1355-ver8.patch, > ZOOKEEPER-1355-ver9-1.patch, ZOOKEEPER-1355-ver9.patch, > ZOOKEEPER=1355-ver3.patch, ZOOOKEEPER-1355-test.patch, > ZOOOKEEPER-1355-ver1.patch, ZOOOKEEPER-1355.patch, > loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10-1.patch Newer version has a couple fixes: 1. Force disconnect if server set reconfig causes the server we're connected to to no longer be in list. 2. More intelligent timeout if we've tried all the servers so we don't spin. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10-1.patch, > ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10.patch Uploading ZOOKEEPER-1355-ver10.patch. This includes all the code that was in ZOOKEEPER-1355-ver9-1.patch as well as all the C code to support this feature. > Add zk.updateServerList(newServerList) > --- > > Key: ZOOKEEPER-1355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 > Project: ZooKeeper > Issue Type: New Feature > Components: java client >Reporter: Alexander Shraer >Assignee: Alexander Shraer > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver2.patch, > ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, > ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, > ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, > ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, > ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, > ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf > > > When the set of servers changes, we would like to update the server list > stored by clients without restarting the clients. > Moreover, assuming that the number of clients per server is the same (in > expectation) in the old configuration (as guaranteed by the current list > shuffling for example), we would like to re-balance client connections across > the new set of servers in a way that a) the number of clients per server is > the same for all servers (in expectation) and b) there is no > excessive/unnecessary client migration. > It is simple to achieve (a) without (b) - just re-shuffle the new list of > servers at every client. But this would create unnecessary migration, which > we'd like to avoid. > We propose a simple probabilistic migration scheme that achieves (a) and (b) > - each client locally decides whether and where to migrate when the list of > servers changes. The attached document describes the scheme and shows an > evaluation of it in Zookeeper. We also implemented re-balancing through a > consistent-hashing scheme and show a comparison. We derived the probabilistic > migration rules from a simple formula that we can also provide, if someone's > interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1400) Allow logging via callback instead of raw FILE pointer
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1400: - Attachment: case-2739.patch > Allow logging via callback instead of raw FILE pointer > -- > > Key: ZOOKEEPER-1400 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1400 > Project: ZooKeeper > Issue Type: Improvement > Components: c client >Affects Versions: 3.4.2 > Environment: Linux >Reporter: Marshall McMullen > Fix For: 3.4.2 > > Attachments: case-2739.patch > > > The existing logging framework inside the C client uses a raw FILE*. Using a > FILE* is very limiting and potentially dangerous. A safer alternative is to > just provide a callback that the C client will call for each message. In our > environment, we saw some really nasty issues with multiple threads all > connecting to zookeeper via the C Client related to the use of a raw FILE*. > Specifically, if the FILE * is closed and that file descriptor is reused by > the kernel before the C client is notified then the C client will use it's > static global logStream pointer for subsequent logging messages. That FILE* > is now a loose cannon! In our environment, we saw zookeeper log messages > ending up in other sockets and even in our core data path. Clearly this is > dangerous. In our particular case, we'd omitted a call to > zoo_set_log_stream(NULL) to notify C client that the FILE* has been closed. > However, even with that bug fixed, there's still a race condition where log > messages in flight may be sent before the C client is notified of the FILE > closure, and the same problem can happen. > Other issues we've seen involved multiple threads, wherein one would close > the FILE*, and that's a global change that affects all threads connected > within that process. That's a pretty nasty limitation as well. > My proposed change is to allow setting a callback for log messages. A > callback is used in preference to a raw FILE*. If no callback is set, then it > will fallback to the existing FILE*. If that's not set, then it falls back to > stderr as it always has. > While refactoring this code, I removed the need for the double parens in all > the LOG macros as that wasn't necessary and didn't fit with my new approach. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-829) Add /zookeeper/sessions/* to allow inspection/manipulation of client sessions
[ https://issues.apache.org/jira/browse/ZOOKEEPER-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-829: Attachment: case-120.patch Revised patch to work with zookeeper 3.4.3. > Add /zookeeper/sessions/* to allow inspection/manipulation of client sessions > - > > Key: ZOOKEEPER-829 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-829 > Project: ZooKeeper > Issue Type: New Feature > Components: server >Reporter: Todd Lipcon > Attachments: 829.diff, case-120.patch > > > For some use cases in HBase (HBASE-1316 in particular) we'd like the ability > to forcible expire someone else's ZK session. Patrick and I discussed on IRC > and came up with an idea of creating nodes in /zookeeper/sessions/ id> that can be read in order to get basic stats about a session, and written > in order to manipulate one. The manipulation we need in HBase is the ability > to write a command like "kill", but others might be useful as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
