Re: API to get a whole subtree

2023-10-11 Thread Ted Dunning
The last time I heard of a discussion along these lines, such an API was frowned upon a bit because it is susceptible to having a very large amount of returned data and thus being having a strong potential for causing disruption for other uses, particularly if the entire returned result has to be

Re: Name of sequence node is not unique

2023-06-17 Thread Ted Dunning
gt; > > > > > On Fri, Jun 16, 2023 at 11:33 AM Josef Roehrl > > > wrote: > > > > > > > > I wanted to add 2 things. > > > > > > > > > > > > 1. Increasing to a 64-bit counter certainly solves the problem, >

Re: Name of sequence node is not unique

2023-06-15 Thread Ted Dunning
things at less than 100,000 per second On Thu, Jun 15, 2023, 17:03 Li Wang wrote: > Thanks a lot for your inputs, Ted. > > On Thu, Jun 15, 2023 at 2:52 PM Ted Dunning wrote: > > > Breaking a semantic invariant is a dangerous solution here. > > Totally agree. We sho

Re: Name of sequence node is not unique

2023-06-15 Thread Ted Dunning
Breaking a semantic invariant is a dangerous solution here. It would be better to use a longer integer. On Thu, Jun 15, 2023, 13:35 Li Wang wrote: > Thanks for the response, Enrico. > > Please see comments below. > > On Thu, Jun 15, 2023 at 5:47 AM Enrico Olivelli > wrote: > > > Li, > >

Re: Switch to Reloadj4 on old but active branches 3.5, 3.6 and 3.7

2022-01-30 Thread Ted Dunning
Another approach is the bridge that the logging project just released. https://twitter.com/ted_dunning/status/1487876548384346115?s=20=xe0GsxjqJUHyw5LcJc_8DQ On Sun, Jan 30, 2022 at 1:13 AM Szalay-Bekő Máté wrote: > Thank you Enrico! > > I like the reload4j idea for all our older, but still

Re: Logback

2022-01-18 Thread Ted Dunning
I believe that the primary contributor to logback was highly skeptical that the recent problems could possible affect logback. That isn't a good attitude for security problems. It isn't just a matter of patch rate. There is also the question of community size. Is logback effectively a one-man

Re: Logback

2022-01-10 Thread Ted Dunning
ionally we keep log4j1 still an option as the > backend. > > Apologies I didn’t have time to take a look at slf4j-simple for our tests > yet, but looks like this option has already got support from multiple folks > in the community, so worth a shot. > > Thanks > > Andor >

Re: Logback

2022-01-07 Thread Ted Dunning
I have been watching the private and public mailing lists for Apache Logging as part of $dayjob as well. I read the mood there differently. The most recent comment I remember was a confirmation that "no bugfixes or security patches are planned for log4j1". Log4j2 really is much larger than

Re: CVE-2021-37136/37137

2021-09-12 Thread Ted Dunning
Thanks for the contribution! Btw... this tiny update is probably fine, but if you ever want to contribute something more substantial, we will need to know who you are. Your account name and signature give no real hint and we need to know where the code in an Apache project comes from. As I

Re: [VOTE] Apache ZooKeeper release 3.6.3 candidate 2

2021-06-10 Thread Ted Dunning
Justin, Thanks for the feedback! On Thu, Jun 10, 2021 at 8:23 PM Justin Mclean wrote: > HI, > > I'm not on your PMC but I took a look at your source release and I think > there a few things that need to be added to the LICENSE file as per the > guidelines here [1] > > These BSD licensed

Re: Performance impact of enabling Prometheus Metrics

2021-05-03 Thread Ted Dunning
On Mon, May 3, 2021 at 6:50 PM Li Wang wrote: > ... > > Please let me know if you have any thoughts or questions. > My thought is that you have done some nice work.

Re: Performance impact of enabling Prometheus Metrics

2021-04-26 Thread Ted Dunning
n a separate thread, > > so it doesn't impact the "main" thread. > > > > Not sure about the idea of merging at the end of a reporting period. Can > > you elaborate a bit on it? > > > > Thanks, > > > > Li > > > > On Mon, Apr 26, 2021 a

Re: Performance impact of enabling Prometheus Metrics

2021-04-26 Thread Ted Dunning
ad. > > Not sure about the idea of merging at the end of a reporting period. Can > you elaborate a bit on it? > > Thanks, > > Li > > On Mon, Apr 26, 2021 at 9:11 PM Ted Dunning wrote: > > > Would it help to keep per thread metrics that are either reported >

Re: Performance impact of enabling Prometheus Metrics

2021-04-26 Thread Ted Dunning
Would it help to keep per thread metrics that are either reported independently or are merged at the end of a reporting period? On Mon, Apr 26, 2021 at 8:51 PM Li Wang wrote: > Hi Community, > > I've done further investigation on the issue and found the following > > 1. The perf of the read

Re: [RESULT] [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

2021-03-27 Thread Ted Dunning
which are binding: > > - Szalay-Bekő Máté > - Norbert Kalmar > - Patrick Hunt (binding) > - Mohammad arshad > - Enrico Olivelli (binding) > - Rakesh Radhakrishnan (binding) > > There are no disapproving votes. Ted Dunning reported some issues he > encountered while te

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

2021-03-25 Thread Ted Dunning
ld expect the current test suite to trip on weird corner cases on > networked filesystems such as NFS, let alone something more exotic. Can > you confirm that the above is not a "vanilla" filesystem? If so, that > would explain the failure. > > (Not saying that aspect

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

2021-03-24 Thread Ted Dunning
Repeated tests. Similar, but not identical failures. On Wed, Mar 24, 2021 at 4:09 PM Ted Dunning wrote: > > I had these test failures. That doesn't look good. > > I haven't been keeping track, however, and am not sure that they are > problems: > > [*ERROR*] *Fa

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

2021-03-24 Thread Ted Dunning
st number of cores On Wed, Mar 24, 2021 at 3:10 PM Ted Dunning wrote: > > I am starting tests now. > > I had an issue (self-inflicted) when I tried to run two tests at the same > time. > > > > On Wed, Mar 24, 2021 at 1:22 PM Damien Diederen > wrote: > >>

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

2021-03-24 Thread Ted Dunning
I am starting tests now. I had an issue (self-inflicted) when I tried to run two tests at the same time. On Wed, Mar 24, 2021 at 1:22 PM Damien Diederen wrote: > > Dear all, > > Thank you for the reviews and testing. > > Glad that we have not met yet another snag so far! > > I have counted 5

Re: ConnectionLoss Upon Length Error - Failed to Write > 1 MB

2021-01-08 Thread Ted Dunning
1)] WARN > > > > > org.apache.zookeeper.ClientCnxn - Session 0x1003e7613ab0005 for > sever > > > > > localhost:2181, Closing socket connection. Attempting reconnect > > except > > > > > it is a SessionExpiredException. > > > > > ja

Re: ConnectionLoss Upon Length Error - Failed to Write > 1 MB

2021-01-08 Thread Ted Dunning
Let's be clear from the start, storing large data objects in Zookeeper is strongly discouraged. If you want to store large objects with good consistency models, store the data in something else (like a distributed file system or key value store), commit the data and then use ZK to provide a

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

2021-01-06 Thread Ted Dunning
Also, software like ZK should lag rather than lead (I realize that moving to 11 is hardly leading). There is considerable pain that we inflict when we move JDK requirements forward. On Wed, Jan 6, 2021 at 10:20 AM Enrico Olivelli wrote: > In my opinion we can stay on 8, we should change this

Re: Time to Cut 3.7.0 ?

2021-01-05 Thread Ted Dunning
Sorry about not noticing the open vote. (year end, time off and such) IN any case, I am very busy right now. I may or may not have time to look at the release. On Tue, Jan 5, 2021 at 3:28 AM Enrico Olivelli wrote: > Hi ZooKeepers, > I feel it is time to deliver 3.7.0 to the public. > > Any

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

2020-10-22 Thread Ted Dunning
Allesandro, Would your answer be different if you considered a strategy that allows JRE8 to run ZK but requires JDK11 to build it? On Thu, Oct 22, 2020 at 6:36 AM Szalay-Bekő Máté wrote: > Hello All, > > A few reflections: > > - I don't think that backporting fixes from a JDK 11 only version

Re: [VOTE] Apache ZooKeeper 3.6.2 candidate 1

2020-09-04 Thread Ted Dunning
I did the following (minimal) checks: * downloaded binary distro * started single node ZK cluster * verified simple operations in zkCli * verified no lag in typing on OSX using JDK 12 * verified checksum and signature * downloaded source distro * verified that compile and packaging succeeded *

Re: [VOTE] Apache ZooKeeper release 3.6.2 candidate 0

2020-09-03 Thread Ted Dunning
On Thu, Sep 3, 2020 at 1:58 PM Patrick Hunt wrote: > On Thu, Sep 3, 2020 at 1:54 PM Ted Dunning wrote: > > > OK. Did it with the correct version this time. I saw no typing delays in > > zkCli.sh. > ... > Hm, no idea - I tried the regular mac terminal (I use iterm2) an

Re: [VOTE] Apache ZooKeeper release 3.6.2 candidate 0

2020-09-03 Thread Ted Dunning
OK. Did it with the correct version this time. I saw no typing delays in zkCli.sh. $ git checkout release-3.6.2-0 $ On Thu, Sep 3, 2020 at 12:28 PM Ted Dunning wrote: > > Argh. > > Will check again. > > > > On Thu, Sep 3, 2020 at 11:10 AM Enrico Olivelli > wr

Re: [VOTE] Apache ZooKeeper release 3.6.2 candidate 0

2020-09-03 Thread Ted Dunning
Argh. Will check again. On Thu, Sep 3, 2020 at 11:10 AM Enrico Olivelli wrote: > Ted > The branch is release-3.6.2 and not 3.6.1 > Thanks foe testing > Enrico > > Il Gio 3 Set 2020, 19:50 Ted Dunning ha scritto: > > > On Thu, Sep 3, 2020 at

Re: [VOTE] Apache ZooKeeper release 3.6.2 candidate 0

2020-09-03 Thread Ted Dunning
On Thu, Sep 3, 2020 at 9:23 AM Patrick Hunt wrote: > ... > > > Il giorno mer 2 set 2020 alle ore 22:03 Patrick Hunt > > > > ha scritto: > > > > > >> I'm also noticing a very long lag when using the cli > > > > I am not experiencing this problem, I am on Mac, tested on branch-3.6 and > >

Re: Re: Sequential Consistency Guarantees

2020-09-01 Thread Ted Dunning
Stanislav, A ZK client should only ever see state move forward in terms of ZXID. This is true even if a client disconnects and reconnects. If you have multiple connections to a ZK cluster, each connection will satisfy this constraint with respect to itself, but one could be behind the other.

Re: Sequential Consistency Guarantees

2020-08-27 Thread Ted Dunning
Can you describe your evidence in more detail as you understand it? On Thu, Aug 27, 2020 at 10:26 AM Stanislav Kozlovski wrote: > Hello! > > We were investigating an issue with consistent reads in ZK which made us > question our assumptions. > We believe that ZK offers sequential consistency

Re: Sequential Consistency Guarantees

2020-08-27 Thread Ted Dunning
I think that the answer depends a bit on what you mean by "client". If you mean client connection (as in the result of opening a connection to a Zookeeper cluster), then you should see things in order so that reads following a successful and acknowledged commit should see only states at or after

Re: May violate the ZAB agreement -- version 3.6.1

2020-08-27 Thread Ted Dunning
How is it that participant A would have a later zxid than the leader? In particular, it seems to me that it should be impossible to have these two facts be true: 1) a transaction has been committed with zxid = z_0. This implies that a quorum of the cluster has accepted this transaction and it

Re: racing risk in MultiTxn's serialize

2020-08-19 Thread Ted Dunning
No significant changes should ever be made to any transaction as it passes through the pipeline. The code you point out is the creation of the multi. The mutation is in the same block that created it and no references escape before the endVector call is made. I request again. Can you please stop

Re: racing risk in MultiTxn's serialize

2020-08-18 Thread Ted Dunning
Why do you think that this code is a problem? The instance in question is limited to a single thread. On Tue, Aug 18, 2020 at 10:27 AM happen <370119...@qq.com> wrote: > Hi > > > I think may be there is a little risk when running > MultiTxn's serialize txns.get(vidx1) step. Because probably

Re: racing risk in MultiTxn's serialize

2020-08-18 Thread Ted Dunning
Hey there, THanks for your email, but could you format your messages without HTML? It is very hard to read and understand what you are saying otherwise. On Tue, Aug 18, 2020 at 10:27 AM happen <370119...@qq.com> wrote: > Hi > > > I think may be there is a little risk when running >

Re: Creating a public fork of ZooKeeper

2020-08-13 Thread Ted Dunning
It is totally fine to release modified versions of Zookeeper. Just don't call it Zookeeper. You can (and likely should) say that it is based on (or powered by) Zookeeper, but don't claim it is the same thing if it isn't. But are you sure that you wouldn't really rather contribute directly to

Re: ZK Transaction API multi()

2020-07-28 Thread Ted Dunning
we will move the excessive number of writes out of ZK >> to >> > achieve higher scalability. >> > >> > Lastly, I greatly appreciate your insightful explanation! FYI, I am very >> > happy to receive prompt replies from you, Ted! >> > >> > Best,

Re: ZK Transaction API multi()

2020-07-27 Thread Ted Dunning
This sounds like you are using ZK outside of the intended design. The idea is that ZK is a coordination engine. If you have such high write rates that ZK is dropping connections, you probably want a distributed database of some kind, perhaps one that uses ZK to coordinate itself. ZK is a form of

Re: ZK Transaction API multi()

2020-07-27 Thread Ted Dunning
On Mon, Jul 27, 2020 at 1:40 PM Huizhi Lu wrote: > Hi Zookeeper Devs, > > Hope this email finds you well! > > I am working on some stuff that needs ZK multi(). I would like to confirm a > few things about this API. > > 1. Is this a real transaction operation in ZK? My understanding is, it is a >

Re: How to make a quorum of 40million znodes be in serving status after leader shut down

2020-07-24 Thread Ted Dunning
That sounds like you are using ZK in a way that is more appropriate for a database. Particularly if you are putting watches on a large fraction of those 40 million znodes. ZK is not intended to scale up or out. It is intended to serve as a coordination layer for other systems that do. As such

Re: Handle Java 9/11 additions of covariant return types to java.nio.ByteBuffer methods

2020-04-27 Thread Ted Dunning
Maybe we could write a preprocessor with macros so that we could automagically insert different code for different java versions! (in case you didn't notice, this is a joke suggestion) On Mon, Apr 27, 2020 at 10:16 AM Enrico Olivelli wrote: > Optional 2 is the best. > > I have fallen into

Re: Implement checksum against malformed packets

2020-04-08 Thread Ted Dunning
The only way to corrupt a message transmitted on an encrypted link is to break the encryption. There is a tiny argument to be made that messages should be checksummed for protection even without the encryption, but these links should always be encrypted. The chance of a 160 bit checksum being

Re: ZooKeeper snapCount Tuning

2020-04-03 Thread Ted Dunning
On Fri, Apr 3, 2020 at 10:01 AM Patrick Hunt wrote: > ... > Makes sense. For eg. SSD characteristics are vastly diff from spinning > media. super true. > I suspect it would be worth looking into this in even more depth - > we pre-allocate certain files, perhaps that's no longer necessary,

Re: ZooKeeper 3.6.1...is it time ?

2020-03-19 Thread Ted Dunning
What internal class? Where is it used? Why are others using JDK 13 not seeing the problem that you described? On Thu, Mar 19, 2020 at 8:58 PM Christopher wrote: > Apologies, but it's a bit hard for me to explain briefly. Here's an attempt > to be as brief as I can: > > Newer JDKs add a

Re: Looking for go binding

2020-01-31 Thread Ted Dunning
Cool. I should be more trusting of experienced folks. Thanks for the quick response. On Fri, Jan 31, 2020 at 12:14 PM Jordan Zimmerman < jor...@jordanzimmerman.com> wrote: > No design discussions there. > > > Jordan Zimmerman > > > On Jan 31, 2020, at 1:16 PM, Giorgio

Re: Looking for go binding

2020-01-31 Thread Ted Dunning
I hope that design decisions aren't being taken in that slack channel. It is important to keep email lists as the design decision channel to allow people in other locations to participate effectively. Simply reporting decisions back after they are baked is not sufficient. On Fri, Jan 31, 2020

Re: Looking for go binding

2020-01-31 Thread Ted Dunning
There is some early work around a grpc binding for Zookeeper. That would make a high quality go binding fairly trivial. On Fri, Jan 31, 2020 at 10:16 AM Giorgio Zoppi wrote: > Hello smart people, > we are looking for binding in go, possibly with grpc communication. > Is there any working/high

Re: ZooKeeper Meetup @ Facebook is 1 Week Away! (January 29th 2020)

2020-01-28 Thread Ted Dunning
ks, > > Jeelani > > On 1/22/20, 12:57 PM, "Ted Dunning" wrote: > > Mohamed, > > Could you please answer the question about whether there will be an NDA > required at the door? > > > > On Wed, Jan 22, 2020 at 12:51 PM Mohamed Jeelan

Re: RC failure root cause: ICMP throttling settings on mac

2020-01-23 Thread Ted Dunning
uld be unfortunate if there > were an artificial cap on the number of nodes in a machine-local test > cluster, especially if it were related to an ICMP storm scenario. > > On Thu, Jan 23, 2020, 8:11 AM Ted Dunning wrote: > > > I think that this is far outside the normal operation

Re: RC failure root cause: ICMP throttling settings on mac

2020-01-23 Thread Ted Dunning
I think that this is far outside the normal operation bounds and has an easy work-around. First, it is very uncommon to run more than 5 ZK nodes. Running 23 on a single host is bizarre (viewed from an operational lens). Second, there is a simple configuration change that makes the strange

Re: ZooKeeper Meetup @ Facebook is 1 Week Away! (January 29th 2020)

2020-01-22 Thread Ted Dunning
Mohamed, Could you please answer the question about whether there will be an NDA required at the door? On Wed, Jan 22, 2020 at 12:51 PM Mohamed Jeelani wrote: > ZooKeeper Meetup @ Facebook is just 1 week away! (January 29th 2020) - > Please RSVP if you haven’t yet at -

Re: ZooKeeper Meetup @ Facebook - Jan 29th

2020-01-16 Thread Ted Dunning
Mohamed, I still didn't get an answer about what level of NDA will be required by FB in order to attend this event. Without that information, I can't come. On Fri, Jan 10, 2020 at 10:01 AM Mohamed Jeelani wrote: > Lets Meetup!  (Please Register at https://zkmeetup2020.splashthat.com/) > >

Re: ZooKeeper Meetup @ Facebook - Jan 29th

2020-01-10 Thread Ted Dunning
This is a great thing. One question that I have is whether or not FB will require the fairly intrusive NDA to be signed by all attendees. Years ago, that was a real problem with meetups at FB. Hopefully it isn't a problem any more. On Fri, Jan 10, 2020 at 10:01 AM Mohamed Jeelani wrote: >

Re:

2019-10-28 Thread Ted Dunning
Nice work on this root cause analysis!! On Sun, Oct 27, 2019, 6:07 PM Fangmin Lv wrote: > Hi everyone, > > Sharing the following informations in ZK dev as well for more visibility. > > Spent most of the time in the last few weeks on investigating the root > cause, sorry if I haven't responed

Re: thoughts about extension to multi semantics

2019-08-17 Thread Ted Dunning
On Sat, Aug 17, 2019 at 4:01 PM Jordan Zimmerman wrote: > > > ... > > I don't understand that. Watches can be set in a multi. > > Not in the public API: > https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/Op.java > - is it supported in the

Re: thoughts about extension to multi semantics

2019-08-17 Thread Ted Dunning
On Sat, Aug 17, 2019 at 10:19 AM Jordan Zimmerman < jor...@jordanzimmerman.com> wrote: > Some thoughts: > > It doesn't really help with any of the "standard" recipes as they all need > to set watches. I don't understand that. Watches can be set in a multi. > Not to open a can of worms, but if

Re: thoughts about extension to multi semantics

2019-08-17 Thread Ted Dunning
On Sat, Aug 17, 2019, 8:33 AM Michael Han wrote: > >> I would guess that it actually gets very, very little > > Can't speak for others, but for the zookeeper clusters I maintain > internally, multi was used extensively in certain use cases. > Cool! Very glad to hear it. >

Re: thoughts about extension to multi semantics

2019-08-17 Thread Ted Dunning
Being lazy, I would suggest only the tests we already have. Existence and version. On Sat, Aug 17, 2019, 5:30 AM Enrico Olivelli wrote: > Il sab 17 ago 2019, 08:01 Ted Dunning ha scritto: > > > It definitely sounds like a nice feature. > > > > The important questi

Re: thoughts about extension to multi semantics

2019-08-17 Thread Ted Dunning
where > the state is stored in a fine grained approach across multiple zNode, > instead of in a single zNode. > > On Fri, Aug 16, 2019 at 11:52 AM Ted Dunning > wrote: > > > The recent discussion about if/then/else idioms in ZK has raised the > > thought that it

thoughts about extension to multi semantics

2019-08-16 Thread Ted Dunning
The recent discussion about if/then/else idioms in ZK has raised the thought that it might be nice to have some extended semantics. One version that I could see would be to to extend the current multi-op to allow multiple alternatives. The idea would be that there would effectively be multiple

Re: [jira] [Updated] (ZOOKEEPER-1112) Add support for C client for SASL authentication

2019-08-14 Thread Ted Dunning
Kudos to you for this! On Wed, Aug 14, 2019 at 10:55 AM Damien Diederen wrote: > > Greetings, all, > > Note sure how notifications work on this project, but in case it did not > pop up on your radar: I have picked up ZOOKEEPER-1112, which was dormant > since 2011, and pushed a first set of

Re: Code to clean up transaction logs needs snapshots before it works?

2019-08-14 Thread Ted Dunning
On Wed, Aug 14, 2019 at 5:35 AM Koen De Groote wrote: > ... > I've read that the log files are preallocated. I see them as being 65MB a > piece. > Yes. Preallocation of logs is an important performance trick. The point is that if a file doesn't change length when you write to it, then the file

Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-13 Thread Ted Dunning
On Tue, Aug 13, 2019 at 10:41 AM Andor Molnar wrote: > Hmmm... and what should Kafka do if it wants to see all events? Use > RabbitMQ? :) Kafka should not be using ZK to store messages. And synchronization of configuration doesn't require message passing. Read the original ZK paper [1] to

Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-13 Thread Ted Dunning
If you want to see all events, use Kafka. On Tue, Aug 13, 2019 at 8:20 AM Jordan Zimmerman wrote: > Also see https://issues.apache.org/jira/browse/ZOOKEEPER-1416 < > https://issues.apache.org/jira/browse/ZOOKEEPER-1416> > > There are many use cases where a client wants to see all events from

Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-08 Thread Ted Dunning
On Wed, Aug 7, 2019 at 5:55 PM Michael Han wrote: > Related discussions: > https://lists.apache.org/list.html?d...@kafka.apache.org:lte=1M:KIP-500 > > >> Why would this lead to any discrepancy? It seems to me that the > controller, will read an even newer state in such a scenario. > > I think in

Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-07 Thread Ted Dunning
On Wed, Aug 7, 2019 at 11:31 AM Karolos Antoniadis wrote: > In the paragraph that Michael mentioned, among others it is written: "For > example, when a partition leader changes its ISR in ZK, the controller will > typically not learn about these changes for many seconds." Why would it > take

Re: Wrongly written characters in Zookeeper r3.5.5's programmer's guide

2019-08-07 Thread Ted Dunning
Yes. That is a typo. They meant to write entity On Wed, Aug 7, 2019 at 7:36 AM sun wrote: > Hi > My English is very poor, I’m not sure is there a wrongly written > characters in the bottom of ‘Znodes’ article? > link: https://zookeeper.apache.org/doc/r3.5.5/zookeeperProgrammers.html > > > >

Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-02 Thread Ted Dunning
The core issue in these situations in my experience is that having the quorum as a separate service can be a pain point. This misunderstanding about how watches work and why they don't provide the data is just a symptom of this. Having an integrated quorum is very attractive from the point of view

Re: [DISCUSS] Should we move contrib and recipes into separate repos?

2019-05-22 Thread Ted Dunning
I think that the key question is whether independent releases make sense. If you have two things that must always be released in synchrony, then having them in the same repo is a good idea. If you have two things that are released completely independently, then separate repos make sense. On

Re: Crypto Policy (was: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 5)

2019-04-29 Thread Ted Dunning
Other changes in u211+ substantially improve how Java 8 applications behave in containers. I am seeing this more and more with customers. Combined with the crypto issues, it might be worth a release note suggesting that if you are going to compile with Java 1.8, you should use a recent release at

Re: Crypto Policy (was: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 5)

2019-04-29 Thread Ted Dunning
See https://royvanrijn.com/blog/2018/05/java-and-docker-memory-limits/ for more info on the problem I was referring to. On Mon, Apr 29, 2019, 12:36 PM Ted Dunning wrote: > Other changes in u211+ substantially improve how Java 8 applications > behave in containers. I am seeing thi

Re: Question about the leader-based atomic broadcast

2019-01-09 Thread Ted Dunning
All updates to data in Zookeeper are modified to be idempotent before they are accepted into the leader's queue. That means that items in the queue can be committed in groups and once each group is acknowledged by a quorum of servers, it can be deleted from the queue. Any server not in the

Re: Maven migration and CI

2019-01-03 Thread Ted Dunning
Wow. This has been a long time coming! Great news. On Thu, Jan 3, 2019 at 1:34 PM Enrico Olivelli wrote: > Hi Zookeepers > Now that we have Maven it is time to set up new CI jobs and compare the > results with the Ant based build. > > My proposal is to set up a few jobs and see the results.

Re: Leader election

2018-12-10 Thread Ted Dunning
. On Mon, Dec 10, 2018 at 10:32 PM Michael Borokhovich wrote: > Makes sense. Thanks, Ted. We will design our system to cope with the short > periods where we might have two leaders. > > On Thu, Dec 6, 2018 at 11:03 PM Ted Dunning wrote: > > > ZK is able to guarantee that the

Re: Leader election

2018-12-06 Thread Ted Dunning
ZK is able to guarantee that there is only one leader for the purposes of updating ZK data. That is because all commits have to originate with the current quorum leader and then be acknowledged by a quorum of the current cluster. IF the leader can't get enough acks, then it has de facto lost

[jira] [Commented] (ZOOKEEPER-3188) Improve resilience to network

2018-12-05 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710015#comment-16710015 ] Ted Dunning commented on ZOOKEEPER-3188: Taking the questions in order: bq. Did we consider

[jira] [Comment Edited] (ZOOKEEPER-3188) Improve resilience to network

2018-12-05 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710015#comment-16710015 ] Ted Dunning edited comment on ZOOKEEPER-3188 at 12/5/18 12:37 PM

Re: question about approach to take

2018-11-13 Thread Ted Dunning
for ZooKeeper to eventually have, but I > don't see why it must make 3.4 and 3.5 while other features are pushed out > to 3.6. > Like any other feature, it would be subject to a vote and isn't a > unilateral decision. > > On Tue, Nov 13, 2018 at 1:59 PM Ted Dunning wrote: > &

Re: question about approach to take

2018-11-13 Thread Ted Dunning
gt; > > Thanks, > Alex > > > > On Mon, Nov 12, 2018 at 11:47 PM Ted Dunning > wrote: > > > There is a JIRA live for the network resilience feature that I mentioned > > previously. > > > > The design document > > < > > > https://doc

question about approach to take

2018-11-12 Thread Ted Dunning
There is a JIRA live for the network resilience feature that I mentioned previously. The design document (also copied into the JIRA) has essentially converged except for two points. These include:

[jira] [Commented] (ZOOKEEPER-3189) Support new configuration syntax for resilient network feature

2018-11-12 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684812#comment-16684812 ] Ted Dunning commented on ZOOKEEPER-3189: Supporting any syntax requires having the semantics

[jira] [Created] (ZOOKEEPER-3189) Support new configuration syntax for resilient network feature

2018-11-12 Thread Ted Dunning (JIRA)
Ted Dunning created ZOOKEEPER-3189: -- Summary: Support new configuration syntax for resilient network feature Key: ZOOKEEPER-3189 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3189 Project

[jira] [Created] (ZOOKEEPER-3188) Improve resilience to network

2018-11-12 Thread Ted Dunning (JIRA)
Ted Dunning created ZOOKEEPER-3188: -- Summary: Improve resilience to network Key: ZOOKEEPER-3188 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3188 Project: ZooKeeper Issue Type: Bug

Re: improving tolerance to network failures

2018-10-23 Thread Ted Dunning
t be to assign weights to each > address (a single server has weight one), and this will reduce to a > weighted random selection problem. > > Overall, I think this proposal has little impact on server side, most > impact is on client side. > > > On Tue, Oct 23, 2018 at 9:34 AM Ted Dunning wrote

Re: improving tolerance to network failures

2018-10-23 Thread Ted Dunning
in production situations? On 2018/10/23 02:02:12, Ted Dunning wrote: > ... > I have started a collaborative document to work on the design approach. > Once that is judged by the community to be sufficiently mature, I will move > it to a JIRA. > > That document is at > https://docs.

improving tolerance to network failures

2018-10-22 Thread Ted Dunning
I am starting work on a project to improve the tolerance of Zookeeper to network failures and would like feedback on the idea. The problem is that with environments where link bonding is forbidden (they exist, trust me), Zookeeper is sensitive to the loss of a single switch or a few network

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-10-25 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218937#comment-16218937 ] Ted Dunning commented on ZOOKEEPER-2770: There is a pull request just now that has 95 files

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-26 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102749#comment-16102749 ] Ted Dunning commented on ZOOKEEPER-2770: The typical approach is to set a limit on number

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-25 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101161#comment-16101161 ] Ted Dunning commented on ZOOKEEPER-2770: Btw I note that there is no metering

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-25 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101158#comment-16101158 ] Ted Dunning commented on ZOOKEEPER-2770: {quote} With that said, is 300 ms a good value

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-25 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100945#comment-16100945 ] Ted Dunning commented on ZOOKEEPER-2770: On second thought, I could imagine that startup

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-25 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100942#comment-16100942 ] Ted Dunning commented on ZOOKEEPER-2770: To put some color on Camille's surprise, I would

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-25 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100603#comment-16100603 ] Ted Dunning commented on ZOOKEEPER-2770: [~fournc], I am not so sure that *I* agree with me

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-10 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081514#comment-16081514 ] Ted Dunning commented on ZOOKEEPER-2770: Who are we? This kind of feature needs

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-06 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077254#comment-16077254 ] Ted Dunning commented on ZOOKEEPER-2770: Would it be a good idea to extend the idea

[jira] [Commented] (ZOOKEEPER-2770) ZooKeeper slow operation log

2017-07-06 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077221#comment-16077221 ] Ted Dunning commented on ZOOKEEPER-2770: I don't see any discussion of this on the mailing

[jira] [Updated] (ZOOKEEPER-2509) Secure mode leaks memory

2016-08-11 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Dunning updated ZOOKEEPER-2509: --- Attachment: 0001-Updated-patch-for-Netty-leak-testing-to-trunk.patch Updated patch (which

[jira] [Commented] (ZOOKEEPER-2509) Secure mode leaks memory

2016-08-10 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416280#comment-15416280 ] Ted Dunning commented on ZOOKEEPER-2509: [~rgs] Thanks for the pointer. I got some hints from

[jira] [Updated] (ZOOKEEPER-2509) Secure mode leaks memory

2016-08-10 Thread Ted Dunning (JIRA)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Dunning updated ZOOKEEPER-2509: --- Attachment: leak-patch.patch This is a fix for the problem along with a unit test

  1   2   3   4   >