Yes, that sounds like a good change to me.
Alex On Sun, Dec 15, 2019 at 4:15 PM Aristedes Maniatis <[email protected]> wrote: > How about a new property: > > dynamicConfigHistory=3 > > which would preserve 3 historic config files. Or > > dynamicConfigHistory=0 > > which would keep none. > > > Does that sound like a reasonable approach? A default value of 0 would > be what most people expect, although it is a change for people already > wanting a folder full of files. > > I agree that dynamicConfigFile should point to the actual dynamic file, > not sometimes to the real file and sometimes to the prefix of the real > file. If there is any history worth keeping, then rolling them over log > style (either with timestamps or config id) is a much more understood > behaviour. > > > > Ari > > > > On 16/12/19 9:53am, Alexander Shraer wrote: > > I wasn't sure whether extracting such information from the log is simple, > > and since reconfigurations may impact the cluster in significant ways (or > > in the extreme bring it down completely) > > an easily accessible record seemed good to have, at least for debugging. > I > > agree that this can be made configurable, and would also not mind very > much > > not having a history at all, if others don't find it very useful. > > However this is a breaking change so probably requires more people to > chime > > in. > > > >> In case of some network issue, where a node repeatedly flaps, why would > > you want to fill the directory with possibly thousands of files? > > > > Automating reconfigations was not part of the release, only the basic > > mechanism was provided and not for example the policy of when you'd want > to > > reconfigure and what changes to do. > > But I agree that an automatic system like that should take care of this > > situation. > > > > > > Alex > > > > > > On Sun, Dec 15, 2019 at 2:26 PM Aristedes Maniatis <[email protected]> > wrote: > > > >> Can you explain a bit more about the use-case for when you'd want to > >> keep the history of the dynamic file. Surely the log file will contain > >> information about peers joining and leaving the cluster and is easier to > >> parse if you care about tracking that sort of thing. > >> > >> In case of some network issue, where a node repeatedly flaps, why would > >> you want to fill the directory with possibly thousands of files? > >> > >> > >> Ari > >> > >> > >> On 15/12/19 2:35pm, Alexander Shraer wrote: > >>> Hi Ari, > >>> > >>> Yes, you're totally right about the design goals. > >>> > >>> A mode where historic files aren't preserved could be useful. This > >>> could perhaps be added to the static config file as a parameter. > >>> > >>> Alternatively / in addition, maybe we could slightly change the way > >> history > >>> is staved. I don't really like the fact that we're actually using > >>> the file name to determine the version of the config (rather than > >>> information inside the file), this is used internally in ZK to decide > >> which > >>> config to use (the one with higher number wins). > >>> This method could fix this issue as well: > >>> - File name always stays the same, addressing your problem, and we > don't > >>> need to edit the static config file every time. > >>> - Dynamic config file contains the config version as a key. > >>> - Before overwriting the dynamic config file, we store a file with the > >>> previous config, including the version in the file name. > >>> > >>> This would change the current behavior a bit, hopefully no one is > relying > >>> on the file name to contain the version. > >>> > >>> This should not be difficult to implement, would you like to open a > Jira > >>> and take a stab at implementing it ? I can review it. > >>> > >>> Something to notice about the "version" of the config - currently when > >> the > >>> config is stored in memory, it appears as a key in the configuration. > >> When > >>> stored in the temporary config file (pre-commit), it appears as an > >> explicit > >>> key, but when committed it does not appear inside the dynamic file - > only > >>> in the file name.This is controlled by the last argument of > >>> QuorumPeerConfig.writeDynamicConfig. > >>> > >>> See also QuorumPeerConfig.java parse() parseProperties() etc and > >>> QuorumPeer.java setQuorumVerifier(). > >>> > >>> Thanks, > >>> Alex > >>> > >>> On Sat, Dec 14, 2019 at 6:32 PM Aristedes Maniatis <[email protected]> > >> wrote: > >>>> Will anything bad happen if I make the config file read-only for > >>>> zookeeper? I assume the design goals here were: > >>>> > >>>> * atomic rewrites of the dynamic config, preserving historic files > >>>> > >>>> * ability for zookeeper to know which was the most recent config file > on > >>>> restart > >>>> > >>>> > >>>> Those goals are a bit unnecessary for me. I don't really care about > >>>> historic configuration, so just writing to a temp file and moving over > >>>> the existing one would work great. Alternatively tracking the current > >>>> file in memory without rewriting the zoo.cfg would also be great, > since > >>>> I don't care about the effort on startup to rediscover peers. > >>>> > >>>> Is there a way to get Zookeeper to play better with not rewriting its > >>>> own config file for my use case? > >>>> > >>>> > >>>> Ari > >>>> > >>>> > >>>> On 12/12/19 5:53am, Alexander Shraer wrote: > >>>>> It will change, the number represents the version of the > configuration, > >>>> and > >>>>> will be updated if you issue a reconfiguration command. Its basically > >> the > >>>>> zxid of the command. > >>>>> > >>>>> > >>>>> Alex > >>>>> > >>>>> On Tue, Dec 10, 2019 at 11:25 PM Aristedes Maniatis <[email protected]> > >>>> wrote: > >>>>>> On 11/12/19 6:21pm, [email protected] wrote: > >>>>>>> Hey Ari, > >>>>>>> > >>>>>>> I directly used the filename zoo.cfg.dynamic.100000000 and > never > >>>>>> had a > >>>>>>> problem. > >>>>>>> Arne > >>>>>> Hmmm... that's an oddly obvious answer. I just assumed the 100000000 > >>>>>> would change randomly. What's even the point of it? > >>>>>> > >>>>>> > >>>>>> Ari > >>>>>> > >>>>>> >
