Al right, I'll try and go forward with that info. Thanks. On Wed, Aug 14, 2019 at 2:55 PM Norbert Kalmar <nkal...@cloudera.com.invalid> wrote:
> Read doesn't matter, there will be no txn logs for read. Only modifications > are logged (state changes to the datatree). > > As for why all txn logs are 64MB: > "preAllocSize > (Java system property: zookeeper.preAllocSize) > To avoid seeks ZooKeeper allocates space in the transaction log file in > blocks of preAllocSize kilobytes. The default block size is 64M. One reason > for changing the size of the blocks is to reduce the block size if > snapshots are taken more often. (Also, see snapCount)." > > Note that all txn is written to one log file. That's why it is 64MB by > default. > I didn't check which version of admin guide I quoted, but here is the one > from 3.4.13: > > snapCount > ZooKeeper records its transactions using snapshots and a transaction log > (think write-ahead log).The number of transactions recorded in the > transaction log before a snapshot can be taken (and the transaction log > rolled) is determined by snapCount. In order to prevent all of the machines > in the quorum from taking a snapshot at the same time, each ZooKeeper > server will take a snapshot when the number of transactions in the > transaction log reaches a runtime generated random value in the > [snapCount/2+1, snapCount] range.The default snapCount is 100,000. > > I'm not sure without checking the code what happens if preAllocSize is > filled (new log file I think), but after a restart, there will be a new log > file created. No way to continue the last one. That's why there could be > multiple log files, and no transaction. But if you haven't reached the > 100.000 transactions (writes that is, reads doesn't count), there will be > no snapshot. > > Regards, > Norbert > > On Wed, Aug 14, 2019 at 2:35 PM Koen De Groote < > koen.degro...@limecraft.com> > wrote: > > > >Without a snapshot, we cannot delete the log files, as we would have no > > >means of recovery. txn logs applied to the snapshot gives us back the > > >state. Without snapshot, all txn logs needs to be "replayed" in a > > recovery. > > >And you need all the log files created since your last snapshot (in this > > >case, all the txn logs as there were no snapshots yet). > > > > Makes sense. > > > > In the heaviest environment I'm hitting around 200 requests per second, > > which all have to get data from zookeeper. Not sure what the impact is in > > terms of snapCount, I didn't set up the system myself and don't fully > grasp > > the internals. > > > > As for the snapCount, that hasn't been touched, so that will be the > default > > in my environments. > > I've read that the log files are preallocated. I see them as being 65MB a > > piece. > > > > Which makes me wonder: how many of those until the process hits 100000 > > snaps? > > > > auto-cleaning is setup, retainCount=3, purgeInterval=1 > > > > Restarting the zookeeper process shouldn't affect this count, I think? It > > doesn't happen often, though it might on test environments. > > > > > > > > On Wed, Aug 14, 2019 at 2:06 PM Norbert Kalmar > > <nkal...@cloudera.com.invalid> > > wrote: > > > > > Hi, > > > > > > Without a snapshot, we cannot delete the log files, as we would have no > > > means of recovery. txn logs applied to the snapshot gives us back the > > > state. Without snapshot, all txn logs needs to be "replayed" in a > > recovery. > > > And you need all the log files created since your last snapshot (in > this > > > case, all the txn logs as there were no snapshots yet). > > > > > > As for why there is no snapshot. What is your load? Per the admin > guide: > > > > > > "snapCount > > > (Java system property: zookeeper.snapCount) > > > ZooKeeper logs transactions to a transaction log. After snapCount > > > transactions are written to a log file a snapshot is started and a new > > > transaction log file is created. The default snapCount is 100,000." > > > > > > By default there will be no auto-cleaning of the snapshot and log > files. > > > Check the autopurge.snapRetainCount and autopurge.purgeInterval > settings > > > for this. > > > > > > Regards, > > > Norbert > > > > > > On Wed, Aug 14, 2019 at 1:21 PM Koen De Groote < > > > koen.degro...@limecraft.com> > > > wrote: > > > > > > > Greetings all. > > > > > > > > I was debugging something an ran into this bit of code: > > > > > > > > > > > > > > https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/PurgeTxnLog.java#L81 > > > > > > > > If I understand it correctly, it seems that this means log.xxxxx > files > > > will > > > > only get deleted if there's a snapshot. > > > > > > > > Which is troublesome, as my dataDir is filling up with log files but > > not > > > a > > > > single snapshot in sight. > > > > > > > > 1: Is this correct behavior? Both the logic of needing a snapshot and > > the > > > > fact that not snapshots are being generated? > > > > > > > > 2: While not having a fix for this, what would be useful to know is: > > can > > > > these log files be freely deleted? Or does the most recent one need > to > > be > > > > kept, or how does it go with this files? > > > > > > > > I thought that running "bin/zkCleanup.sh /data -n 3" would clean up > > both > > > > the snapshots and the logs, but it appears that if there are not > > > snapshots, > > > > the logs aren't cleaned either. > > > > > > > > What are my options here? > > > > > > > > Kind regards, > > > > Koen De Groote > > > > > > > > > >
