Re: zookeeper snapshot files eat up disk space
http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html#Ongoing+Data+Directory+Cleanup /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Tue, Nov 11, 2014 at 9:24 AM, Shlomi Hazan shl...@viber.com wrote: Hi, My zookeeper 'dataLogDir' is eating up my disk with tons of snapshot files. what are these files? what files can I delete? are week old files disposable? This folder only gets bigger... How can I avoid blowing my disk? Thanks, Shlomi
Re: zookeeper snapshot files eat up disk space
That looks like a complete answer. BUT just to be sure: it says Automatic purging of the snapshots and corresponding transaction logs was introduced in version 3.4.0. using 0.8.1.1 means that I will have to purge manually, right? Is there some convention for kafka users? e.g.: delete all but last X=3 maybe? Shlomi On Tue, Nov 11, 2014 at 4:27 PM, Joe Stein joe.st...@stealth.ly wrote: http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html#Ongoing+Data+Directory+Cleanup /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Tue, Nov 11, 2014 at 9:24 AM, Shlomi Hazan shl...@viber.com wrote: Hi, My zookeeper 'dataLogDir' is eating up my disk with tons of snapshot files. what are these files? what files can I delete? are week old files disposable? This folder only gets bigger... How can I avoid blowing my disk? Thanks, Shlomi
Re: zookeeper snapshot files eat up disk space
There is a sidecar jvm project called exhibitor that manages this for you. It's from Netflix so it's a bit aws-centric but still a good source for how to manage those log files. You may also want to look into some zk config settings to make sure your logs are not growing too large by truncating based on space or time. On Tue, Nov 11, 2014 at 9:40 AM, Joe Stein joe.st...@stealth.ly wrote: http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html#Ongoing+Data+Directory+Cleanup /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Tue, Nov 11, 2014 at 9:24 AM, Shlomi Hazan shl...@viber.com wrote: Hi, My zookeeper 'dataLogDir' is eating up my disk with tons of snapshot files. what are these files? what files can I delete? are week old files disposable? This folder only gets bigger... How can I avoid blowing my disk? Thanks, Shlomi
