You may find this interesting, although I don't believe it's exactly what you're looking for:
https://github.com/pinterest/secor I'm not sure how stable and commonly used it is. Additionally, I see a lot of users use MirrorMaker for a "backup," where MirrorMaker copies all topics from one Kafka cluster to another "backup" cluster. I put "backup" in quotes because this architecture doesn't support snapshotting like a traditional backup would. I realize this doesn't address your specific use case, but thought you may find it interesting regardless. Sorry I'm a little late to the thread, too. Alex On Thu, May 5, 2016 at 7:05 AM, Rad Gruchalski <ra...@gruchalski.com> wrote: > John, > > I’m not as expert expert in Kafka but I would assume so. > > > > > > > > > > > Best regards, > Radek Gruchalski > ra...@gruchalski.com (mailto:ra...@gruchalski.com) (mailto: > ra...@gruchalski.com) > de.linkedin.com/in/radgruchalski/ ( > http://de.linkedin.com/in/radgruchalski/) > > Confidentiality: > This communication is intended for the above-named person and may be > confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor > must you copy or show it to anyone; please delete/destroy and inform the > sender immediately. > > > > On Thursday, 5 May 2016 at 01:46, John Bickerstaff wrote: > > > Thanks - does that mean that the only way to safely back up Kafka is to > > have replication? > > > > (I have done this partially - I can get the entire topic on the command > > line, after completely recreating the server, but my code that is > intended > > to do the same thing just hangs) > > > > On Wed, May 4, 2016 at 3:18 PM, Rad Gruchalski <ra...@gruchalski.com > (mailto:ra...@gruchalski.com)> wrote: > > > > > John, > > > > > > I believe you mean something along the lines of: > > > http://markmail.org/message/f7xb5okr3ujkplk4 > > > I don’t think something like this has been done. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > Radek Gruchalski > > > ra...@gruchalski.com (mailto:ra...@gruchalski.com) (mailto: > > > ra...@gruchalski.com (mailto:ra...@gruchalski.com)) > > > de.linkedin.com/in/radgruchalski/ ( > http://de.linkedin.com/in/radgruchalski/) ( > > > http://de.linkedin.com/in/radgruchalski/) > > > > > > Confidentiality: > > > This communication is intended for the above-named person and may be > > > confidential and/or legally privileged. > > > If it has come to you in error you must take no action based on it, nor > > > must you copy or show it to anyone; please delete/destroy and inform > the > > > sender immediately. > > > > > > > > > > > > On Wednesday, 4 May 2016 at 23:04, John Bickerstaff wrote: > > > > > > > Hi, > > > > > > > > I have what is probably an edge use case. I'd like to back up a > single > > > > Kafka instance such that I can recreate a new server, drop Kafka in, > drop > > > > the data in, start Kafka -- and have all my data ready to go again > for > > > > consumers. > > > > > > > > Is such a thing done? Does anyone have any experience trying this? > > > > > > > > I have, and I've run into some problems which suggest there's a > setting > > > or > > > > some other thing I'm unaware of... > > > > > > > > If you like, don't think of it as a backup problem so much as a > "cloning" > > > > problem. I want to clone a new Kafka machine without actually > cloning it > > > > > > > > > > - > > > > I.E. the data is somewhere else (log and index files) although > Zookeeper > > > > > > is > > > > up and running just fine. > > > > > > > > Thanks > >
