What is the recommended backup strategy for Riak with leveldb backends (since just copying the data dirs seems to have issues)? Is riak-admin backup sufficient? Will riak-admin backup also work with secondary indexes and riak-search?
-- Jeremy On Tue, Feb 14, 2012 at 5:05 PM, John Loehrer <[email protected]> wrote: > as requested, I reproduced the bug, then straced the process to track it down > to a specific dir in leveldb: > > /var/lib/riak/leveldb/274031556999544297163190906134303066185487351808 > > Attaching all the non sst data files in that directory for you to look over. > Some of the highlights: > > ----- > > 2012/02/14-21:55:21.088839 48a64940 compacted to: files[ 4 4 51 171 0 0 0 ] > 2012/02/14-21:55:21.088846 48a64940 Compaction error: IO error: > /var/lib/riak/leveldb/274031556999544297163190906134303066185487351808/001060.sst: > No such file or directory > 2012/02/14-21:55:21.088859 48a64940 Compacting 4@0 + 4@1 files > 2012/02/14-21:55:21.090074 48a64940 compacted to: files[ 4 4 51 171 0 0 0 ] > 2012/02/14-21:55:21.090082 48a64940 Compaction error: IO error: > /var/lib/riak/leveldb/274031556999544297163190906134303066185487351808/001060.sst: > No such file or directory > 2012/02/14-21:55:21.090096 48a64940 Compacting 4@0 + 4@1 files > 2012/02/14-21:55:21.091271 48a64940 compacted to: files[ 4 4 51 171 0 0 0 ] > 2012/02/14-21:55:21.091278 48a64940 Compaction error: IO error: > /var/lib/riak/leveldb/274031556999544297163190906134303066185487351808/001060.sst: > No such file or directory > > ----- > > > see that pattern over and over again in the file. > > > ~ John > > > ----- Original Message ----- > From: "David Smith" <[email protected]> > To: "John Loehrer" <[email protected]> > Cc: "Jeremy Raymond" <[email protected]>, [email protected] > Sent: Monday, February 13, 2012 8:09:22 AM > Subject: Re: standby cluster experiment > > > > > On Mon, Jan 23, 2012 at 3:40 PM, John Loehrer < [email protected] > > wrote: > > > At this point one or more of the nodes will start misbehaving, and you will > see the data directory grow at the rate of about 100 MB per second until it > fills up the disk or until you restart riak on that node. > > > > The disk usage problem was caused by a bug in LevelDB -- this should be fixed > as part of the 1.1 release (and the RCs). I believe the root problem you're > seeing, however, is that if you snapshot when a new version of the MANIFEST > file is being written, you can get a partial dump of data from leveldb and > the repair will fail on restart. If you can recreate this problem and send me > the LOG file (generated by leveldb in the vnode data dir), I'd be curious to > see what it says. > > > Thanks, > > > D. > > -- > Dave Smith > VP, Engineering > Basho Technologies, Inc. > [email protected] _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
