Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
I use vac -a to back up several unix systems to my main Plan 9 file server. Currently I'm doing two nightly via cron and two sporadically (laptops); there have been more of each in the past. In addition to storing the scores locally, I wrote a little rc script that lives in /rc/bin/service.auth and accepts scores and stores them on my CPU server. Combined with a simple 9fs addition, I can always "9fs foo.vac" and get foo's history of archives from all my plan9 systems. I've also done restores from p9p, but have not exercised that well. I considered using vbackup instead of vac on the unix side, but it didn't look practical to get at the results from plan9. Also, I really needed to be selective in what I saad backing up (there's a few hundred Gb of post-processed video I don't need to store on one of these systems, and growing daily). This has worked very well for me overall; my only problem has been vtcache exhaustion on some backups, which I haven't been able to track down. Thankfully the error is on backup, not restore, so you know something's gone wrong when it happens. My message about vtcache is at http://9fans.net/archive/2010/02/307. I'm not sure I pushed the absolute latest to sources, but some version of my unix backup script is at /n/sources/contrib/anothy/bin/rc/vacbak.
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
Seems a very logical way to go. On Mon, Nov 22, 2010 at 7:56 AM, Latchesar Ionkov wrote: > I send the venti scores to my email account and burn them on the DVDs > with the arenas. > >Lucho > > On Wed, Nov 17, 2010 at 9:51 AM, David Leimbach wrote: > > I'm giving consideration to maintaining a venti-based setup for my house > for > > all the digital media we have (since getting our Apple TV, we've had more > > stuff to stream around the house). > > I've just now started playing with things like vac/unvac, to backup and > > extract trees of my HFS+ file system and I wonder about a few things. > > What do people do if they ever lose their venti scores? Seems like this > is > > "handle-less" data now, and fairly useless. > > I figure I could keep a vac archive of venti scores, then I'd only need > to > > "remember" one, and name the files I store the scores in reasonably, but > if > > that's lost... what are my options? > > Dave > > > > > >
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
I send the venti scores to my email account and burn them on the DVDs with the arenas. Lucho On Wed, Nov 17, 2010 at 9:51 AM, David Leimbach wrote: > I'm giving consideration to maintaining a venti-based setup for my house for > all the digital media we have (since getting our Apple TV, we've had more > stuff to stream around the house). > I've just now started playing with things like vac/unvac, to backup and > extract trees of my HFS+ file system and I wonder about a few things. > What do people do if they ever lose their venti scores? Seems like this is > "handle-less" data now, and fairly useless. > I figure I could keep a vac archive of venti scores, then I'd only need to > "remember" one, and name the files I store the scores in reasonably, but if > that's lost... what are my options? > Dave > >
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Thursday 18 November 2010 20:40:13 Bakul Shah wrote: > On Wed, 17 Nov 2010 09:44:27 PST David Leimbach wrote: > > On Wed, Nov 17, 2010 at 9:23 AM, dexen deVries > > wrote > > > > > On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: > > > > (...) > > > > I'd be very careful with vac -m and -a on Unix; both have been at the > > > > root of considerable data-loss on a unix venti for me. I'd recommend > > > > vac-ing tarballs, rather than using vac's on unix trees directly. But > > > > your mileage may vary... > > > > > > could you please elaborate a bit about that data loss? > > > traversing symlinks breaks? some files not getting read by vac at all? > > > > > > (I'm interested in using p9p vac+venti in similar manner, but on Linux > > > w/ GNU stuff) > > > > I could imagine vac/unvac not dealing with resource forks or POSIX > > extended attributes and such properly, as well as potentially having > > difficulty with symlinks, but having dealt with stuff like that in > > "xar", I don't think it's too difficult to address. > > > > I may need to read up on venti and see what sorts of data types it > > supports. > > > > Might be time to add some extensions? > > venti doesn't care but vac/unvac do deal with symlinks, fifos > and special devices. The problem with -a is that a > /mmdd/ prefix gets prepended to all paths and these dirs > are readonly (555). unvac coredumps in trying to extract > anything under /. The real problem is that unvac needs to > handle non-empty 555 dirs specially (like tar does). > > Try this on unix: > > mkdir -p a/b > chmod 555 a > tar cf - a | (cd /tmp; tar -xvf -) > vac a | (cd /tmp; unvac -v) > > The basic problem is that venti & friends need some grunt work to > make them bullet/idiot proof. thanks ;) -- dexen deVries ``One can't proceed from the informal to the formal by formal means.''
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, 17 Nov 2010 09:44:27 PST David Leimbach wrote: > > On Wed, Nov 17, 2010 at 9:23 AM, dexen deVries wrote > : > > > On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: > > > (...) > > > I'd be very careful with vac -m and -a on Unix; both have been at the > > > root of considerable data-loss on a unix venti for me. I'd recommend > > > vac-ing tarballs, rather than using vac's on unix trees directly. But > > > your mileage may vary... > > > > > > could you please elaborate a bit about that data loss? > > traversing symlinks breaks? some files not getting read by vac at all? > > > > (I'm interested in using p9p vac+venti in similar manner, but on Linux w/ > > GNU stuff) > > I could imagine vac/unvac not dealing with resource forks or POSIX extended > attributes and such properly, as well as potentially having difficulty with > symlinks, but having dealt with stuff like that in "xar", I don't think it's > too difficult to address. > > I may need to read up on venti and see what sorts of data types it supports. > Might be time to add some extensions? venti doesn't care but vac/unvac do deal with symlinks, fifos and special devices. The problem with -a is that a /mmdd/ prefix gets prepended to all paths and these dirs are readonly (555). unvac coredumps in trying to extract anything under /. The real problem is that unvac needs to handle non-empty 555 dirs specially (like tar does). Try this on unix: mkdir -p a/b chmod 555 a tar cf - a | (cd /tmp; tar -xvf -) vac a | (cd /tmp; unvac -v) The basic problem is that venti & friends need some grunt work to make them bullet/idiot proof.
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 9:23 AM, dexen deVries wrote: > On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: > > (...) > > I'd be very careful with vac -m and -a on Unix; both have been at the > > root of considerable data-loss on a unix venti for me. I'd recommend > > vac-ing tarballs, rather than using vac's on unix trees directly. But > > your mileage may vary... > > > could you please elaborate a bit about that data loss? > traversing symlinks breaks? some files not getting read by vac at all? > > (I'm interested in using p9p vac+venti in similar manner, but on Linux w/ > GNU > stuff) > > -- > dexen deVries > I could imagine vac/unvac not dealing with resource forks or POSIX extended attributes and such properly, as well as potentially having difficulty with symlinks, but having dealt with stuff like that in "xar", I don't think it's too difficult to address. I may need to read up on venti and see what sorts of data types it supports. Might be time to add some extensions? Dave > > > ``One can't proceed from the informal to the formal by formal means.'' > >
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 12:23 PM, dexen deVries wrote: > On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: >> (...) >> I'd be very careful with vac -m and -a on Unix; both have been at the >> root of considerable data-loss on a unix venti for me. I'd recommend >> vac-ing tarballs, rather than using vac's on unix trees directly. But >> your mileage may vary... > > > could you please elaborate a bit about that data loss? > traversing symlinks breaks? some files not getting read by vac at all? I have a number of vac archives that were made by vac-ing my homedir that cannot be mounted (venti can't find some of the scores in the tree). However, they did pass scrubbing (scrub was a tool i wrote that is sorta like half of venti/copy -- it traverses a tree, checks hashes, but doesn't write them out to anything). I have a number of trees in which somehow the SHA1 of the empty string was inserted: 'da39a3ee...'. And I have a number of trees that were venti/copied from one venti to another but somehow damaged (scores missing) after the copy, without seeing errors. vac -a in particular was good at producing archives which either don't mount or are filled with missing scores or zero-scores vs sha1(empty string) problems. Unvac is not useful for anything involving directories, since it outputs dirs which cannot be modified. http://groups.google.com/group/plan9port-dev/browse_thread/thread/8b0e2bc4b45d12b0 is a reasonably common bug from vac, seen on deep-ish directory trees. Two threads i can't find on the google groups interface, from p9p-dev as well: 1) 8/12/09 'venti/vac/vacfs trouble' and 2) 8/17/09 'vac errors'. Both bugs still exist afaik. I've seen this: cache block 501: type 8 score 1f6 iostate 1 addr 502 ref 0 nlock 1 cache block 502: type 8 score 1f7 iostate 1 addr 503 ref 0 nlock 1 cache block 503: type 8 score 1f8 iostate 1 addr 504 ref 0 nlock 1 (output starts at 0x000...000 ;; goes up to score 0x000...200). vtcachebumpblock: no free blocks in vtCachezsh: IOT instruction ./vac -s -v -h tcp!acm.jhu.edu!17034 /export/home happen on deep-ish directory trees as well; I never managaed to track it down, though. Most recently, I've run two venti servers; vac targets the first one, every night the score is venti/copied to the second server. After two weeks, the two servers (with identical initial configuration) had different size and block counts from the http://.../storage. -- vs
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 9:14 AM, Venkatesh Srinivas wrote: > On Wed, Nov 17, 2010 at 11:51 AM, David Leimbach > wrote: > > I'm giving consideration to maintaining a venti-based setup for my house > for > > all the digital media we have (since getting our Apple TV, we've had more > > stuff to stream around the house). > > I've just now started playing with things like vac/unvac, to backup and > > extract trees of my HFS+ file system and I wonder about a few things. > > What do people do if they ever lose their venti scores? Seems like this > is > > "handle-less" data now, and fairly useless. > > I figure I could keep a vac archive of venti scores, then I'd only need > to > > "remember" one, and name the files I store the scores in reasonably, but > if > > that's lost... what are my options? > > Dave > > There is a script floating around (dumpvacroots or somesuch) that lets > you recover vac scores given access to the venti arenas. > > I'd be very careful with vac -m and -a on Unix; both have been at the > root of considerable data-loss on a unix venti for me. I'd recommend > vac-ing tarballs, rather than using vac's on unix trees directly. But > your mileage may vary... > This is mainly a form of secondary backup for me for now, but given what I learn about it over time, it could become a primary. Are there any open problem reports around this? I might be interested in tackling some of these, or at least trying to reproduce them. I still do some rsync based backups anyway. > > -- vs > >
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: > (...) > I'd be very careful with vac -m and -a on Unix; both have been at the > root of considerable data-loss on a unix venti for me. I'd recommend > vac-ing tarballs, rather than using vac's on unix trees directly. But > your mileage may vary... could you please elaborate a bit about that data loss? traversing symlinks breaks? some files not getting read by vac at all? (I'm interested in using p9p vac+venti in similar manner, but on Linux w/ GNU stuff) -- dexen deVries ``One can't proceed from the informal to the formal by formal means.''
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 11:51 AM, David Leimbach wrote: > I'm giving consideration to maintaining a venti-based setup for my house for > all the digital media we have (since getting our Apple TV, we've had more > stuff to stream around the house). > I've just now started playing with things like vac/unvac, to backup and > extract trees of my HFS+ file system and I wonder about a few things. > What do people do if they ever lose their venti scores? Seems like this is > "handle-less" data now, and fairly useless. > I figure I could keep a vac archive of venti scores, then I'd only need to > "remember" one, and name the files I store the scores in reasonably, but if > that's lost... what are my options? > Dave There is a script floating around (dumpvacroots or somesuch) that lets you recover vac scores given access to the venti arenas. I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... -- vs
[9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
I'm giving consideration to maintaining a venti-based setup for my house for all the digital media we have (since getting our Apple TV, we've had more stuff to stream around the house). I've just now started playing with things like vac/unvac, to backup and extract trees of my HFS+ file system and I wonder about a few things. What do people do if they ever lose their venti scores? Seems like this is "handle-less" data now, and fairly useless. I figure I could keep a vac archive of venti scores, then I'd only need to "remember" one, and name the files I store the scores in reasonably, but if that's lost... what are my options? Dave
