Re: [tahoe-dev] Help adjusting buildbot after upgrade?
On Wed, Nov 10, 2010 at 12:04 AM, Kyle Markley wrote: > Zooko et al, > > My system is back up after upgrading the OS from OpenBSD 4.6 to 4.8. I've > more-or-less gotten the buildslaves running again, but I could use some help > there. > > (A) My buildslave name contains 'openbsd46'; I feel like this should be > updated, but I'm nervous about breaking something. How is this done > correctly? Buildmaster and buildslave have to change the name simultaneously. I'll go change it from "starfish-openbsd46-amd64" to just "starfish" in the buildmaster's config file, and the next time you make the same change (in the buildbot.tac file) and restart then it will reconnect. > (B) There are some errors running the build and I suspect they're related to > installation snafus and missing packages. I'd appreciate some guidance here > on what exactly is wrong. What errors? This is all green: http://tahoe-lafs.org/buildbot/builders/Kyle%20OpenBSD-4.6%20amd64/builds/480 (Note that link will probably break when we rename the buildslave. Oh well.) > It's worth mentioning that I had a "fun" upgrade. I made a mistake and > inadvertently blew away my /home partition. (I naively thought that telling > the installer which partition would become /home did *not* imply that I > wanted the partition to be formatted during the install!) I'm restoring my > data from a tahoe backup right now, courtesy the undamaged portion of my > grid. Hooray for backups! (Did the backup succeed?) > The only thing really bothering me right now is tahoe's behavior with > symlinks. It skips them. I didn't have any kind of separate symlink > backup, so I've lost them all. That's a drag, because I had some pretty > labor-intensive piles of symlinks. I wish tahoe would cover symlinks by > recording the link (not the file pointed to) instead of skipping it. Brian replied to this and linked to the relevant issue ticket. > Also, as an FYI, when doing a fresh install of tahoe (1.8.0) on this > otherwise-clean system, I noticed that "python setup.py install" installed > only tahoe itself. It did not install any dependencies, and it did not fail > because the dependencies were absent. I had to install all of the > dependencies manually. Of course "python setup.py build" fetched and built > the dependencies, but I couldn't figure out how to install them from the > source tree to /usr/local/lib. When I first started tinkering with tahoe a > few versions ago, the install would also ensure the dependencies were > installed. If this is all fixed in 1.8.1, ignore this complaint. Please open a ticket for this! To test this we'll probably have to have the buildbot do something like python setup.py install --prefix=test-full-inst-dir and then our build system passes the test only if the following command emits the right output: cd test-full-instdir && ./bin/tahoe --version-and-path Does that sound like an accurate test for your desired feature? Regards, Zooko ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] Help adjusting buildbot after upgrade?
Kyle Markley writes: > On Wed, 10 Nov 2010 07:53:46 -0500, Greg Troxel wrote: >> I think you have a good point about symlinks. Probably you should >> file >> a ticket so this doesn't get lost. It isn't immediately obvious how >> they should work, especially because tahoe does not have a concept of >> "..". I think it can't grow one, because .. is a huge change to >> chained >> capabilities semantics. >> >> backup on the other hand is perhaps done by storing dump or tar >> files in >> tahoe. I have not figured out what I want to do yet. > > > If a backup and restore were known to be happening on the same "kind" > of system, symlinks are easy: a symlink is merely a string as returned > by readlink(2). Tahoe could save such strings in directory entries > and restore them with symlink(2). Without '..', of course, it would > be impossible to use symlinks for navigation within the web interface > -- but for my application I don't care about navigation, I just want > them backed up and restored! > > Across systems, symlinks are more complicated because tahoe would need > to understand the substrings representing (at least) the current > directory, parent directory, and the path separator. On the Perl side > of the universe, there's a library for portably working with filename > strings, and the ideas there may be a good place to start: perldoc > File::Spec I am actually not that concerned about symlinks myself, but doesn't tar have exactly the same set of issues, and all other distributed filesystems? Probably tahoe needs to have a notion of a pathname and a bunch of other things that it has so far mostly avoided by having primitive operations in the WAPI. For example, typing $ tahoe mkdir 'foo/bar' does not cause an error promptly. This seems like perhaps a bug, and I don't think there is any value in making filenames available that can't be referenced on unix. The mkdir is still running, and it's waiting for share placement due to pubgrid firewall issues I think. pgpqksfDN1LyN.pgp Description: PGP signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] Help adjusting buildbot after upgrade?
On 11/10/10 8:05 AM, Kyle Markley wrote: > On Wed, 10 Nov 2010 07:53:46 -0500, Greg Troxel wrote: >> I think you have a good point about symlinks. Probably you should file >> a ticket so this doesn't get lost. It isn't immediately obvious how >> they should work, especially because tahoe does not have a concept of >> "..". I think it can't grow one, because .. is a huge change to chained >> capabilities semantics. Ticket #641 is about 'tahoe backup' being able to backup symlinks. On the Tahoe side, symlinks would be stored as just data (a string with the target of the link), with a flag to inform some eventual restore program that it's a symlink and not a normal file. Nothing on the Tahoe side would be able to follow such links: as you point out, that'd be a complete violation of Tahoe's access semantics. We don't really have a 'tahoe restore' command yet: 'tahoe cp' (from tahoe into the local disk) is close. Maybe #641 should also ask for a --restore-symlinks argument to 'tahoe cp' that would do the inverse of what 'tahoe backup' does with symlinks. Also, maybe 'tahoe cp' (from disk into tahoe) should process symlinks too. Maybe a --symlinks option to 'tahoe cp', which defaults to true for 'tahoe backup'. I think it'd be ok if the "symlinks" that got archived into a tahoe filesystem behaved similarly symlinks that get archived into a tarball: when you unpack it, you can wind up with dangling links that go to crazy places which existed on the original but not on the new disk, but that's just ok. cheers, -Brian #641: http://tahoe-lafs.org/trac/tahoe-lafs/ticket/641 ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] Help adjusting buildbot after upgrade?
On Wed, 10 Nov 2010 07:53:46 -0500, Greg Troxel wrote: I think you have a good point about symlinks. Probably you should file a ticket so this doesn't get lost. It isn't immediately obvious how they should work, especially because tahoe does not have a concept of "..". I think it can't grow one, because .. is a huge change to chained capabilities semantics. backup on the other hand is perhaps done by storing dump or tar files in tahoe. I have not figured out what I want to do yet. If a backup and restore were known to be happening on the same "kind" of system, symlinks are easy: a symlink is merely a string as returned by readlink(2). Tahoe could save such strings in directory entries and restore them with symlink(2). Without '..', of course, it would be impossible to use symlinks for navigation within the web interface -- but for my application I don't care about navigation, I just want them backed up and restored! Across systems, symlinks are more complicated because tahoe would need to understand the substrings representing (at least) the current directory, parent directory, and the path separator. On the Perl side of the universe, there's a library for portably working with filename strings, and the ideas there may be a good place to start: perldoc File::Spec -- Kyle Markley ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] Help adjusting buildbot after upgrade?
I think you have a good point about symlinks. Probably you should file a ticket so this doesn't get lost. It isn't immediately obvious how they should work, especially because tahoe does not have a concept of "..". I think it can't grow one, because .. is a huge change to chained capabilities semantics. backup on the other hand is perhaps done by storing dump or tar files in tahoe. I have not figured out what I want to do yet. pgp1E2VC25zA4.pgp Description: PGP signature ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
