[Tails-dev] Tahoe-LAFS persistence
-- Forwarded message -- From: intrigeri intrig...@boum.org Date: Thu, Jun 5, 2014 at 12:23 AM Subject: Re: [Tails-dev] Tahoe-LAFS persistence To: The Tails public development discussion list tails-...@boum.org Cc: tahoe-dev@tahoe-lafs.org, David Stainton dstainton...@gmail.com Hi, [Snipping large chunks of discussion about how various kinds of downloads are advertised on the Tahoe-LAFS website, as I fail to see what it has to do with Tails. It might be because I've not catched up with the rest of this thread yet.] Zooko Wilcox-OHearn wrote (03 Jun 2014 18:18:16 GMT) : So my question for Tails devs (this is where my ignorance begins): does Tails come with an SFTP client? Yes. The first hit I get when searching our website for SFTP is: https://tails.boum.org/doc/first_steps/introduction_to_gnome_and_the_tails_desktop/index.en.html#index4h1 Additionally, on any relevant Debian system, the command-line sftp client is shipped as part of the openssh-client package, that is installed in Tails. Does it come with a Bittorrent client? No: https://tails.boum.org/support/faq/#index22h2 Can we make Tahoe-LAFS be the third thing next to those two things? I'm not sure I get what you mean by next to those two things. Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
[Tails-dev] Tahoe-LAFS persistence
-- Forwarded message -- From: intrigeri intrig...@boum.org Date: Thu, Jun 5, 2014 at 3:03 PM Subject: Re: [Tails-dev] Tahoe-LAFS persistence To: David Stainton dstainton...@gmail.com Cc: The Tails public development discussion list tails-...@boum.org, tahoe-dev tahoe-dev@tahoe-lafs.org Hi, (Disclaimer: I still have not read this full thread.) David Stainton wrote (05 Jun 2014 13:28:21 GMT) : I think what Zooko is suggesting is that the Tahoe-LAFS debian package be included in the Tails releases. Thanks for clarifying. Now that we have debian packages and a maintainer ... which is great! is there any reason not to? If someone properly integrates Tahoe-LAFS withing Tails (including patching tails-persistence-setup, design doc, and whatever nobody has thought of yet), then I'm happy. I'm yet to see a branch that implements this, and that we can check out and test. Also see https://tails.boum.org/contribute/merge_policy/, particularly the Documentation is not optional section. I'm wondering whether, maybe, the best first step would be to add a page about Tahoe-LAFS in our documentation, that gives an overview of the pre-requisites (Tails installed with Tails Installer, persistent volume configured) and needed additional steps (I guess: have tahoe-lafs installed at every boot with the Additional software packages feature, make the right directories persistent, use it), pointing to the relevant documentation pages. Then, it's easy for anyone interested to try out, and while early testers give it a try, you can work on integrating Tahoe-Lafs within Tails, which is now the main blocker to install the software by default, from my PoV. And the branch that does the integration already has the user doc ready, it just needs to drop the bullet point about adding tahoe-lafs to the list of Additional software packages. But if someone feels bold enough to try and do it all in one single iteration, well, I'm happy too. Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Fwd: [Tails-dev] Tahoe-LAFS persistence
-- Forwarded message -- From: intrigeri intrig...@boum.org Date: Thu, Jun 5, 2014 at 6:07 PM Subject: Re: [Tails-dev] Tahoe-LAFS persistence To: David Stainton dstainton...@gmail.com Cc: The Tails public development discussion list tails-...@boum.org, tahoe-dev tahoe-dev@tahoe-lafs.org Hi, David Stainton wrote (05 Jun 2014 16:58:52 GMT) : Since you didn't read the entire thread... I want to make it explicitly clear that there is most definitely not one proper integration design for Tails and Tahoe-LAFS... AND Tahoe-LAFS cannot help with persistence... wrong tool for the job. This part I got, as Zooko made it clear a while ago on the ticket. Thanks for explaining anyway :) When I'm talking of persistence in my last email, it was about making the Tahoe-LAFS configuration persistent, *not* about using Tahoe-LAFS to host the persistent volume. There could be another track to follow... that might be better for the first iteration: Do the minimal amount of work possible to get Tails to ship Tahoe-LAFS... No fancy backup system. Just Tahoe-LAFS + a GUI + user docs... to help users configure their Tahoe client. What do you think of this? Looks good. Still, will need to persist the Tahoe-LAFS configuration. Hence my proposal. Then, it's easy for anyone interested to try out, and while early testers give it a try, you can work on integrating Tahoe-Lafs within Tails, which is now the main blocker to install the software by default, from my PoV. And the branch that does the integration already has the user doc ready, it just needs to drop the bullet point about adding tahoe-lafs to the list of Additional software packages. OK... clearly am confused and do not understand. My apologies for misunderstanding you : It sounds like you are saying that writing docs for non-existent software would make it easy for existent software to be used. Have you actually tried installing Tahoe-LAFS on Tails, in a way that it's re-installed automatically on every boot, and you don't have to reconfigure it every time you start Tails? If you haven't, please do. I suspect this will make my previous email much clearer :) Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [Tails-dev] Tahoe-LAFS persistence
Hi, I'm a Tahoe-LAFS core dev. I'm excited about the possibility of Tahoe-LAFS becoming really useful to Tails users, and I'm grateful to others, especially David Stainton, for moving this forward. On Sun, Jun 1, 2014 at 12:46 PM, David Stainton dstainton...@gmail.com wrote: Tahoe documentation could mention this: https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/quickstart.rst OK... I volunteer to update https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/quickstart.rst to mention install Tahoe-LAFS using the Debian package. Well, hold on, if people start by going to the Tahoe-LAFS home page (https://Tahoe-LAFS.org), then they see a prominent big blue button labeled Download Tahoe-LAFS. If they click that button, it takes them to https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Installation , which prominently advertises Debian (as well as Ubuntu, Arch Linux, NetBSD, Slackware, SmartOS, and NixOS), and which *less* prominently offers a hyperlink named Run this release from source. which takes them to the quickstart.rst. So, it seems like if they got to the quickstart.rst from that path, then they don't really need an added link or text on quickstart.rst pointing them back the way they came! What if they got to the quickstart.rst from a different path? Well, my first suggestion would be to find the thing that pointed them to quickstart.rst, and have it point them to https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Installation instead. 2. Tails persistent volume assistant feature additions: It's good to have a concrete integration proposition! Looks good to me (understandable and usable), but i'm curious about other people's opinions. And... would that not be better to have, like you propose, 3 columns, but the third being Tahoe, so users can choose to persist localy, OR on Tahoe, OR both - by selecting one column, the other, or both. But... Tahoe column should be grey if Tahoe has not been configured. I'd like to raise a warning flag here to indicate that it might be a mistake to categorize Tahoe-LAFS as being a persistence solution. It does *not* function super well when used as a local filesystem. Think of it instead as an alternative to SFTP. Or as an alternative to Bittorrent, which offers an *upload* action in addition to a download action. Here is my write-up for how Tails users could potentially use Tahoe-LAFS instead of SFTP: https://labs.riseup.net/code/issues/6227#note-20 On the other hand, as that write-up suggests, Tahoe-LAFS *might* be okay as an alternative to a (local) persistence solution, as long as the user doesn't mind the long lags on every action, doesn't mind that it breaks if it can't connect to the Internet, and doesn't use an app that tries to do a lot of small writes to disk, and doesn't use an app that times-out and gives up if its reads or writes take more than a few seconds to complete. So, I don't want to say Stop! Do not go forward and experiment with Tahoe-LAFS as a persistence tool., because it *might* work. But I do want to say Why don't we use it for what we know it is good at, and we can also do an experiment in using it as a persistence tool, and let it be okay if that experiment fails.. So my question for Tails devs (this is where my ignorance begins): does Tails come with an SFTP client? Does it come with a Bittorrent client? Can we make Tahoe-LAFS be the third thing next to those two things? 4. Tahoe-LAFS backup GUI applet I'll be opening a Tahoe-LAFS trac ticket about this one soon. Of course it will have to be written for the same desktop environment that Tails uses. I agree that this is lower priority than hacking the persistent volume assistant and writing a Tahoe-LAFS client configuration assistant. I will work on those first. I agree that Tahoe-LAFS is good for backup. By the way, I'm the founder of a company trying to commercialize Tahoe-LAFS, and we'd like to offer some free storage service as a way to help this effort along. I haven't figured out how to make it useful to Tails users (and devs) while limiting the amount of expense it would cost us to maintain it. One idea I just had is to hack our backend servers to reject any uploads of ciphertext files that are larger than about 100KB. I am guessing that this might make it still usable for config files, text documents, HTML pages, etc., but not usable for the photo, music, and movie files that might otherwise quickly start costing us too much to maintain. Any thoughts about that? (Obviously, someone could work-around that limitation by breaking their large files into many pieces, but I assume that wouldn't happen, at least not soon, because anyone who understands the system well enough to do that understands that doing that would force us to shutdown the free service, simply by ramping up our costs. If that's their goal, they could accomplish it just as well by writing a script to upload an infinite stream of files read from /dev/urandom.) Regards, Zooko
Re: [Tails-dev] Tahoe-LAFS persistence
On Sun, Jun 1, 2014 at 3:11 PM, Greg Troxel g...@ir.bbn.com wrote: This can be viewed as a bug in tahoe :-) But seriously, fixing the FUSE interface would be a great contribution. It's not clear to me how efficient the FUSE interface has to be before it isn't the limiting issue; tahoe is not a fast filesystem. Like Leif mentioned in reply, there *is* the SFTPd that comes with Tahoe-LAFS, and that is supported by the current Tahoe-LAFS core devs. Like gdt, I also don't know whether its performance would satisfy for this use case. Are you proposing to store the capabilities to access the persistent data on the local media (removable flash, I'm assuming)? I've come into this thread somewhat late, but the security and usability properties are not entirely clear to me. This is a key question (so to speak). How is the cap stored? The cap is basically just the encryption key plus the URL to reach the remote server. (The remote server stores only ciphertext, remember.) So, the cap needs to be protected exactly like a symmetric encryption key needs to be protected: if it is copied, the copier can read your files, and if it is lost, then *you* can never again read your files. I usually advise people to print the cap out on paper. (resident old-school unix crank on the tahoe list) We love you, gdt. Never change. Regards, Zooko Wilcox-O'Hearn Founder, CEO, and Customer Support Rep https://LeastAuthority.com Freedom matters. ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [Tails-dev] Tahoe-LAFS persistence
On Sun, Jun 1, 2014 at 10:14 PM, Leif Ryge l...@synthesize.us wrote: … many interesting things, including his sketch for a successor to, or extension of, Tahoe-LAFS (chisel) … *** *** BACK TO THE NEAR FUTURE *** *** … I look forward to seeing Tahoe integrated with Tails, but I am a little bit concerned about a potential pitfall which I think should be communicated to users somehow: there is no way to delete the ciphertext of immutable files … This is rather different from a typical access control based system where one can simply change their password and/or ask the server to delete everything quickly. We could implement this: add a feature that allows you to ask a server to quickly delete a ciphertext. This would be analogous to having a way to contact the owner of your SFTP server and ask her to delete a ciphertext that you earlier uploaded into the write-only incoming/ directory on that SFTP server. There are two or three practical engineering reasons that we haven't implement this, but the one I want to emphasize here is that we haven't implemented it because it doesn't provide good assurance of safety! If you contact the owner of the SFTP server and ask her to remove the ciphertext that you previously uploaded, and she writes back Okay, I removed it., then how do you know she actually deleted it? So, it isn't so much that Tahoe-LAFS is *less safe* than other alternatives in this way, as that we think those other alternatives are equally unsafe, and indeed it would offer a false sense of safety to add this feature. (Actually, Tahoe-LAFS is probably *more* safe than most alternatives, because every file and directory has an independent encryption key, so if a key of yours leaks or gets compromised, the exposure might be limited, and objects that are protected by other keys may remain safe.) Regards, Zooko Wilcox-O'Hearn Founder, CEO, and Customer Support Rep https://LeastAuthority.com Freedom matters. ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [Tails-dev] Tahoe-LAFS persistence
1. the Tahoe-LAFS debian package: This part is done. Great! Congratulation :) Tahoe documentation could mention this: https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/quickstart.rst OK... I volunteer to update https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/quickstart.rst to mention install Tahoe-LAFS using the Debian package. 2. Tails persistent volume assistant feature additions: It's good to have a concrete integration proposition! Looks good to me (understandable and usable), but i'm curious about other people's opinions. And... would that not be better to have, like you propose, 3 columns, but the third being Tahoe, so users can choose to persist localy, OR on Tahoe, OR both - by selecting one column, the other, or both. But... Tahoe column should be grey if Tahoe has not been configured. Since Tahoe-LAFS is not a posix compliant filesystem... we cannot easily create a persistent volume that only stores data on a Tahoe grid. There is an ugly FUSE hack but it is extremely ineffient. So there should be three options per persistent file-set: 1. do not persist 2. persist to local media 3. persist to local media AND a Tahoe-LAFS grid For the use case where you only want to store the data in the Tahoe grid... then simply use the Tahoe commandline tools to upload the file(s). And also, who volunteers to hack the persistent volume assistant? I volunteer to hack the persistent volume assistant. I would certainly help out with documentation as well. 3. periodic Tahoe-LAFS backup scheduler I suggest to raise the idea in Tahoe mailing list... once it exists we can see if it's interesting for Tails :) I agree! Daira from the Tahoe-LAFS dev team pointed me to the relevant Tahoe trac tickets. I will be carefully reviewing these... and chatting with the Tahoe-LAFS dev team when I need advice/assistance/ideas. 4. Tahoe-LAFS backup GUI applet I'll be opening a Tahoe-LAFS trac ticket about this one soon. Of course it will have to be written for the same desktop environment that Tails uses. I agree that this is lower priority than hacking the persistent volume assistant and writing a Tahoe-LAFS client configuration assistant. I will work on those first. Onward! David ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [Tails-dev] Tahoe-LAFS persistence
David Stainton dstainton...@gmail.com writes: Since Tahoe-LAFS is not a posix compliant filesystem... we cannot easily create a persistent volume that only stores data on a Tahoe grid. There is an ugly FUSE hack but it is extremely ineffient. This can be viewed as a bug in tahoe :-) But seriously, fixing the FUSE interface would be a great contribution. It's not clear to me how efficient the FUSE interface has to be before it isn't the limiting issue; tahoe is not a fast filesystem. So there should be three options per persistent file-set: 1. do not persist 2. persist to local media 3. persist to local media AND a Tahoe-LAFS grid Are you proposing to store the capabilities to access the persistent data on the local media (removable flash, I'm assuming)? I've come into this thread somewhat late, but the security and usability properties are not entirely clear to me. For the use case where you only want to store the data in the Tahoe grid... then simply use the Tahoe commandline tools to upload the file(s). That seems like it could easily be: 4. persist to tmpdir and then upload to tahoe, deleting the tmp file. I think the state of the FUSE interface isn't all that relevant if you're going to add code for tahoe anyway. The tahoe cp interface is very dos/mtools, but quite workable, even if it would be better to be able to use a standard VFS interface. Greg (resident old-school unix crank on the tahoe list) ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [Tails-dev] Tahoe-LAFS persistence
On Sun, Jun 01, 2014 at 11:11:29AM -0400, Greg Troxel wrote: David Stainton dstainton...@gmail.com writes: Since Tahoe-LAFS is not a posix compliant filesystem... we cannot easily create a persistent volume that only stores data on a Tahoe grid. There is an ugly FUSE hack but it is extremely ineffient. This can be viewed as a bug in tahoe :-) But seriously, fixing the FUSE interface would be a great contribution. It's not clear to me how efficient the FUSE interface has to be before it isn't the limiting issue; tahoe is not a fast filesystem. While Tahoe's native FUSE interface bitrotted long ago, there are two FUSE interfaces which are currently usable: - Tahoe has an SFTP server can be mounted with FUSE's sshfs, like any other SFTP server - the python-fs module (a general filesystem abstraction library) has a Tahoe client, using Tahoe's web api, and can expose it (like any python-fs object) via FUSE. The big problem with using these fuse mounts for many applications is that Tahoe mutables don't provide random-access writes. So if you put, say, your firefox profile on a Tahoe-backed FUSE mount... every write+sync to Firefox's places.sqlite etc will involve re-uploading the whole thing and firefox will only be usable for brief moments at a time if at all (I think - I haven't tried it). * *** The following section of this email is not about Tahoe+Tails. *** *** If you're just interested in that, skip to BACK TO THE NEAR FUTURE. *** * In my opinion, this is not something that should be fixed by improving Tahoe's current mutable files, but rather by replacing them since they have several other shortcomings. Most importantly (imo): - They don't preserve history (each write overwrites the previous version, so if you have a write capability you can also overwrite) - They aren't lockable (if you have uncoordinated writes to a file, you're gonna have a bad time. so, you must be very careful sharing writecaps.) - There is no asymmetric encryption (if you have a write capability, you can also read). - They aren't deduplicated at a file level, much less at a block-level My hand-wavey ideal solution to these problems (chisel) involves hashsplit (BUP-style) asymmetrically-encrypted immutable files, references to which are added to a directory which is a *decentralized add-only set*. So, there can be write-only capabilities which can neither read nor delete data after they've written it, and because the directory is an add-only set instead of an append-only file there can be multiple writers without coordination. I've got a rough idea about how to do this, and a little bit of code... hopefully I'll find time to work on it more soon. I'm building this separately from Tahoe, but intending to (optionally) use Tahoe immutable files underneath, and I'd like to eventually be able to expose a FUSE interface that *does* allow random access writes to files. Probably Tahoe will need some performance improvements for it to work well on top of it, though. Another undesirable thing about Tahoe's current mutables is that write caps contain RSA private keys which are rather cumbersome to write down. If they were ECC private keys they could be generated from memorable secrets (which is potentially dangerous but quite convenient) or at least shorter secrets (because RSA requires much larger keys than ECC for a given security level). But none of this is very relevant to the issue of using Tahoe for Tails persistence in the immediate future, so... *** *** BACK TO THE NEAR FUTURE *** *** So there should be three options per persistent file-set: 1. do not persist 2. persist to local media 3. persist to local media AND a Tahoe-LAFS grid Are you proposing to store the capabilities to access the persistent data on the local media (removable flash, I'm assuming)? I've come into this thread somewhat late, but the security and usability properties are not entirely clear to me. I think the current plan for Tails persistence on Tahoe is to persist to the USB disk (as Tails already does) and run tahoe backup on a regular schedule (and/or when the user manually triggers it, and/or triggered by some sort of inotify-driven agent). So, yes, the user should store their root cap on Tails' encrypted persistent partition, and also back it up elsewhere (on another usb stick, or on paper). The Tails persistence setup tool should then have an option to restore from an existing Tahoe root cap. I guess the restore could also be done to a ramdisk on a Tails system without USB persistence, and maybe something could even be done using unionfs to combine a ramdisk with a fuse-mounted tahoe directory to avoid needing to download the whole thing? I wonder how that would work.
Fwd: [Tails-dev] Tahoe-LAFS persistence
-- Forwarded message -- From: BitingBird bitingb...@riseup.net Date: Fri, May 30, 2014 at 2:06 AM Subject: Re: [Tails-dev] Tahoe-LAFS persistence To: The Tails public development discussion list tails-...@boum.org David Stainton: Hi, [...] 1. the Tahoe-LAFS debian package: This part is done. Great! Congratulation :) Tahoe documentation could mention this: https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/quickstart.rst 2. Tails persistent volume assistant feature additions: Right now the Tails persistent volume assistant has a user configurable list of persistence futures which correspond to sets of files that can be persisted. The user can selects which to persist or not persist. For each of these items there should be a third option: persist to Tahoe-LAFS grid and persist to local volume. It's good to have a concrete integration proposition! Looks good to me (understandable and usable), but i'm curious about other people's opinions. And... would that not be better to have, like you propose, 3 columns, but the third being Tahoe, so users can choose to persist localy, OR on Tahoe, OR both - by selecting one column, the other, or both. But... Tahoe column should be grey if Tahoe has not been configured. And also, who volunteers to hack the persistent volume assistant? The persistent volume assistant should also prompt the user for some information on how to configure Tahoe-LAFS. Somebody wants to write the text? I looked at Tahoe documentation, and it seemed that there is no GUI for that. https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/running.rst looks complete, if not very user-friendly, so linking to that could be a good start. Once Tahoe persistence is implemented, documentation: https://tails.boum.org/doc/about/features/index.en.html should mention Tahoe, probably in the Encryption Privacy part. https://tails.boum.org/contribute/design/#index44h3 should mention it, right after 3.6.17 Persistence feature. Maybe there should be a sub-ticket from #6227 about the documentation. 3. periodic Tahoe-LAFS backup scheduler This daemon could be part of Tahoe-LAFS... there is nothing Tails specific about it. [...] I suggest to raise the idea in Tahoe mailing list... once it exists we can see if it's interesting for Tails :) 4. Tahoe-LAFS backup GUI applet [...] Idem, probably interesting ideas for Tahoe developers. But from what I read from the documentation, I suggest a GUI to setup Tahoe client first. As it is, it's quite a geek tool, IMO. Tahoe-LAFS runs just fine on Tails. That's a good start :) Cheers, BitingBird ___ Tails-dev mailing list tails-...@boum.org https://mailman.boum.org/listinfo/tails-dev To unsubscribe from this list, send an empty email to tails-dev-unsubscr...@boum.org. ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
