Avi, Congratulations on deploying this new service!
More responses inline: On Wed, Jul 17, 2013 at 9:23 PM, Avi Freedman <freed...@freedman.net> wrote: > [✂ ...] > > We wanted to let everyone in the community know that Havenco > is launching storage (and VPN services) in beta this month, and the > storage service is going to offer LAFS nodes (like rentanode.nl > had been doing) as well as S3-compatible buckets. > Nice! The S3-compatible service is unrelated to LAFS, correct? Speaking of S3 buckets, I've often wondered what benefits and challenges are present for providing an S3 interface on top of a LAFS gateway. The S3 client could speak to a local gateway to preserve the confidentiality properties of LAFS, while itself being unaware of LAFS (and only understanding S3). Presumably many S3 clients depend on different distributed update semantics than are provided by LAFS, though, so for many applications this may be an architectural mistake. Still, writing the adaption from S3 client -> proxy -> LAFS webapi seems fairly straightforward. The proxy might be configured with a single writecap, and it may store the index of buckets to bucket contents as a LAFS directory, or that mapping may be represented outside of LAFS proper, especially if some kind of distributed update mechanism is in place. > [✂ ...] > > The architecture we're running for LAFS to do accounting is one > that the LA team helped us validate - we're doing one Linux uid > per customer and running 10 tahoe procs each in a separate directory > across separate machines, and running RAID underneath so that we > can ignore the potential performance issues with client-side > resync right now. > Those uids / directories are for LAFS storage nodes, right? So each storage node is in a separate LAFS grid, and each customer uses their own grid? This makes sense for accounting: the provider can examine the share sizes for each separate node and charge the customer accordingly. This is also how Least Authority's TLoS3 (and soon to be announced descendent service) operate. (Disclaimer: I work for Least Authority, although not directly on their products. I did not vet anything in this email with anyone there and the views are my own, and obviously biased.) It's a shame though, IMO, that all users cannot share a single grid in the service. This could have two benefits: First, customers can share caps with each other; Second, the overhead for the service provider may be much lower (for example ~100 storage node hosts for tens of thousands of customers). It seems "the Right Way" to enable this is to implement the accounting roadmap merged into the open source trunk. It is described here for all interested: https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/proposed/accounting-overview.txt That design is flexible in a way that would benefit service providers, but also friend nets, or myriad other use cases. > If there's a lot of interest the plan is to develop or fund > development towards progress on the accounting-related tickets; > we'd prefer to work in a mode where all the ciphertext from > all the customers are interspersed. Just hard to see doing > that now with no way to track usage. Yes! There's a lot of interest, at least from me! ;-) If anyone reading this list is looking for what technology to donate time or funding to, I propose the accounting featureset for LAFS. IMO, this is more fundamental for scaling grid sizes or branching out into more use cases than any other proposed feature set. Of course, distributed introduction and UI / usability features are crucial for wide spread adoption... but to me it seems accounting is the last infrastructural / incentive-engineering hurdle. > > We've got the basics of node and introducer setup going but would > love to have some LAFS users do some testing with us next week. > I'll make sure that anyone who does it can have 50GB free of > private backend for a few years; if you're on this list as an > enthusiast, dev, or potential dev we'd just like to support the > community so that LAFS can fulfill Zooko+team's vision of > empowering user freedom [he puts it much better than I could > though]. Well, I'm really booked these days, but I'd love to help with user testing. How will it be organized? I recommend sharing a voice connection and screen share and watching the users walk through the entire process. Again, as a Least Authority employee this would also be competitive research. ;-) However, I see a lot of potential for LAFS service providers cooperating. For example, if a single service provider can operate a single grid for all customers, then how much more effort does it take to have a single grid across providers? > [✂ ...] > we'll try > to put together a LAFS for service provider FAQ based on some > of those discussions, and keep future silly questions and > roadmap conversations on the tahoe-dev list. A service provider FAQ would be awesome. Also, if people with experience operating friend nets put together a FAQ for that usage, it would be a great complement. BTW, I have a high tolerance for "silly questions" since I ask many myself. ;-) In other words, don't be shy on the list or on IRC. > > If we get some new-user usage of LAFS, we'll also try to > summarize the questions we get about LAFS use and user/usability > feedback to the list every so often. > That would be invaluable. On this list, sometimes I wonder how many people get tripped up that we never hear from. OTOH, when people are paying for a service, and the service operator's next meal comes from the service, there's much stronger feedback. > In our limited testing the first thing that sysadminnish > people have asked for is the ability to have something like > ssh key passphrases for caps. > Interesting. I wonder if something like this could be designed which also has the "stateless" property of caps, meaning the string of bytes is all that's necessary to access that content (assuming one's on the correct grid). Maybe a passphrase could deterministically generate the cryptographic keys in the cap derivation chain? This would not work for convergent encryption on immutables, but it may be possible for mutables or non-convergent immutables. > Thanks again to the LAFS team and the community... > > [Again as a summary - if you're interested in helping us test > or just adding some free nodes to your play, test, dev, or > prod pool for a few years, please send me an off-list note > and we'll get you info in the next week] > > Thanks, > > Avi (working with the havenco tech team) > Regards, nejucomo > _______________________________________________ > tahoe-dev mailing list > tahoe-dev@tahoe-lafs.org > https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev _______________________________________________ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev