TCC 2014-10-02, Synopsis
attending: amiller, daira, juan, vitalik, nathan, za, zooko Zooko started with a Layered Architecture perspective on tahoe using bits of this presentation: https://tahoe-lafs.org/~zooko/RealWorldCrypto2014/build/slides/#15 Layers: ~~~ 0. Storage 1. Network 2. Routing 3. Agoric (Social/Economic) note: agoric layer at the top which is not implemented Layer 0, Data Storage and Security: --- The guiding principle for this layer is that references and authorities are unified as capabilities. Data stored in tahoe is either mutable or immutable. Both types are associated with a set of 'diminishing' capabilities, where for mutable data there exists a write capability a read capability and a verifying capability. These capabilities are orderd from strongest to weakest and weaker capabilities are derivable from stronger ones, but not vice versa. Immutable data does not have write capabilities. Immutable Capability Implementation Details: The following is mainly a summary of the detail in this reference: http://eprint.iacr.org/2012/524.pdf 0. verify capability contents: * a sha256d of: 0. the root of a merkle tree over the cyphertext 1. the root of a merkle tree over the erasure coded shares 1. read capability contents: 0. the verify capability 1. the symmetric encryption key used to encrypt the file Mutable Capability Implementation Details: '' 0. verify capability contents: 0. a sha256d of the public verification key from the signing key-pair 1. a merkle tree over the shares 1. read capability contents: 0. the verify capability 1. sha256d of the private signing key from the signing key-pair (see next capability) (you can derive the encryption key from this) 2. read-write capability contents: 0. the verify capability 1. sha256d of the private signing key from the signing key-pair 2. the AES-CTR encrypted signing key from the signing key-pair An interesting property is that one RSA authenticates the clear text after decryption, to prove the write secret was held by the encrypter. Daira notes: Use a discrete log system should be used for this because of its efficiency. Layer 1, Network Layer: --- An overview of the fundamental topology of a tahoe grid: https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/network-and-reliance-topology.svg The most common interface to tahoe: https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/frontends/webapi.rst * Nathan: Note that CAPS in tahoe depend on the network layer in some cases? There is/Is there layering conflation/violation? * Daira: Current interfaces between the crypto and network layers are not cleanly separated. Write enablers formerly derived from storage_server TUBID, now it's simply a shared secret. Also formerly lease renewal IDs. I am unsure whether the answer to Nathan's question was: * No, capabilities are now generated independently of networking layer data. OR: * Yes, though we've removed the conflation from write-enablers, some issues still remain. https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/frontends/webapi.rst There are plenty of problems with the tahoe network protocol. Layer 1a, Cloud Backend: Daira: The bottom 2 nodes depicted in the network-and-reliance-topology.svg are part of the cloud-backend, which interfaces with standard cloud storage provider APIs, to use them as tahoe storage nodes. https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/specifications/backends/raic.rst Note the bottom two nodes in the network-and-reliance-topology.svg. Layer 2, DHT/routing: - All-to-all: ''' every client attempts to be aware of every servers' state. Server selection: ' is based on a random permutation of the servers which is a function of the data being transferred. -- This is called rendezvous hashing. Brian starting using it 7 years ago. Layer 3, Agoric: This term was coined by Mark Samuel Miller Amiller: Are there examples of projects using only Layer 0? Zooko: No. People should/could innovate on the DHT layer, and keep the network layer. Accounting: ''' Problem: When a client asks to up/down-load, there's no current tracking of transactions. Solution: Each storageserver keeps a record of which requests have been made. As a function of requests (or other factors) storageservers implement economic policies Nathan: There is certainly not a clean separation from layer 2 here. Daira: How was this done at MojoNation? Zooko: Believed that the Agoric layer was the failing of MojoNation. Had a rolling second-price auction. - on request: start a timer - if timer runs out: honor
Re: Tahoe-LAFS cryptoparty at Noisebridge (San Francisco, CA)
I hope to attend virtually. On Sat, Sep 6, 2014 at 12:10 PM, David Stainton dstainton...@gmail.com wrote: Dear Tahoe-LAFS enthusiasts, I am organizing a Tahoe-LAFS cryptoparty at Noisebridge in the next coming weeks and I'd be delighted if some of you would attend. The explicit goal of this cryptoparty is to show participants an alternative to the inferior mainstream cloud storage... and create a collectively operated Tahoe-LAFS grid which is only accessible via Tor Hidden Services. https://www.noisebridge.net/pipermail/noisebridge-discuss/2014-September/045259.html https://www.noisebridge.net/pipermail/noisebridge-discuss/2014-August/045127.html We have not decided upon a date for this event yet. Cheers, David Stainton contact info → https://www.lumiere.net/~mrdavid/contact.txt ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Tahoe Meetup
We're planning a physical meetup at the Longmont Hackerspace this Saturday at 13:00. http://www.meetup.com/LongmontHackerSpace/events/201949142/ Hopefully some of you can make it! I have a few confirmations already. Feel free to RSVP! -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: upcoming Tahoe Summit: Jun30+Jul01, SF
Interested! Don't know if I can make it. On Tue, Jun 17, 2014 at 5:43 PM, Brian Warner war...@lothar.com wrote: We're trying to assemble a small Tahoe Summit for 2014 in a few weeks. It'll happen in San Francisco, on Monday 30-Jun and Tuesday 01-Jul, in the evenings (say 6-10pm). We're currently looking into gathering at Noisebridge, and then eating/drinking at the Sycamore across the street. Daira is organizing it and will be in town. I (Brian) will be there too. If you're interested and can make it, please respond to this message so we can get a sense of how many people will be there. yay summit! -Brian ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: If the machine run the introducer down,what can i do
On Sun, Mar 9, 2014 at 7:59 PM, xiao_s_yuan xiao_s_y...@163.com wrote: If the machine run the introducer down and can not run again,is that means I should create a new introducer on the other machine and change the furl in all the storage and client nodes? And If I can still find the file or directory in the grid? ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev I think you are correct, you need a new introducer, and the nodes in the grid need to know its furl. -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] src to generate tahoe.cfg
On Thu, Jun 27, 2013 at 8:32 PM, Callme Whatiwant nejuc...@gmail.comwrote: It looks like the code lives here: ./src/allmydata/scripts/create_node.py There are various string templates there which are used to create an initial config file. I'm curious why you choose to automatically modify the config file? I'm not sure about the core developers, but I prefer to keep a clean separation between files which are only read by lafs and only edited by humans versus files which are read/written by lafs but not edited by humans (except for unusual situations). Could the feature you want to implement be a modification of the main tahoe program to implement some dynamic feature? Regards, Nathan ps: I discovered the file above with this (on a linux system inside the repository checkout directory): $ find ./src/ -type f -name '*.py' | xargs grep 'tahoe.cfg' Nice pattern. -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] py-zfec on github now includes darcs history
On Sun, Jun 9, 2013 at 3:33 PM, Richard Kiss h...@richardkiss.com wrote: I realized I should just rebase on the darcs history import I produced, so the github repo at https://github.com/richardkiss/py-zfec is now rebased with the darcs history. In darcs, the top level included misc and zfec. I moved everything in zfec to the top level so it would be a little more standard, and so the README.rst file would display. This might break build scripts, but if so, they should be easy to fix (substitute zfec/zfec = zfec). Feel free to use this as a github base (ie. if you do a clone and push, I'll delete my version). -- http://richardkiss.com/ I don't know which tool you used to port darcs-to-git, and you may already be aware of this, but if you're not, you should know that there is a discrepancy in the way that darcs records patches, and git records commits. Git does not record commits of empty trees. A git blob is a sha1 indexed object that includes (among other things) the data in a file. A git tree is an object with a reference to a blob, or to another tree. An important caveat is that there must be at least one blob referred to by a tree (even if the reference is indirect via multiple trees), for that tree to exist. In git file contents are mapped into blobs, and directory contents are mapped into trees. Unfortunately, this means that there is not a map from an empty directory into a git object, because an empty directory contains no files-or-directories, and git-trees are required to refer to at least one blob. In darcs on the other hand an empty directory can be added as its own patch. The result of this discrepancy between data models is that porting tools must make a decision about how to report darcs patches that correspond to the addition of a single directory. I don't know if this is an issue in the zfec code base, but it's something that porters ought to be aware of. -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
Re: [tahoe-dev] perl/ruby/sed scripts?
On Mon, Jan 21, 2013 at 3:28 PM, Tom Vecchione tvecchi...@yahoo.com wrote: Hi: Has anyone written any perl/ruby/sed (etc) scripts to modify a number of tahoe.cfg files in place on a number of tahoe storage/client servers? If so, is there a central repository for helper scripts like this? I seem to remember Zooko mentioning something like this a couple of weeks ago. Thanks, Tom V ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev Tom, here's some of the functionality you're looking for (written in Python): https://github.com/zancas/leastauthoritymainserver/blob/80a2a445552698c48c8798e7c609965e0aea815e/lae_automation/server.py#L22 and: https://github.com/zancas/leastauthoritymainserver/blob/80a2a445552698c48c8798e7c609965e0aea815e/lae_automation/server.py#L307 (Most of this code was written by davidsarah, but we haven't yet formally migrated our old (darcs) repository to Git yet, so attribution is inaccurate.) The TAHOE_CFG_TEMPLATE string is a tahoe.cfg file template that is filled out via string interpolation in the bounce_server function. The bouncer_server function then makes calls to run and write which are in turn wrappers around functionality found in the fabric remote system administration package: http://pypi.python.org/pypi/Fabric/1.5.2 I could adapt the above to something a little more specific to your needs if you'd like (and I find time!). I'm certain there are other solutions floating around, I'd be interested if you find a better one! -- -- ظ ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
[tahoe-dev] Physical Meetup 2013-12-10: Minutes of the Meeting
Summary: This was a brief meeting. The technical content was focused on the Lease Database documentation. https://github.com/davidsarah/tahoe-lafs/blob/5161b1fb8dea9ad9dce23db5f8f7c89c3d7ea82d/docs/proposed/leasedb.rst Discussion: freddyb (having reviewed the document prior to the meetup) felt that an understanding of the pre-leasedb architecture was implied by the format of leasedb.rst. warner mentioned a Theory of Operation (I believe this is what he was referring to: https://en.wikipedia.org/wiki/Theory_of_operation) and suggested the following format: 1) motivation 2) description of the Steady State operating result 3) subsequent description of edge cases (e.g. startup) Zooko suggested that the section currently titled Motivation should be an appendix. (I'm unclear on whether or not this is consistent with the format warner advocated.) Warner described some aspects of peer-to-peer communication in the context of Tahoe to Tom, at the edge of my hearing (i.e. I have no details to report). freddyb and I discussed the notion of a share he said (approximately): Your data is divided into a set of shares a subset of which is sufficient to recover that data. [Caveat Emptor: We didn't talk about the fact that the subset is not strict, in the sense that you may need _all_ of the shares. For example, in the LeastAuthority TLOS3 case there's only 1 share. We assume decryption capability.] Our conversation was stimulated by my experience scanning the document related code in preparation for the meeting. The following assumes immutables as context: Before the Lease Database existed leases were stored with the data in ShareFiles. https://github.com/tahoe-lafs/tahoe-lafs/blob/a9272522d5aff5342d08892992bf669b047c17fe/src/allmydata/storage/immutable.py#L39 These ShareFiles then provided methods both for data management (reading, writing, etc.) and for _meta_data management (renew_lease, etc.). There did not exist a class implementing an interface for a share as an object independent of the ShareFile. The accounting architecture permits the excision of the ShareFile class. Classes implementing the IShareBase, IShareForWriting, and IShareForReading interfaces replace the data management functionality, while the AccountingCrawler class handles lease data. Because the storage media is abstracted away in the cloud backend there are different types of IShareBase implementing classes: https://github.com/davidsarah/tahoe-lafs/blob/5161b1fb8dea9ad9dce23db5f8f7c89c3d7ea82d/src/allmydata/interfaces.py#L476 The AccountingCrawler is defined in the allmydata/storage directory as it uses the abstract backend object to interface with backends in general: https://github.com/davidsarah/tahoe-lafs/blob/5161b1fb8dea9ad9dce23db5f8f7c89c3d7ea82d/src/allmydata/storage/accounting_crawler.py#L13 In short, as reflected in the document the commonly understood concept of the Share is now more closely reified as a IShareBase than was possible in the pre-accounting architecture. We did not discuss the states of a share and the possible transitions between them. I propose those topics as the agenda for the next meetup! Changes to the content of the document as a result of the meeting are here: https://github.com/zancas/tahoe-lafs-1/blob/bc0e3766805f3f8842b6158ccf0b49d44d09ef30/docs/proposed/leasedb.rst -- -- ظ P.S.-- Dinner afterwards was great fun! ___ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
