Thanks for the feedback, Steve. There's a bigger picture issue here we have to think about. There's two ways you can approach this design. First, you can imagine the "perfect world" scenario, namely if you were given a green field, and you built things without pre-existing boundary conditions, how would you want it all to look. This is not academic -- it's the ultimate goal you're shooting for, even though we all know "perfect" doesn't exist, that where we set the proverbial bar. Second, you look at how existing real world environments are deployed, with all the flaws, and you figure out how to make your software work for the crap people currently have deployed. This is the "real world" approach.
My strategy is pretty always to dream big (the perfect world), but code for reality. We want to the code to encourage and support the best possible configuration, but we have to make it possible to leverage EFS in the world people have today. Now... on to specifics. I think we need to assume that we will have absolutely zero control over the Kerberos realm or the PTS database for the near- to mid-term. While I would love to extend to EFS to support building and managing new OpenAFS cells from the ground up, I think that's a very long way off. That will fall into the "perfect world" category. For now, our focus will have to be on coding EFS to adopt to the environments deployed by whoever the early adopters are, but ALL of that work will have to done in the context of a longer term design. I hope that makes sense. Regarding the uid/gid requirement -- that's really a requirement for using EFS with NFSv3, which is all it supports today. I think that requirement can be relaxed in the early OpenAFS-based implementations, simply because we'll be using native AFS security (everything will be ACL based -- the code won't give a damn what uid/gid is on the contents of /efs anymore). Let me sum up some of the initial assumptions I think we need to work with as we approach this: (1) EFS is going to have minimal administrative control over OpenAFS We won't be managing PTS, we will just be using it. We will only know about the fileserver and vice partitions that the administrator defines in EFS, not necessarily the entire cell. That means that the initial release is NOT going to have all the native cell-management stuff I plan to rewrite eventually (remember vldbaudit, ptsaudit, etc?). (2) EFS will assume that each OpenAFS cell is independent of the others We can't do this with NFSv3 -- we have to assert and assume that all of the "cells" are part of the same "realm", by which I mean: uid/gid namespace. This is why the NFSv3 implementation is insanely insecure: noone really meets that requirement, even in the current EFS 2 production environment (don't get me started...) We are going to need to model: (2.1) One Kerberos realm, Multiple AFS Cells with Integrated/synced PTS (2.2) One Kerberos realm, Multiple AFS Cells with Independent PTS (2.3) Multiple Kerberos Realms, Multiple AFS Cells with Independent PTS I think in the OpenAFS implementations we can *almost* ignore uids/gids. They are used to grant access and lock down releases in the NFSv3 case, and that logic will change entirely to ACLs and PTS groups/users (preferably groups, of course). Because the support for cross-realm trust is mature enough for both Kerberos and OpenAFS, I think it becomes possible to securely configure inter-domain distribution of data, although I don't think that will scale globally to the public Internet. I'm really think of (2.3) for sites that use multiple INTERNAL Kerberos realms to manage their environment. Now, the above discussion is academic, since we really need to get an idea of what the real world environment looks like at the sites that are candidates for adopting EFS. We can talk for months about the 1000 different ways one can or should configure all of this, but we need to focus on the few real world scenarios that are first in line. On Sat, Sep 25, 2010 at 12:11 AM, Steven Jenkins <[email protected]>wrote: > On Fri, Sep 24, 2010 at 1:28 PM, Phillip Moore > <[email protected]> wrote: > > I spent the morning working on this brain dump: > > http://docs.openefs.org/efs-core-docs/DevGuide/Future/OpenAFS.html > > Right now, there's a race on between the two sites that are trying to be > the > > first to bring up an EFS 3 domain, outside of my team (actually -- the > first > > other than me, personally, I think). > > One of those sites wants to use EFS for OpenAFS environments, and there > is > > nothing I personally want more than to get work with OpenAFS again, so > I'm > > rooting for them :-P > > Anyway, check out the doc if you have a few minutes. It's FAR from > > complete, and really just a brain dump, but it's touches on the bulk of > the > > issues we're going to have to figure out in order to make this work. > > And make it work is precisely what I intend to do.... > > I've put together some thoughts. I'd be happy to provide a git diff > of your doc if you prefer. Otherwise, read on... > > Disclaimer: this was a brain-dump. There is clearly more work/thinking > that > needs to occur. > > The 3 inter-related issues of EFS domains, Kerberos realms and uid/gid > mappings across domains and realms is really about stitching those > together. > There are several projects in OpenAFS (or underway) that will probably > help address this: > > - existing cross-(Kerberos)realm work in OpenAFS. Currently, there is > some limited cross-realm support (documentation is in the krb.conf and > krb.excl man pages for OpenAFS). By using that support, names from > foreign realms can be treated as local. Assuming all AFS ids are > in sync across each cell, one can then configure each cell to trust the > other cells (assuming that a cell maps to a Kerberos realm). > > That's a pretty unrealistic scenario. It might be useful in some > cases, but it won't be useful in the assumed more general case where > the cells have not been centrally managed and thus AFS ids are not > in sync across cells. > > - PTS extensions: c.f., > https://datatracker.ietf.org/doc/draft-brashear-afs3-pts-extended-names/ > to provide a mechanism to map among AFS cells and Kerberos realms, > as well as help with inconsistent uids in those cells & realms. > > Based on that, we should be able to view N realms as a logical whole > (e.g., by first defining a 'canonical' mapping of uids, then building > a mapping database. Note that with the krb.excl file, we can exclude > id's in certain realms, so a migration could conceivably happen in > a controlled fashion). > > At a rough glance, these will get us very, very close. Someone should > touch base with Derrick and do some proof-of-concept mappings to verify > this. I don't know the status of that work -- Derrick mentioned today that > he has some code, but it's not ready for anyone else to start playing with. > > - Various people who have played with using AD (or LDAP) as the > backing store for PTS. There are also other ways to solve this problem > that > have been discussed but aren't necessarily in the 'here's some code' stage. > These projects may well play a part in a solution set to the cross-realm, > cross-cell, inconsistent uid namespace problem. > > Misc notes: > > 1- We shouldn't require uid/gid consolidation to occur as a prequisite to > adopting OpenEFS -- the organizational maturity bar for that is simply too > high. > > 2- It's worth writing up a sample document describing how we want > migration from your current multi-cell, multi-realm environment to EFS to > occur. > > Creating mount points: more recent versions of OpenAFS (i.e., virtually > anything that will be found in production) support dynamic mounting of > volumes via a magic syntax (the exact syntax escapes me at the moment -- > I've not used it but only seen it mentioned a few times and was > unable to locate the actual syntax) > > e.g., > /afs/example.org:user.wpmoore > > would be a path that would automagically mount the volume user.wpmoore. > > so the necessary fs mkmount could be done as follows: > > fs mkmount /afs/example.org:user.wpmoore/some/path some.volume > > without requiring any special pre and post mount hacking. > > rsync: my understanding is that incremental vos dump/restore > is quite a bit better now (if I recall correctly, in 2008, Ali Ferguson > from Morgan said at the OpenAFS workshop that Morgan was using it and > that he was confident that the bugs had been shaken out of it, but that > was after numerous failed attempts over the years). > > It would be useful to track pros and cons of volumes being per domain or > per cell. I don't think most people can make an argument either way (i.e., > the number of people who can seriously discuss the tradeoffs is, well, > tiny). I think we need to translate this to more 'non-insider-language' > so that the various users can weigh in on how they would need this to work. > Off the cuff, I don't see why anyone would really want per-domain over > per-cell, but there may be some failure scenarios where it would be useful. > > > Steven > _______________________________________________ > EFS-dev mailing list > [email protected] > http://mailman.openefs.org/mailman/listinfo/efs-dev >
_______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev
