On Mon, Oct 15, 2012 at 6:36 AM, Chris Webb <ch...@arachsys.com> wrote: > > Whilst we can mount the shares on each host and then use qemu's 9p > passthrough/proxy support to access the mountpoint, going via the host > kernel and vfs like this feels quite inefficient. We would be converting > back and forth between vfs and 9p models several times needlessly. > > Instead, I'm wondering about the feasibility of connecting the 9p stream > directly from qemu's virtio-9p-pci device to a socket opened on a > 9p-over-TCP export from the fileserver. Am I right in thinking that qemu's > -fsdev proxy gives me access to a file descriptor attached to the 9p stream > to/from the guest, or is the protocol between virtfs-proxy-helper and qemu > re-encoded within qemu first? >
A passthrough makes perfect sense, a couple summers ago we had an Extreme Blue team working on using 9p for a cloud hosting environment -- while they were primarily working on gatewaying through a host operating system we also discussed doing 9p passthrough (primarily for test, but my other motive was looking at direct 9p to back-end server connections). I'm copying that team on this message to see if they have any additional thoughts. On one end you lose a bit in that you are no longer taking advantage of the host file system cache, which can be useful, particularly if there is any consolidation among the different guests -- but as you point out, you eliminate several copies and transitions through kernel space by just going direct. > Secondly, assuming I can somehow get at the 9p streams directly (either with > an existing option or by adding a new one), I'd like to restrict guests to > the relevant user's subdirectory on the fileserver, and have been thinking > about doing this by filtering the 9p stream to restrict 'attach' operations. There's all sorts of magic you can work here, almost all of it can be implemented on the file server assuming how much you trust your guests and the intermediate host. I'm by no means a security expert, but there are three relatively easy paths: a) start a server instance per user on the file server ahead of time, while this is a little obnoxious, it's by far the quickest path -- you control the port that the virtual images connect to, so as long the user is only able to connect to "his" file server, you are good. There are all the issues involved with uid mapping/etc. on the server side, but there are multiple ways of securing the file server to constrain the user to his/her own hierarchy. Some of the uid mapping and other security techniques in the qemu server could probably be extracted to their own stand-alone server relatively easily. b) have qemu snoop and validate attach operations -- this may be what you were suggesting. Essentially you can hardcode the attach to only validate from a single user (or restrict it to a set of users). An alternative is to overload protocol semantics and have the initial version & attach (which could be sent by qemu) carry some significance with the server -- hardcoding the protocol parameters and user under whose authority all subsequent requests fall under. This leaves much of the implementation details to the server c) you can use the authentication mechanisms within the protocol (Tauth/Rauth and the afid) to independently authenticate users on the server. There are some examples of this in the xcpu code and of course in the original Plan 9 server/client/auth system. This is probably the most work intensive, but would be protocol and gateway (in this case qemu) neutral -- putting most of the work on the client and server > Fortunately, 9p uses client-chosen fids rather than server filesystem inode > numbers which would immediately scupper any simple attempts to implement a > secure chroot proxy of this kind. Looking at the 9p2000.L protocol, it > doesn't look obviously difficult, but I've not really worked with 9p before, > and could well be missing security complications. (I'm not sure whether > there's risk of symlinks being interpreted server side rather than client > side, for example.) The embedded server in qemu should have all the bits you need to restrict hierarchy, you can alternatively use private namespace and/or chroot games to further guarantee isolation -- but since the qemu server also deals with the uid mapping issues, it might be the better starting point since the team that built it was looking at doing something very similar to what you want to do (albeit through proxying a host mounting distributed file system). Good luck, and feel free to ping me with any 9p questions, I may be less helpful on any qemu side implementation details I'm afraid. -eric