Re: Patchwatcher security improvements
On Mon, 8 Sep 2008, Ambroz Bizjak wrote: Hi, I've abandoned my chroot aproach to improving security in patchwatcher. Instead I've implemented the ability to run untrusted code as a user different than the one running patchwatcher. This is because creating a chroot where Wine could be compiled and tested proved to be too difficult and platform-dependent. This seems like an almost perfect task for a virtual machine: * set up you virtual machine to taste * take a snapshot * to test a patch, fire up the virtual machine * have it test the patch * after the test or when it times out, revert it to the snapshot * rinse (done in the step above), repeat This could be done with VirtualBox, but maybe other alternatives based on Xen or KVM or some such would be better. The main issue I see with this is that the OpenGL / DirectSound tests will not run on the real hardware (as usual), but maybe a Xen-like approach could help there. It would also make it easy to test on FreeBSD / Solaris, at least if based on something like VirtualBox (not sure about the Xen-like approaches). -- Francois Gouget [EMAIL PROTECTED] http://fgouget.free.fr/ Advice is what we ask for when we already know the answer but wish we didn't -- Eric Jong
Re: Patchwatcher security improvements
Francois Gouget wrote: On Mon, 8 Sep 2008, Ambroz Bizjak wrote: Hi, I've abandoned my chroot aproach to improving security in patchwatcher. Instead I've implemented the ability to run untrusted code as a user different than the one running patchwatcher. This is because creating a chroot where Wine could be compiled and tested proved to be too difficult and platform-dependent. This seems like an almost perfect task for a virtual machine: * set up you virtual machine to taste * take a snapshot * to test a patch, fire up the virtual machine * have it test the patch * after the test or when it times out, revert it to the snapshot * rinse (done in the step above), repeat This could be done with VirtualBox, but maybe other alternatives based on Xen or KVM or some such would be better. The main issue I see with this is that the OpenGL / DirectSound tests will not run on the real hardware (as usual), but maybe a Xen-like approach could help there. It would also make it easy to test on FreeBSD / Solaris, at least if based on something like VirtualBox (not sure about the Xen-like approaches). Yep. Virtualizaion has 3D shortcomings. I can see the way how to use pbuilder/pdebuild toolchain on dedicated user account in Debian to automate this in pretty safe and easy way. pbuilder uses fakeroot/chroot for this and its use is a nobrainer, hellish easy and effective. But this is limited to Debian systems only. Positive is that we still have access to 3DHW (although not concurrent/parallel). Anybody has experience with User-mode Linux kernels for that? ~*~ Another environment is OpenSolaris. There we can leverage technologie of zones ZFS for cheap pseudovirtualization and fast FS recovery using FS snapshots. ~*~ IMO there is no silver bullet to bite all problems on all OS. We can build OS-specific toolchains around patchwatcher and I think that's more viable alternative. Cheers Hark
Re: Patchwatcher security improvements
On Wed, Sep 10, 2008 at 4:37 AM, Francois Gouget [EMAIL PROTECTED] wrote: This seems like an almost perfect task for a virtual machine: ... The main issue I see with this is that the OpenGL / DirectSound tests will not run on the real hardware (as usual) I just came off a project (Zumastor) which used a virtual machine in its test loop as you suggest. We ended up using qemu because kvm was too broken at the time. The whole experience left a bad taste in my mouth, so I'm pushing ahead with a more lightweight approach to be able to make progress on the key problem. If somebody wants to run patchwatcher inside VMs, great, that's definitely a safe way to go, and events might push us in that direction someday.
Re: Patchwatcher security improvements
On Wed, Sep 10, 2008 at 5:06 AM, Vit Hrachovy [EMAIL PROTECTED] wrote: I can see the way how to use pbuilder/pdebuild toolchain on dedicated user account in Debian to automate this in pretty safe and easy way. pbuilder uses fakeroot/chroot for this and its use is a nobrainer, hellish easy and effective. But this is limited to Debian systems only. Positive is that we still have access to 3DHW (although not concurrent/parallel). Yes. We used pbuilder in the automated test for zumastor, and were tied to Debian as a result. We obviously need to avoid requiring that for patchwatcher, which has to run on non-Debian systems. (BTW, we had some difficulty with unreliable distribution mirrors; the only way to get pbuilder to be reliable was to point to a local archive of all the packages.) Anybody has experience with User-mode Linux kernels for that? That's getting even further away from the hardware... IMO there is no silver bullet to bite all problems on all OS. We can build OS-specific toolchains around patchwatcher and I think that's more viable alternative. Indeed.After I finish refactoring patchwatcher, the build slaves will be pretty simple, and it'll be easy to put together custom build slaves for various environments. In particular, a pbuilder-based build slave for Debian / Ubuntu seems like a good idea (as long as you use a local package archive to avoid the flakiness I mentioned above). - Dan
Re: Patchwatcher security improvements
On Wed, Sep 10, 2008 at 5:02 AM, Dan Kegel [EMAIL PROTECTED] wrote: On Wed, Sep 10, 2008 at 4:37 AM, Francois Gouget [EMAIL PROTECTED] wrote: This seems like an almost perfect task for a virtual machine: Incidentally, I documented how to produce a really small vmware image for Ubuntu at http://kegel.com/linux/jeos-vmware-player/ (I used this as a demo platform for Zumastor, and wanted the demo to be as easy to download as possible.)
Re: Patchwatcher security improvements
Francois Gouget wrote: This seems like an almost perfect task for a virtual machine: * set up you virtual machine to taste * take a snapshot * to test a patch, fire up the virtual machine * have it test the patch * after the test or when it times out, revert it to the snapshot * rinse (done in the step above), repeat This could be done with VirtualBox, but maybe other alternatives based on Xen or KVM or some such would be better. The main issue I see with this is that the OpenGL / DirectSound tests will not run on the real hardware (as usual), but maybe a Xen-like approach could help there. It would also make it easy to test on FreeBSD / Solaris, at least if based on something like VirtualBox (not sure about the Xen-like approaches). Seems like a good idea. I suggest only to use Linux as the host OS and run guests with KVM or VirtualBox, or even UML for Linux builds. Then we can run any OS that uses X as a guest and configure it to use the host's X display (through the virtual network). This should theoretically allow OpenGL through the XGL protocol, but VMGL can also be used if it doesn't work well. And sound hardware would be emulated by the VM software. I think I'll try getting a small Gentoo system to run in UML with a read-only root fs and make it boot as fast as possible. To try a patch, I would give it read access to the master Wine tree on the host, it would copy it to a writable temp folder and try it out. After it's finished or if the external timeout elapses, the UML process will be terminated and all of its writable storage will be reverted.
Re: Patchwatcher security improvements
Ambroz wrote: I think I'll try getting a small Gentoo system to run in UML with a read-only root fs and make it boot as fast as possible. To try a patch, I would give it read access to the master Wine tree on the host, it would copy it to a writable temp folder and try it out. After it's finished or if the external timeout elapses, the UML process will be terminated and all of its writable storage will be reverted. Right. That's how the refactored patchwatcher is designed. There's a shared directory containing one subdirectory for each build slave. Each slave is expected to somehow get a read/write mount to its own subdirectory of the shared directory. The master watches the mailing list and puts incoming patches into an inbox directory. Each patch series is called a job, and gets its own subdirectory of inbox. The master dispatches a job to a build slave by moving the job directory into one of the build slave's directories. The build slaves watch for jobs to appear in their directory. When one appears, then do all the builds it implies, then create a log file. The master notices the log file and moves that job out of the slave's subdirectory. So the slave can be in another real machine, another virtual machine, or running as another user; anything as long as it can get read/write access to its subdirectory of the shared directory. - Dan
Re: Patchwatcher security improvements
Dan Kegel wrote: So the slave can be in another real machine, another virtual machine, or running as another user; anything as long as it can get read/write access to its subdirectory of the shared directory. The problem with your design right now is that you want to run the slave in some isolated environment and expect it to be secure. The build slave itself is a mission-critical process and putting it in a quarantine to run together with untrusted code allows malicious patches to interfere with its operation. This means an attacker can just kill it from inside his patch, causing the whole patch building operation to fail, or corrupt the baseline tree, or send hundreds of fake emails through the slave interface. So I plan to run the build slave itself in a trusted environment, but make it quarantine individual build operations (similar to my previous design with user switching). This way the impact of an attack is highly limited - all it can theoretically do is fake his own patch results.
Re: Patchwatcher security improvements
Ambroz wrote: The problem with your design right now is that you want to run the slave in some isolated environment and expect it to be secure. The build slave itself is a mission-critical process and putting it in a quarantine to run together with untrusted code allows malicious patches to interfere with its operation. This means an attacker can just kill it from inside his patch, causing the whole patch building operation to fail, or corrupt the baseline tree, or send hundreds of fake emails through the slave interface. It can't directly send fake emails, since the build slave doesn't have the email password, but it could certainly disrupt the build slave and make it give bogus and malicious results. The design isn't for security, it's for ease of prototyping and plugging in new build slaves. So I plan to run the build slave itself in a trusted environment, but make it quarantine individual build operations (similar to my previous design with user switching). This way the impact of an attack is highly limited - all it can theoretically do is fake his own patch results. Yes, good, just please don't change the interface to the build master, all your changes should be encapsulated in a custom build slave. - Dan
Re: Patchwatcher security improvements
On Mon, Sep 8, 2008 at 9:01 PM, Austin English [EMAIL PROTECTED] wrote: On Mon, Sep 8, 2008 at 6:06 PM, Dan Kegel [EMAIL PROTECTED] wrote: Interesting.One of my goals is to support Solaris and BSD; have you tried your stuff there? What about OS X? Yes, of course.
re: Patchwatcher security improvements
Interesting.One of my goals is to support Solaris and BSD; have you tried your stuff there? I'm currently refactoring patchwatcher.sh; I've pulled the generic stuff out into libpatchwatcher.sh and the wine-specific stuff into wine-slave.sh. Your changes will fit nicely into wine-slave.sh, I hope. I'm surprised you had to give up on the chroot... I was planning on trying to run just wine-slave.sh in a chroot jail, since it's the only part that would actually try to run any part of the wine build system. - Dan
re: Patchwatcher security improvements
Interesting.One of my goals is to support Solaris and BSD; have you tried your stuff there? Not yet, but that stuff is pretty generic, so it shouldn't be hard to get it to work. I'm surprised you had to give up on the chroot... I was planning on trying to run just wine-slave.sh in a chroot jail, since it's the only part that would actually try to run any part of the wine build system. Creating the chroot itself is really hard and has to be done for each platform separately. The basic system and also all development tools and Wine dependencies have to be copied properly. And there are the tests with their own bunch of requirements. In the end you would end up with a chroot that is not much different from the base system itself. And it doesn't really bring many security benefits. Many potentially insecure interfaces have to be exposed anyway (/proc, X server with OpenGL, sound hardware). If a clean and dedicated system is used and permissions are properly configured, running stuff on the base system shouldn't really be a problem. Furthermore, using some advanced access control system (like SELinux) would probably be easier to configure and more efficient.
Re: Patchwatcher security improvements
Also, it's possible some of your changes won't be needed after the refactoring... I plan to run wine-slave as a different user anyway... That doesn't solve much; although in may look clean, it is not secure. The user should have a limited amount of resources to work with. Your way, for example, it can write the whole master Wine tree. With my patches, the master tree is read-only for the user, and it only has its own copy to work with which is never used again. I plan to further improve things. In particular, killing stall processes is not implemented securely now. As I have already mentioned, additional access control is needed to produce a fully solid system. For example, disk access should be limited (think about world-writable folders and stuff like ~/.bashrc), and memory usage should be limited as well (could patchwatcher get killed when the patch starts consuming memory?). Considering the refactoring, I see you are just some moving stuff into its own file; I can easily adjust my code.
Re: Patchwatcher security improvements
On Mon, Sep 8, 2008 at 6:06 PM, Dan Kegel [EMAIL PROTECTED] wrote: Interesting.One of my goals is to support Solaris and BSD; have you tried your stuff there? What about OS X?
