[Emc-developers] the buildbot machine is ready
John Kasunich and I talked about playing some more with Buildbot http://buildbot.net as a way to move the emc2 compile farm forward. We're not going to turn the existing compile farm off (at least not yet); we're going to stand up a parallel system and see if it's good enough to switch over to. I have set up a machine at http://emc2-buildbot.colorado.edu to that end. It's running Ubuntu Server 8.04.1. The CU firewall exceptions just went through, so both its webserver and the buildmaster are accessible directly over the net. The old nappy URL is no longer needed. I'm running two buildslaves as KVM VMs on my laptop whenever i'm in network range, but it would sure help if other folks ran some slaves too. Buildslaves are easy and fun! All you need is a computer that can run python, check out and compile our tree, and make outgoing TCP connections. Slaves can be behind a NAT firewall. Setting up buildslaves is easy. You're welcome to have copies of my VMs, which will run anywhere with only minor tweaks. Or you can install your own by using something like this untested procedure (sorry about the linebreaks): # disable the screensaver or set it to only blank the screen (not soak up CPU cycles with fancy eye candy) # make a dedicated user (by convention i've been calling the user farmer, and that's what this document calls it, but you can use any name you want) # do a cvs checkout as farmer (in a throw-away directory out of the way) to make sure sure you can, and to make sure you have the ssh hostkey of the CVS machine cvs -d :ext:[EMAIL PROTECTED]:/cvs -z3 checkout -d /tmp/remove-this-dir emc2 # install emc2 build deps: libpth-dev, tetex-extra cd debian; ./configure sim dpkg-checkbuilddeps apt-get install build-essential $WHATEVER_ELSE_IS_NEEDED # let the farm user run sudo make setuid without a password by adding this line to /etc/sudoers: farmer ALL = ALL, NOPASSWD: /usr/bin/make setuid # install buildbot.deb, on dapper you need to uncomment universe in sources.list apt-get install buildbot # you get these three variables from the buildmaster admin # on dapper and probably other older versions of buildbot, the command is just slave instead of create-slave # a reasonable standard for build-slave names might be: distro(-rtai)?-arch, for example Ubuntu6.06-rtai-x86 or Ubuntu8.04-x86-64 buildbot create-slave ~/BuildBot/slave/$SLAVE_NAME $MASTER $SLAVE_NAME $SLAVE_PASSWORD # edit buildbot.tac to say usepty = 0 vi ~/BuildBot/slave/$SLAVE_NAME/buildbot.tac # do this whenever you want your build-slave to be running, or (on Ubuntu Hardy at least) add it to /etc/default/buildbot buildbot start ~/BuildBot/slave/$SLAVE_NAME Let's see how this goes! -- Sebastian Kuzminsky how many no money boys are crazy how many boys are raw how many no money boys are rowdy how many start a war - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
Sebastian Kuzminsky wrote: # let the farm user run sudo make setuid without a password by adding this line to /etc/sudoers: farmer ALL = ALL, NOPASSWD: /usr/bin/make setuid This part raises a red flag for me, as I mentioned on IRC last night. If you set this passwordless sudo, then it is theoretically possible for somebody to check a trojan makefile into our CVS, and a few minutes later it would run on your box as root. If your buildbot system is a dedicated virtual machine used for nothing else, the risk is probably tolerable. I would NOT make this change to /etc/sudoers if farmer is a user on a non-virtual machine that you use for other things. The odds of such a thing happening are slim - Joe Hacker can't commit a trojan, only someone with commit access to the server could do it. And, the CVS logs would tell us exactly who it was, so we could give them the beating they so richly deserve. But the risk needs to be acknowledged. Note that the sudo make setuid step of the build process is NOT needed if all we want to do is test compiles. It IS needed if we want to run the test suite, which would be nice to do. (The current compile farm is not running the test suite.) I'm not sure if sim-only system need to run the make setuid step or not. We'll be hashing out the details over the next few days. Regards, John Kasunich - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
On Fri, Nov 14, 2008 at 03:09:43PM -0500, John Kasunich wrote: I'm not sure if sim-only system need to run the make setuid step or not. Nope. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
On Friday 14 November 2008 21:09:43 John Kasunich wrote: Sebastian Kuzminsky wrote: # let the farm user run sudo make setuid without a password by adding this line to /etc/sudoers: farmer ALL = ALL, NOPASSWD: /usr/bin/make setuid This part raises a red flag for me, as I mentioned on IRC last night. If you set this passwordless sudo, then it is theoretically possible for somebody to check a trojan makefile into our CVS, and a few minutes later it would run on your box as root. If your buildbot system is a dedicated virtual machine used for nothing else, the risk is probably tolerable. I would NOT make this change to /etc/sudoers if farmer is a user on a non-virtual machine that you use for other things. The odds of such a thing happening are slim - Joe Hacker can't commit a trojan, only someone with commit access to the server could do it. And, the CVS logs would tell us exactly who it was, so we could give them the beating they so richly deserve. But the risk needs to be acknowledged. They guy has root access to the machine, so he can manipulate the CVS database and obfuscate the commit. Heck, he can even start a telnet/ssh session or whatever. He's root! I don't think there is a solution for this, however. If you want to run a component of the repository (be it the makefile or the setuid programs itself) as root, you need to trust your committer. You could run it in qemu or whatever, but what is it good for then, if you don't have the real hardware access to test RT... (Two seperate machines could be a solution, well...) -- Greetings Michael. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
Michael Buesch wrote: On Friday 14 November 2008 21:09:43 John Kasunich wrote: Sebastian Kuzminsky wrote: # let the farm user run sudo make setuid without a password by adding this line to /etc/sudoers: farmer ALL = ALL, NOPASSWD: /usr/bin/make setuid This part raises a red flag for me, as I mentioned on IRC last night. If you set this passwordless sudo, then it is theoretically possible for somebody to check a trojan makefile into our CVS, and a few minutes later it would run on your box as root. If your buildbot system is a dedicated virtual machine used for nothing else, the risk is probably tolerable. I would NOT make this change to /etc/sudoers if farmer is a user on a non-virtual machine that you use for other things. The odds of such a thing happening are slim - Joe Hacker can't commit a trojan, only someone with commit access to the server could do it. And, the CVS logs would tell us exactly who it was, so we could give them the beating they so richly deserve. But the risk needs to be acknowledged. They guy has root access to the machine, so he can manipulate the CVS database and obfuscate the commit. The CVS database isn't on any machine that a malicious committer has access to. The attacker would only have root access on the slave machine, which uses an anonymous checkout from the CVS server. Heck, he can even start a telnet/ssh session or whatever. He's root! To what? Again, the hypothetical attacker has no login for the CVS server. I don't think there is a solution for this, however. If you want to run a component of the repository (be it the makefile or the setuid programs itself) as root, you need to trust your committer. These scripts don't run on the CVS server, they run on machines that volunteers (like you :) ) would set up. The potential issue is that you, as a volunteer, could allow root access to your machine. That's why John K suggested that a safe thing to do is to use a VM only. You could run it in qemu or whatever, but what is it good for then, if you don't have the real hardware access to test RT... (Two seperate machines could be a solution, well...) Yeah, RT testing opens up interesting issues. What if some RT module crashes the machine? Also the testing can't be comprehensive, since the buildbot machines are unlikely to have any hardware other than a parallel port (if that), so we can't actually test all of the RT drivers anyway. - Steve - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
Stephen Wille Padnos wrote: Yeah, RT testing opens up interesting issues. What if some RT module crashes the machine? Also the testing can't be comprehensive, since the buildbot machines are unlikely to have any hardware other than a parallel port (if that), so we can't actually test all of the RT drivers anyway. If a buildslave crashes, the buildmaster will notice it and send an email to the admin for that buildslave. And man oh man, I would love to set up a big Rube Goldberg machine to do actual physical-real-world testing. Steppers turning shafts with encoders, motors moving levers into switches, etc etc. It'd be awesome! You could have two machines next to each other, one running a trusted version of EMC2, administering the test to the experimental one. You could do actual closed-loop validation of the whole system. Just need a patron to fund house the test system... -- Sebastian Kuzminsky how many no money boys are crazy how many boys are raw how many no money boys are rowdy how many start a war - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
On Fri, 2008-11-14 at 14:14 -0700, Sebastian Kuzminsky wrote: Stephen Wille Padnos wrote: Yeah, RT testing opens up interesting issues. What if some RT module crashes the machine? Also the testing can't be comprehensive, since the buildbot machines are unlikely to have any hardware other than a parallel port (if that), so we can't actually test all of the RT drivers anyway. If a buildslave crashes, the buildmaster will notice it and send an email to the admin for that buildslave. And man oh man, I would love to set up a big Rube Goldberg machine to do actual physical-real-world testing. Steppers turning shafts with encoders, motors moving levers into switches, etc etc. It'd be awesome! You could have two machines next to each other, one running a trusted version of EMC2, administering the test to the experimental one. You could do actual closed-loop validation of the whole system. Just need a patron to fund house the test system... I don't understand this thread that well, but what are the basics of what is needed? A pair of Internet accessible PC's running EMC2, stepper and servo axes? I have a T1, static IP's, second hand PC's and a few other bits. Kirk http://www.wallacecompany.com/machine_shop/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
On Friday 14 November 2008 21:58:56 Stephen Wille Padnos wrote: These scripts don't run on the CVS server, Ok, I thought this would run on the machine running the server. -- Greetings Michael. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
[Emc-developers] hal_vti strangeness
I'm going through the issues Coverity has found in our TRUNK, and while fixing a couple of bugs in hal_vti i noticed something strange. hal_vti.h defines MAX_CHANS to be 4, and that's used a lot in hal_vti.c. But hal_vti.c defines MAX_CHAN (without the S) to be 8, and uses that in one place in rtapi_app_main(). Looks like a bug to me, but maybe Alex Joni or someone who knows the code can check it out and verify. -- Sebastian Kuzminsky how many no money boys are crazy how many boys are raw how many no money boys are rowdy how many start a war - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
Kirk Wallace wrote: On Fri, 2008-11-14 at 14:14 -0700, Sebastian Kuzminsky wrote: Stephen Wille Padnos wrote: Yeah, RT testing opens up interesting issues. What if some RT module crashes the machine? Also the testing can't be comprehensive, since the buildbot machines are unlikely to have any hardware other than a parallel port (if that), so we can't actually test all of the RT drivers anyway. If a buildslave crashes, the buildmaster will notice it and send an email to the admin for that buildslave. And man oh man, I would love to set up a big Rube Goldberg machine to do actual physical-real-world testing. Steppers turning shafts with encoders, motors moving levers into switches, etc etc. It'd be awesome! You could have two machines next to each other, one running a trusted version of EMC2, administering the test to the experimental one. You could do actual closed-loop validation of the whole system. Just need a patron to fund house the test system... I don't understand this thread that well, but what are the basics of what is needed? A pair of Internet accessible PC's running EMC2, stepper and servo axes? I have a T1, static IP's, second hand PC's and a few other bits. Ignore my daydreaming above ;-) What we could use right now is a couple of machines that can check out and compile our software, and can run python code (buildbot and twisted). We dont need static IPs or anything fancy, though those things wont disqualify you. It's good but not necessary if the machines can stay up and be connected to the net for long periods of time. These computers would become buildslaves, which means they'd be pretty similar to what farm slots are in the current Compile Farm. -- Sebastian Kuzminsky how many no money boys are crazy how many boys are raw how many no money boys are rowdy how many start a war - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
Michael Buesch wrote: I don't think there is a solution for this, however. If you want to run a component of the repository (be it the makefile or the setuid programs itself) as root, you need to trust your committer. You can set sudo to allow only specific programs from specific directories to be executed. That includes even specific system command, like cp, mv, mkdir, etc. Jon - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] the buildbot machine is ready
It seems like a buildbot would be a great application to put on a live CD image have it boot and run possibly even no hard drive. It would even be possible to setup a cron job to reboot (view as purging memory based nasties). It is hard to hack a cdrom filesystem!. cheers On Fri, 2008-11-14 at 23:47 +0100, Michael Buesch wrote: On Friday 14 November 2008 23:34:27 Jon Elson wrote: Michael Buesch wrote: I don't think there is a solution for this, however. If you want to run a component of the repository (be it the makefile or the setuid programs itself) as root, you need to trust your committer. You can set sudo to allow only specific programs from specific directories to be executed. That includes even specific system command, like cp, mv, mkdir, etc. Yeah, well. But we _want_ the setuid applications from the emc2 repository to run as root. And a committer has full control over the sources... ;) No matter if you use sudo or some other mechanism. We use sudo in the first place to get setuid root bit set on those applications the attacker has full sourcecode control over. So the question really only is: 1) Do we need these root hacks do do proper tests? 2) If yes, can we setup a sandbox for the test? A an acceptable sandbox would probably only be to reset the complete harddisk image to a known state, make an emc2 checkout/compile/testrun and discard the harddisk contents. -- = Lawrence Glaister VE7IT mailto:[EMAIL PROTECTED] 1462 Madrona Drive Nanoose Bay, B.C.http://members.shaw.ca/swstuff Canada V9P 9C9 http://gspy.sourceforge.net = - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] Quirk when building from CVS trunk with documentation
Jon Elson wrote: Last week I made a CD from the 8.04 iso file, installed on a blank machine, and then did an anonymous checkout of the development trunk. I added all the packages as described in the wiki page for building from source. I added the synaptic repositories, and did : wget http://www.linuxcnc.org/hardy/emc2-install.sh sudo apt-get build-dep emc2 sudo apt-get install build-essential libpth-dev When I did the ./configure --enable-run-in-place --enable-build-documentation I got a message that convert could not be found. When I removed the --enable-build-documentation, it built correctly (but of course without the docs). Not a critical problem, but I thought whoever was responsible for this should know. There are a few extra packages needed to build documentation on 8.04. I think they are: dvips groff tetex-extra imagemagick I had also installed emc2-dev, but I'm not sure what (if anything) it pulled in. I have a list (which I can't access right now to confirm this) that doesn't include imagemagick, but I may have already installed it - it's one of the must have packages for me :) It also may have been from emc2-dev. - Steve - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] Quirk when building from CVS trunk with documentation
Stephen Wille Padnos wrote: [snip] There are a few extra packages needed to build documentation on 8.04. I think they are: dvips groff tetex-extra imagemagick Oops, according to this wiki page: http://wiki.linuxcnc.org/cgi-bin/emcinfo.pl?Installing_EMC2#Building_emc2_with_documents, you need dvipng, not dvips. - Steve - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers