-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all,
This is my first Architecture Team email, so if I'm doing it wrong, please let me know. Francis suggested bringing these concerns to this forum. We have two systems that, broadly speaking, have the same function: the job system and the build system. They both exist to provide a way for Launchpad to schedule and run processes, and gather their output. You might think that the build system's purpose is to build packages, but Soyuz already has put in a bunch of work to generalize it, and it's now able to perform "builds" for Rosetta that don't output a package. This is why it's under lp.buildmaster, not lp.soyuz. (Though shouldn't it be lp.services.buildmaster?) If I had to characterize the differences between the build system and the job system... 1. The build system always runs its code in a chroot, while the job system runs it natively. This means that the job system can have lower startup costs, but cannot build packages for other distros. 2. The build system can run virtualized, which means it can safely run untrusted code. 3. The build system distributes its work across multiple machines, using a central buildmaster. The Job system hasn't been tested on multiple machines, but is intended to use the database to allow multiple machines to coordinate. I think that we should be working to converge these systems, so that we can take advantage of their commonalities. Both are proven, production systems that should be modified carefully. But we can start by converging their data models. I bring this up because Soyuz is currently in the middle of redesigning the build system, and this is therefore a good opportunity to bring more similarity to our systems. The Code Import system is also a candidate for convergence, but the Code team isn't currently redesigning it. AIUI, part of the redesign of the build system involves creating a Buildjob table with many fields that are similar to the Job table. I propose that the buildjob table should hold only fields that are not generally relevant to jobs, plus a reference to the Job table. This may entail adjusting the Job table to better fit the build system's needs. This will give us a common namespace for builds and jobs, so that we can look them up the same way, and make it easier to display them together. It will also give us common representations and vocabulary for those data that are common. As time goes on, I expect the systems will come to depend on each other more. For example, when doing Recipe builds, we could catch problems earlier by having an initial Job that ran the recipe, before handling the actual package generation off to a Build. Having overviews that include both systems will become important. Aaron -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvi4WsACgkQ0F+nu1YWqI03RACfYfmbTQujNvqPZvuDVzsY8caQ GkwAmgOt8wnQ7I+Q21OceXRDjzvDaWR1 =NSi6 -----END PGP SIGNATURE----- _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
