-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Dan Williams wrote: > On Tue, 2006-10-10 at 11:19 -0400, John Richard Moser wrote: > > > Dan Williams wrote: >>>> On Mon, 2006-10-09 at 22:49 -0400, John Richard Moser wrote: >>>> Right now I'm trying to get pilgrim to work and build good dev images on >>>> a freshly installed Fedora Core 5 system. There are a number of things >>>> I need to do once I get this working. >>>> >>>> - First off, document it, because it's not on the wiki that I can >>>> find. >>>> >>>> - Get a list of the RPMs used to build the image >>>> >>>>> You can look at the build logs for the builds (if they are available) or >>>>> you can look at the build logs for the builds you do with the tools. >>>>> Yum will tell you exactly what's getting installed into the chroot. > Huh.. the bootlog for me tells me a lot of junk was installed but no RPM > names. I probably can divine the names out of it. > > For reference > http://olpc.download.redhat.com/olpc/streams/development/build94/devel_jffs2/build.log > is what I'm looking at for an example build log; mine is not working, > it's spitting out broken crap. :( > >> Huh? It's all there; look for "Dependencies Resolved" and everything >> below that until "Transaction Summary" is the stuff that's going to be >> installed in the chroot. > >> avahi-tools i386 0.6.11-6.fc6 olpc_rhbase 56 k > >> These lines tell you a few things; that avahi-tools, version >> 0.6.11-6.fc6 from the repo olpc_rhbase is being installed, and that it's >> the i386 version of the package. That should be all you need to get the >> RPM name that's being installed: > >> <name>-<ver/release>.<arch>.rpm > >> avahi-tools-0.6.11-6.fc6.i386.rpm > Ah, yes thanks. I'm a debian guy, I don't know how to abstractly construct RPM names from that stuff :) >>>> - Script converting those RPM names to URIs of SRPMs >>>> >>>>> The RPMs come from 3 different repositories: Fedora Core Development, >>>>> Fedora Extras Development, and OLPC Development. You can get the the >>>>> SRPM name from which the RPM was generated using a bit of python. >>>>> Remember that one SRPM can generate > 1 RPM :) I can point you to the >>>>> right places for the Python code if you'd like. > Yeah I know no python (accidentally hacked some a while back but haven't > touched it since), so pointing me at a chunk of code would be useful. :) > >> That would be _so_ much easier to do this with python. Maybe you could >> use Perl too if you know it. > >>>>> a = "avahi-tools i386 0.6.11-6.fc6 olpc_rhbase >>>>> 56 k" >>>>> b = a.strip().split() >>>>> b >> ['avahi-tools', 'i386', '0.6.11-6.fc6', 'olpc_rhbase', '56', 'k'] >>>>> rpm_name = "%s-%s.%s.rpm" % (b[0], b[2], b[1]) >>>>> rpm_name >> 'avahi-tools-0.6.11-6.fc6.i386.rpm' > >> For the distro-rebuild stuff with plague, see: > >> http://people.redhat.com/dcbw/distro-rebuild.py > >> That's a somewhat hacky script to unpack the SRPM, change the Release # >> by adding .1 to the end, and rebuild the SRPM, then enqueue the SRPM in >> a plague buildsystem. > >>>> - Rebuild those SRPMs (I hear there's this thing called 'mock' that >>>> helps with this en masse) >>>> >>>>> Mock builds one SRPM in a chroot. It does not do builds en-masse, it >>>>> builds one SRPM and does it well. It was decided not to pollute mock >>>>> with a ton of policy and options, because different people want to build >>>>> a set of RPMs differently. >>>>> Tools like plague will automate the queuing of multiple SRPMs across >>>>> different architectures using remote and/or local build hosts. But mock >>>>> knows nothing about networks, nor should it. > Cool. I'll look plague up. > >>>>> What you want to do is: >>>>> 1) Create a build strategy; do you build glibc first? do you rebuild >>>>> gcc first? Bootstrapping a distro sucks, but you probably don't want to >>>>> do the whole thing. > I'll rebuild each package, it doesn't really matter what order because > I'm not making ABI changes so the libraries will link together fine. > Bootstrapping isn't necessary to drop in a simple compiler option > (unless it's -fstack-protector or such). > >> Ok, your life just got a lot easier. > >>>>> 2) Once you've got the list of SRPMs you'd like to build and the order >>>>> in which you'd like to build them, change the RPM _mock_ macro for >>>>> RPM_OPT_FLAGS to include the CFLAGS you want, then create a script that >>>>> feeds the SRPMs in that order through mock. I can dig this up; it means >>>>> you change the mock config file (in /etc/mock) for your target and add a >>>>> line for your optflags. > I'll peek at google and see if there's a manpage.... nope. I'll look in > /etc/mock later, yum is busy on that box. > >>>From the Fedora Extras builders: > >> [EMAIL PROTECTED] ~]$ cat /etc/mock/fedora-devel-i386-core.cfg > >> <snip> > >> config_opts['macros'] = """ >> %%vendor Fedora Project >> %%distribution Fedora Extras >> %%packager Fedora Project <http://bugzilla.redhat.com/bugzilla> >> %%_topdir %s/build >> %%_rpmfilename %%%%{NAME}-%%%%{VERSION}-%%%%{RELEASE}.%%% >> %{ARCH}.rpm >> """ % config_opts['chroothome'] > >> </snip> > >> If you look at, for example, /usr/lib/rpm/redhat/macros, you can get an >> idea of the optflags you want to override: > >> optflags: i386 %{__global_cflags} -m32 -march=i386 -mtune=generic >> -fasynchronous-unwind-tables > >> with your own stuff. So, in the mock config file for the target for >> which you are building your stuff, you'll want to add the >> config_opts['macros'] section and do something like: > >> my_optflags = "-Os -whatever -goreallyfast" >> config_opts['macros'] = """ >> %%optflags %s >> """ % my_optflags ...... kay, I think I got that. We'll see when I get there :) > >>>>> 3) When each SRPM is done building, you have two choices. Do you want >>>>> to use the just-built RPM for building other RPMs? Or do you not care? >>>>> Remember that since this is building inside a chroot, you need to get, >>>>> for example, /lib/libc-2.4.so from somewhere. Currently that is pulled >>>>> from the Fedora Core Development repository, and it _won't_ be rebuilt >>>>> with any of your changes. If you're just changing CFLAGS, I don't think >>>>> this will be a problem. If you want to do linker changes or something >>>>> like that, then it would be a problem. > Yeah, it's just CFLAGS. The linker changes I wanted are standard in > Fedora Core 6.... and I have FC5. Eh, it won't matter as long as I > build 2 different images but maybe I should upgrade my laptop to Rawhide > anyway (I have no idea how, I'm a debian guy). > >> It shouldn't matter that you have FC5, because all the building is done >> in the chroot, and you're going to pull from the devel repository >> anyway. So the tools & linker that's actually used to rebuild the >> packages in the chroot will already be from FC6, since mock will go >> fetch them. > Sweet. That makes life much much easier. >>>> - Make a stream to build using a local repository of rebuilt RPMs >>>> >>>>> You create a local repository of RPMs using createrepo. You then point >>>>> the pilgrim config file for your stream at this repository, and you >>>>> remove any reference to other repositories, because they can pollute >>>>> your package pool (unless you do something like bumping the Release # of >>>>> the package). I may have some scripts that can bump the release number >>>>> slightly to ensure that the built packages will be "newer" to RPM. > I can just build everything and stick it in my repo. > >>>> - Get a stream built of mine and of the original RPMs >>>> >>>> Right now I'm getting things like: >>>> >>>> install-info: no such file or directory for /usr/share/info/sed.info.gz >>>> error: %post(sed-4.1.5-5.fc6.i386) scriptlet failed, exit status 1 >>>> >>>>> I hope we don't install info at all. But you will run into this because >>>>> the %post, %postun, %pre, and/or %preun sections of the RPMs may be >>>>> trying to use utilities that aren't on the system or in your chroot yet. >>>>> Most of these are harmless and can be ignored. If they stop your >>>>> install, that's a problem, but means you aren't using the right tool to >>>>> install them. >>>> For a lot of things (sed, readline, cpio, cpp, gzip, findutils, tar...), >>>> not sure why. Last time I did this I got a bunch of empty directories >>>> that should have contained images. >>>> >>>>> Some of these, we plain may not install. It's quite hard to imagine why >>>>> we would include 'cpp' by default. I know it's used for more than >>>>> software development, but chances are we don't care about what uses it >>>>> in the default images. >>>> I figure once I get the building to actually work, I'll move on to >>>> figuring out how to get a list of RPMs used, and then turn those into >>>> URLs for SRPMs, and work from there. Not sure where to start with this >>>> though, the build.log has some stuff but it's cryptic and may be a >>>> little challenging to parse. >>>> >>>>> Rebuilding a bunch of stuff is hard work, and it's manual work, because >>>>> unless you're on a source-based distro like Gentoo or LFS, you never do >>>>> this, your distro release team hits the rebuild switch when it's >>>>> required. And they have all the infrastructure set up already. > bash scripts. The reason Linux pwns Windows. :) > >>>> Is there any documentation at all for this? I know I'm not there yet; >>>> but I'm primarily concerned with figuring out exactly what RPMs are used >>>> (so I can build proper RPMs from SRPMs) and switching over to a local >>>> repo (so I can build an image without an -O2 enabled optimization that I >>>> believe may be problematic on the Geode and do some tests). >>>> >>>>> You can use a tool like plague >>>>> (http://www.fedoraproject.org/wiki/Projects/Plague ) to automate the >>>>> rebuild process, even across multiple machines. It depends on your >>>>> needs, if you feel comfortable scripting the mock rebuild process >>>>> yourself, or using a tool like plague (which runs the Fedora Extras >>>>> buildsystem http://buildsys.fedoraproject.org/build-status/ ) which >>>>> requires some initial setup. > mmm... scripting it myself may be less to learn. > > Aside, I've got an ext3 .img and a jffs2 tree now, but no jffs2 .img. Hmm. > >>>>> Dan - -- We will enslave their women, eat their children and rape their cattle! -- Bosc, Evil alien overlord from the fifth dimension -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIVAwUBRSv9pAs1xW0HCTEFAQL22w//ZEJvmvThDspTy9kakPbEV4OMOZMa/QEn x2gw6UK9OsXAWkGvwpOT6m9CzOMVy8DUawFZm2FsgRl5OAW6+fcoJihDsKdhRgZ1 qoRMqBEAb1B2YAxUAponekZxzk3XyvdqIog/thGp4fpZKY8xJu/VPfr/w9Kb/ZLJ g7qEIHnhT7UQ0fo9ul9vYcr4B9thkwW+U8FGl9Jv081ZCV3emedcc+bQfrcWH3cv i6qzL+KOLJTno1LRHrrZFhaqbiWCYqzDpj9/wxSRWtnjKoZNtkT5ZfX+UDP5fqEW ffwXW1U/fP0BJOdzM9VarqjWW0syXRq2g81NJQveobc8I15MIQkiqXqBrAHFeEE2 997UweOgKaafrTVrXupPWtuEmikpoKdVRVIR2UkOhSaB8HjGd4Yzrlp6KCO/TmYG 8vmQU+2Wi8/0sN55Zn31XJpA1lm/LAb9SNE8pZbor/e/0BnsXejso4gCBkbM7EZV y9aDYSeizYVI87n6UL5O/SjQ/xJUkBhlYpVxQ9N6Sd9QR6BbsVapYnhVQ+PGKMqj rBp+gr54NFIhOT2y0LLFCG+a6ImLwmsjONB+0vU0Gnz55S/TD+LV4rPGvUMhd8Df LMVJKUhC9fIYXb6IXqsX6blZpVUYekiFnjo64PktHVTcagFFZBcmSE9X46dVDsh8 SlvSy4yVcgk= =cLUz -----END PGP SIGNATURE----- _______________________________________________ Devel mailing list [email protected] http://mailman.laptop.org/mailman/listinfo/devel
