typo (was Re: debootstrap and debian-ports)
Sorry, debian-ports archive key is http://ftp.debian-ports.org/archive/archive_2013.key and debian snapshots are http://snapshot.debian.org/archive/debian-ports/ (note the final /) J.-H. Chatenet -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131219010207.GA1958@debirf-dominicain
Re: debootstrap and debian-ports
Thank you for your answer Le mercredi 18 décembre à 17h 26mn 40s (+), Thorsten Glaser a écrit : > Michael Schmitz dixit: > > > your finding that packages from both unstable and unreleased are needed is > > correct (along with the complication that some may not be availabe at any > > given > > time). > > There’s another problem: even in the main Debian archive, “unstable” > is *not* guaranteed to be debootstrap’able, and has regularily been > broken. > > Good news for m68k though: eglibc, gcc-4.8 and linux are no longer > in “unreleased”. In fact: > > tg@freewrt:~ $ > u=/var/lib/apt/lists/ftp.de.debian.org_debian-ports_dists_unreleased_main_binary-m68k_Packages > tg@freewrt:~ $ # test idempotency > tg@freewrt:~ $ grep-dctrl -r -P . <$u | diff -u - $u | wc > 0 0 0 > tg@freewrt:~ $ # get me all source packages that have packages in > unreleased/m68k > tg@freewrt:~ $ grep-dctrl -r -P . -n -s Source:Package <$u | sort -u > atari-bootstrap > atari-fdisk > gcc-4.6 > gcj-4.6 > glib-networking > gnat-4.6 > google-gadgets > libbluray > m68k-vme-tftplilo > m68kboot > mesa > mysql-5.1 > vmelilo > webkit > I am reading after my last post, and I realize that I tryied hard to implement something already available in grep-dctrl... > We can group them by: > > • architecture-specific packages > atari-bootstrap > atari-fdisk > m68k-vme-tftplilo > m68kboot > vmelilo > > • architecture-specific patches, packages going away in sid soon anyway > gcc-4.6 > gcj-4.6 > gnat-4.6 > mysql-5.1 (actually already gone) > > • maintainer refuses integrating our patches > libbluray (maybe ping again?) > mesa(refusal also upstream) > > • patches need to be updated against current versions of the packages > google-gadgets (waits on webkit/gtk) > webkit > > • “Build without libproxy, for bootstrapping.” > glib-networking > > Nice work indeed (same thing for debian-alpha). It is now possible (checked mid-november) to install from m68k on debian-ports a lxc container with qemu user mode emulation thanks to Laurent Vivier's qemu-m68k (https://gitorious.org/qemu-m68k/qemu-m68k) > None of them is, however, strictly needed for debootstrap > (although the architecture-specific packages may be needed > when d-i’ing a system). I read somewhere that Aurélien > regularily creates snapshots of debian-ports – which means > that we can install m68k from these, Right Now™. > > deb http://ftp.debian-ports.org/debian-snapshot/2013-12-12/ unstable main > > This should work. Maybe Aurélien can “freeze” one of these, > if needed? > I am not sure : are these snapshots apt-gettable ? Due to the way they are taken (cp -al, lines 37, 38, 39 of archive-snapshot in mini-dak, and archive-reindex-meta line 87 ), every package or metadata is kept, but the Release files which seem to be hard links towards the current ones (Is it correct ?) > > -- > > Back to debootstrap. Yes, it needs support for multiple versions > (already has some, atm) and the unreleased distribution right now. > > I guess APT’s ordering (from a given package, always use the > dpkg-numerically largest version, ignoring all dpkg-numerically > smaller versions, period) would work for now, as we don’t have > the arch:all/arch:any mix in the minbase, base or buildd set > much (except libsemanage-common). Everything else needs a very > complicated solver (such as, use an older libsemanage-common > that works with the libsemanage1 version in the archive) and > is out of scope for the sh-based debootstrap. > > > bye, > //mirabilos (short, caught the flu) > -- > │ untested > │ tut natürlich > │ was auch sonst ... > │ fijn ☺ Gute Besserung ! J.-H. Chatenet -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131219005316.GA1928@debirf-dominicain
Re: debootstrap and debian-ports
jhcha54008 dixit: >Custom mini-repositories for installation >- > >One may download the missing packages from >http://snapshot.debian.org/archive/debian-ports. Indeed, but – as I said – the regular debian-ports archive is also weekly snapshotted, and Aurélien can persist them TTBOMK (like etch-m68k was). I’ve got a manually created mini-repository for m68k, but experience shows acceptance of these is questionable even if done by a DD, *and* you need custom archive keys, so I think it’s best to stick to “more official” ways if these contain all needed packages in unstable (or debootstrap’s patched to handle unreleased). bye, //mirabilos -- "Using Lynx is like wearing a really good pair of shades: cuts out the glare and harmful UV (ultra-vanity), and you feel so-o-o COOL." -- Henry Nelson, March 1999 -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1312182157360.19...@herc.mirbsd.org
Re: debootstrap and debian-ports
Michael Schmitz dixit: > your finding that packages from both unstable and unreleased are needed is > correct (along with the complication that some may not be availabe at any > given > time). There’s another problem: even in the main Debian archive, “unstable” is *not* guaranteed to be debootstrap’able, and has regularily been broken. Good news for m68k though: eglibc, gcc-4.8 and linux are no longer in “unreleased”. In fact: tg@freewrt:~ $ u=/var/lib/apt/lists/ftp.de.debian.org_debian-ports_dists_unreleased_main_binary-m68k_Packages tg@freewrt:~ $ # test idempotency tg@freewrt:~ $ grep-dctrl -r -P . <$u | diff -u - $u | wc 0 0 0 tg@freewrt:~ $ # get me all source packages that have packages in unreleased/m68k tg@freewrt:~ $ grep-dctrl -r -P . -n -s Source:Package <$u | sort -u atari-bootstrap atari-fdisk gcc-4.6 gcj-4.6 glib-networking gnat-4.6 google-gadgets libbluray m68k-vme-tftplilo m68kboot mesa mysql-5.1 vmelilo webkit We can group them by: • architecture-specific packages atari-bootstrap atari-fdisk m68k-vme-tftplilo m68kboot vmelilo • architecture-specific patches, packages going away in sid soon anyway gcc-4.6 gcj-4.6 gnat-4.6 mysql-5.1 (actually already gone) • maintainer refuses integrating our patches libbluray (maybe ping again?) mesa(refusal also upstream) • patches need to be updated against current versions of the packages google-gadgets (waits on webkit/gtk) webkit • “Build without libproxy, for bootstrapping.” glib-networking None of them is, however, strictly needed for debootstrap (although the architecture-specific packages may be needed when d-i’ing a system). I read somewhere that Aurélien regularily creates snapshots of debian-ports – which means that we can install m68k from these, Right Now™. deb http://ftp.debian-ports.org/debian-snapshot/2013-12-12/ unstable main This should work. Maybe Aurélien can “freeze” one of these, if needed? -- Back to debootstrap. Yes, it needs support for multiple versions (already has some, atm) and the unreleased distribution right now. I guess APT’s ordering (from a given package, always use the dpkg-numerically largest version, ignoring all dpkg-numerically smaller versions, period) would work for now, as we don’t have the arch:all/arch:any mix in the minbase, base or buildd set much (except libsemanage-common). Everything else needs a very complicated solver (such as, use an older libsemanage-common that works with the libsemanage1 version in the archive) and is out of scope for the sh-based debootstrap. bye, //mirabilos (short, caught the flu) -- │ untested │ tut natürlich │ was auch sonst ... │ fijn ☺ -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1312181709050.19...@herc.mirbsd.org
Re: debootstrap and debian-ports
Hi, your finding that packages from both unstable and unreleased are needed is correct (along with the complication that some may not be availabe at any given time). For m68k, Thosten Glaser is perhaps the most experienced in dealing with bootstrapping an install. Cheers, Michael Looking on the web for a recommended way of using debootstrap with debian-ports, I couldn't find a definitive answer. Here are my findings - may it help ? Perhaps someone wiser could comment and shed some light on the matter ? debootstrap and debian ports Recently (or not) the question arose on the mailing list : http://lists.debian.org/debian-alpha/2012/06/msg7.html http://lists.debian.org/debian-m68k/2013/08/msg00108.html The install from debian-ports comes with two hurdles : 1/ Packages from both "suite" unstable and unreleased are needed (by contrast, a normal install pulls packages from one sole suite chosen among oldstable, stable or unstable) 2/ There are often missing, yet to build dependencies absent of the repositories. Let's concentrate on 1/ debian-ports hosts packages for unofficial architectures until they are integrated in the main Debian archive. There are 3 suites : unstable, unreleased and experimental, and in each one component : main (one have to download non-free firmwares from the main Debian archive). From the description on http://www.debian-ports.org : "The unreleased suite : this suite is used to upload packages which need hacks to build until the changes are integrated into Debian." In an ideal world, the changes would be included in the next upstream version of the package. And this new version entering into unstable would replace the hacked one in unreleased. In the real world, it is often necessary to keep different versions in the archive (think of the toolchain). This results in several versions of one package spread among unstable and unreleased. libstdc++6 is the most prominent example : one for each gcc-${version} source package. From these versions, debootstrap should install the newest one. debootstrap and multiple versions - debootstrap is designed to download and parse the Packages file from one sole suite (the Packages file contains stanzas describing each a version of a package in the suite). The names of all packages of priority required, of priority important (unless told otherwise) and their dependencies (unless told otherwise) are collected. (function work_out_debs, line 18 of /usr/share/debootstrap/scripts/unstable, in debootstrap version 1.0.55) For each name, the version selected for installation is the one in the last stanza of the first consecutive sequence of stanzas of this package name. (function download_debs, line 623 of /usr/share/debootstrap/functions, function pkgdetails_perl, lines 1095, 1180 and 1186 of /usr/share/debootstrap/functions, in debootstrap version 1.0.55) This amounts to choose the newest version iff the Packages file is sorted, i.e. all stanzas of a given package name are grouped together, and this group is sorted by ascending version number. Some order in Packages files The program mini-dak at work at debian-ports currently assembles a Packages file with "Architecture: all" packages first, and then the ones of the given architecture. apt-sortpkgs (of package "apt-utils") sorts separately each of these groups. However, it is a mere lexical sort on package name and then on version string to break tie. Epochs and version comparison rules are not taken into account. (line 50 of cmdline/apt-sortpkgs.cc line 604 of apt-pkg/contrib/strutl.cc in apt version 0.9.14.1) (line 39 of archive-reindex-meta in mini-dak) Some packages (e.g. sensible-utils 0.0.0 and 0.0.9 on m68k, iproute 20120521-3 and 1:3.12.0-1 everywhere) have moved from one architecture to "Architecture: all" in newer versions (or conversely) - both versions are present in the archive... ("debootstrap --no-resolve-deps" actually ends up in a fatal error : "sensible-utils" is counted twice when collecting the names, once when selecting the version to install. Is it a bug in debootstrap ? This isn't a common use case though) A solution (?) -- I would suggest to modify debootstrap to download Packages files from both unstable and unreleased, to concatenate them and to sort by package name and by ascending version number. Some previous (unsuccesful) thoughts - See also http://bugs.debian.org/690210 - Give precedence to packages in unreleased (or in unstable either) : this is wrong if a newer version exists in the other suite. It depends on the history of package changes (e.g. libstdc++6 on alpha and m68k). - Replace selected versions from unstable by newer versions from unreleased if available : it doesn't address the architecture <-> all problem. - Concatenate Packages files from unstable