Re: make -j in Debian packages
On Wed, 28 Jun 2006, Adam Borowski wrote: > On Wed, Jun 28, 2006 at 02:06:26AM -0700, Don Armstrong wrote: > > On Wed, 28 Jun 2006, Adam Borowski wrote: > > > Let's allow maintainers to use make -jX according to their common > > > sense, requiring obeying an env variable to opt out. > > > > Why not just have some ENV variable (CONCURRENCY_LEVEL?) which > > specifies the maximum -j; the package maintainer is free to choose any > > level equal to or below that. > > > > If CONCURRENCY_LEVEL is not set, the rules file must assume > > CONCURRENCY_LEVEL=1 (or not specify -j?); this will keep what should > > be the current status quo the same, and allow buildd's and other > > builders to set this variable to a reasonable level for their system. > > I would rather have it default to "a reasonable value on an average > uniprocessor box, according to the maintainer's common sense". > Otherwise, nearly no one will have this variable set and thus the > whole benefit will be lost. And cutting the build time by half or > more is a GREAT thing to have. The only places setting the variable matters is on buildds and when people are building the package; I think the buildd people will be aware of it if it helps, and maintainers will have to know of it if they implement it. > This is certainly not what you want for a package build; unlimited > -j is useful for other uses of make. I was more thinking of cases where the upstream Makefile already specifies a -j if one isn't set. > > This has the disadvantage of not automatically using -j for every > > package and requiring maintainer buy in to see results... but > > presumably those packages where it actually makes a difference > > will adopt it just so maintainer builds don't take as long. > > Why would you restrict this just to maintainer builds? I don't mean to restrict it. > Speeding up buildds and user builds would be a worthy goal, too. An > obscure env var hardly anyone knows about means that hardly anyone > will be affected. Because maintainers will be the one making changes to rules files to implement CONCURRENCY_LEVEL (or whatever is decided on), they'll be more likely to do so if there's a benefit for them. Don ARmstrong -- There is no such thing as "social gambling." Either you are there to cut the other bloke's heart out and eat it--or you're a sucker. If you don't like this choice--don't gamble. -- Robert Heinlein _Time Enough For Love_ p250 http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: new tar behavior and --wildcards
On Wed, 2006-06-28 at 10:36 +0200, Bill Allombert wrote: > > Here, the only way seems to be putting an entry in NEWS.Debian (for > > users script, ie things not under our control). Good idea, Christian. > In addition, I would suggest we reinstate the previous behaviour, but > display a warning when wildcards are used but --wildcards is not set. The problem with this is that generating a new warning can also cause people to need to update scripts, since lots of people seem to parse the output of commands like tar in wrapper scripts. So, I'm not convinced that this is really a good idea. I'm also always hesitant to deviate Debian default behavior for utilities like tar from upstream. All in all, I'm not yet convinced that reverting to the old wildcard behavior is the right thing to do. I've only heard about problems in a few (four?) packages so far, and all of them are Debian-specific programs that should be easy for us to update. I see no need for panic, though it's obviously and clearly regrettable that the packages in questions are ones that affect processes like building and testing Debian packages. Bdale -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Another weird tar issue (100 character filenames)
On Tue, 2006-06-27 at 13:09 +0200, Jeroen van Wolffelaar wrote: > But on the other hand, according to the 'be strict in what you send, > liberal in what you accept' mantra, it makes sense for tar to not create > tarfiles which in the past have caused issues for certain programs while > there's a perfectly fine way to create tarfiles which cannot trigger > such bugs. Might the patch in #230910 need to be re-crafted and re-applied? The code has changed enough that it wasn't obvious to me when moving to the new upstream that we still needed this change, but perhaps we do. If you have time to try the attached patch and let me know if it solves your problem, I'd appreciate it... I *think* this is a relatively equivalent set of changes for the current version of tar. Bdale Index: create.c === RCS file: /cvs/bdale/debian/tar/src/create.c,v retrieving revision 1.22 diff -u -u -r1.22 create.c --- create.c 22 Jun 2006 21:06:13 - 1.22 +++ create.c 28 Jun 2006 23:22:15 - @@ -633,7 +633,7 @@ return write_short_name (st); } else if (NAME_FIELD_SIZE - (archive_format == OLDGNU_FORMAT) - < strlen (st->file_name)) + <= strlen (st->file_name)) return write_long_name (st); else return write_short_name (st); @@ -1310,7 +1310,7 @@ block_ordinal = current_block_ordinal (); assign_string (&st->link_name, link_name); if (NAME_FIELD_SIZE - (archive_format == OLDGNU_FORMAT) - < strlen (link_name)) + <= strlen (link_name)) write_long_link (st); st->stat.st_size = 0; @@ -1595,7 +1595,7 @@ } buffer[size] = '\0'; assign_string (&st->link_name, buffer); - if (NAME_FIELD_SIZE - (archive_format == OLDGNU_FORMAT) < size) + if (NAME_FIELD_SIZE - (archive_format == OLDGNU_FORMAT) <= size) write_long_link (st); block_ordinal = current_block_ordinal ();
Re: sending debian-private postings to gmail
On Tue, Jun 27, 2006 at 09:22:45PM +0100, Aigars Mahinovs wrote: > Ian Jackson <[EMAIL PROTECTED]> said: > > I'm one of the small minority of people who have a very negative > > opinion about gmail. I realise I'm a bit of a kook on this subject > > and I'd ideally I'd like to avoid having an enormous flamewar about > > it. > > However, it has come to my attention that at least one developer > > appears to be reading debian-private at their gmail account. > I am one of those developers. I have never though that such action could > be considered a violation of debian-private policy and some reasons for > that have already been raised. In fact I do think that we should encrypt > the postings to debian-private for both privacy and flamecontrol > reasons. At this encryption stage headers of the messages should be > stripped and only stored on the server. Only most important headers > (like from, subject and date) would be embedded in the encrypted > payload. > Unless we go that far and realise such system, I see no reason to single > out Google on the storage of the mail messages from debian-private. I would expect developers to exercise the same judgement with regard to any mail provider that they have reason to believe is analyzing mail for, or delivering mail to, parties other than the intended recipient. Google is singled out only in the sense that it's well-known, widely used, and has a published policy of analyzing received mail for its advertisers. If anything, it's commendable that Google has been open about the existence of this practice, but I still share Ian's concern that it makes GMail an unsuitable mail store for -private mail. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ signature.asc Description: Digital signature
Bug#375903: ITP: libgetopt++ -- C++ wrapper for getopt to allow arbitrary location of options within sources
Package: wnpp Severity: wishlist Owner: "Adeodato Simó" <[EMAIL PROTECTED]> * Package name: libgetopt++ Version : 0.0.2-p22 Upstream Author : Robert Collins <[EMAIL PROTECTED]> * URL : http://people.ubuntu.com/~robertc/rbtcollins%40hotmail.com--barch/libgetoptplusplus/ * License : GPL Programming Lang: C++ Description : C++ library for command line parsing libgetopt++ is a C++ library to parse options that a program gets specified from the command line. It has the following features: * Minimal footprint in main.cc, and no header or source changes outside the user of an option when the option is altered or a new option added. * Multiple option sets can co-exist safely. The default option set is a singleton, but additional static sets can be created and used. * Easy to use: adding a new option is simply a case of adding a static variable for the option, in the scope that the option needs to be visible. * There are multiple concrete Option classes provided: Bool, String, StringCollector. * Extensible: simply create a new subclass of Option to implement a new Option type, and use it in your program. At the moment, only config-manager needs libgetopt++, that I know of. -- Adeodato Simó dato at net.com.org.es Debian Developer adeodato at debian.org Listening to: David Bowie - Five years
Re: new tar behavior and --wildcards
On Tue, Jun 27, 2006 at 03:40:49PM +0200, Thijs Kinkhorst wrote: > In that case just reverting the Debian change isn't the right way. If > you think that the change is wrong, then you should make upstream fix > it: changing behaviour of tar is one thing, but having different > behaviour of a basic tool like tar on different distros is very > undesirable. Sarge and Etch are such different distros. Cheers, -- Bill. <[EMAIL PROTECTED]> Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 07:50:50PM +0300, Lars Wirzenius wrote: > ke, 2006-06-28 kello 18:43 +0200, Adam Borowski kirjoitti: > > What do you think about going with Don Armstrong's suggestion > > ($CONCURRENCY_LEVEL), while handling the default (no env var) my > > way (decent memory => parallel, little memory => -j 1) instead of > > Ingo's (-j 1 unless explicitely set)? > > I'm in favor of Ingo's approach. I don't think it is a good idea to > hardcode assumptions of what is sufficient memory size into rules files; > that's the kind of thing that is best done centrally. Still, the buildd admin has no way to estimate how much a sub-process of a package is going to use, the maintainer has at least a rough idea. Since the maintainer's action is needed anyway, he can as well provide this estimate. And then the buildd admin can set CONCURRENCY_LEVEL to 1 to centrally disable any parallelism of this kind. > (Also, any rules file snippet should be as simple as possible. Otherwise > fixing it everywhere when a bug is found is going to be tedious.) Exactly, or even better, it could be first debugged until it makes everyone happy, then put into a common repository (debhelper?). This way, fixing a bug would require bothering just a single person. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
problem with root on LVM on RAID (was: RFT: please test mdadm/experimental)
Hi, Thank $DEITY for testers. We discovered another issue when root is on LVM is on RAID. If that's your setup, please hold off on using mdadm 2.5 until I managed to fix #375879. Sorry for the inconvenience, and again, thanks to Alec Berryman for his help. He also spotted the last issue. -- Please do not send copies of list mail to me; I read the list! .''`. martin f. krafft <[EMAIL PROTECTED]> : :' :proud Debian developer and author: http://debiansystem.info `. `'` `- Debian - when you have better things to do than fixing a system "the public is wonderfully tolerant. it forgives everything except genius." -- oscar wilde signature.asc Description: Digital signature (GPG/PGP)
Re: make -j in Debian packages
ke, 2006-06-28 kello 18:43 +0200, Adam Borowski kirjoitti: > What do you think about going with Don Armstrong's suggestion > ($CONCURRENCY_LEVEL), while handling the default (no env var) my > way (decent memory => parallel, little memory => -j 1) instead of > Ingo's (-j 1 unless explicitely set)? I'm in favor of Ingo's approach. I don't think it is a good idea to hardcode assumptions of what is sufficient memory size into rules files; that's the kind of thing that is best done centrally. (Also, any rules file snippet should be as simple as possible. Otherwise fixing it everywhere when a bug is found is going to be tedious.) -- Connection resented by peer -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 03:42:07PM +0200, Wouter Verhelst wrote: > SMP buildd systems currently run multiple instances of buildd at > the same time, rather than expecting a package to specify make -j > itself. Having three packages that specify 'make -j 4' on a > multiprocessor buildd host that _already_ runs six builds in > parallel is going to be devastating. I admit, this is news to me. > While parallel builds can indeed help speed up things even on > uniprocessor systems, they do this at a price; they require a _lot_ > more RAM than non-parallel builds. Modular C code can require an order of magnitude less memory than templated C++. What I propose is parallelizing only those packages which are not heavy on RAM, on the maintainer's discretion. If all packages had a similar memory profile, this would be an optimization as it would equalize buildd loads somewhat. > A uniprocessor parallel build will go faster for as long as your > buffercaches can keep make, the compiler, and the most often used > header files in memory; as soon as those need to be swapped out, > your performance will go down dramatically. Since not all machines > have the same amount of RAM, it is impossible to say in a generic > way when this point is going to be. Right, that is why the debian/rules snippet I suggested checks just the amount of RAM without even looking at how many CPUs are present. Indeed, it would fail to recognize several instances of buildd running at once, but this is such a rare case that it can be handled with an env variable we can agree upon. > In any case, I have nothing against doing parallel builds by > default (at least, not as long as it can be properly disabled and > regulated), Cool! This is exactly what this thread is about. What do you think about going with Don Armstrong's suggestion ($CONCURRENCY_LEVEL), while handling the default (no env var) my way (decent memory => parallel, little memory => -j 1) instead of Ingo's (-j 1 unless explicitely set)? I think that the sample script I uploaded in kbtin would do the job; if it is not good enough, it can be improved. > but do not claim that it's a good thing "also for the buildd's", > because you are sadly mistaken. Still, my way is already an improvement over gem: gem used unconditional -j4 and it happens to be a C++ package. (Well, well. Because this happened just days after I uploaded a version with unconditional -j4 to mentors, it's likely it's me who's the real culprit. Gem's maintainer could have looked at my package and copied -j4, the choice of 4 instead of 2 kind of suggests that.) Another idea is to use -l; it trims concurrency down to 1 whenever system load exceeds a given value. Anyway, it's possible to come to an agreement that will make everyone happy, and I believe we're close. Meep? -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
How to securely read debian-private for sure
Hi,
On Tue, Jun 27, 2006 at 09:22:45PM +0100, Aigars Mahinovs wrote:
> Ian Jackson <[EMAIL PROTECTED]> said:
> > I'm one of the small minority of people who have a very negative
> > opinion about gmail. I realise I'm a bit of a kook on this subject
> > and I'd ideally I'd like to avoid having an enormous flamewar about
> > it.
I claim I was guilty too. Now I use my local ISP's mailbox. No better
though.
> > However, it has come to my attention that at least one developer
> > appears to be reading debian-private at their gmail account.
>
> I am one of those developers. I have never though that such action could
> be considered a violation of debian-private policy and some reasons for
> that have already been raised. In fact I do think that we should encrypt
> the postings to debian-private for both privacy and flamecontrol
> reasons. At this encryption stage headers of the messages should be
> stripped and only stored on the server. Only most important headers
> (like from, subject and date) would be embedded in the encrypted
> payload.
>
> Unless we go that far and realise such system, I see no reason to single
> out Google on the storage of the mail messages from debian-private.
>
> Currently GMail is the most trustable mail storage location that I have
> available. And it is also the only location I do use - all my mail from
> all other locations is redirected there. Even if I do redirect
> debian-private mail to another place, that will simply mean that I will
> stop reading it.
Let's make known technical solution more available for all of us.
I know using any external ISP mail box is no better. I do not have
fixed IP host to get mail stably.
BSMTP should work. I am talking about debian.net VHOST MX service for DD
here. This method keeps messages on the Debian machine until you pick
it to your local PC. So far, I am not successful.
I am writing HOWTO but can anyone help me really doing this.
See:
http://wiki.debian.org/DebianServiceForDD
This is based on famous old mails:
http://lists.debian.org/debian-devel/2001/02/msg00965.html
http://db.debian.org links
and poking around people.debian.org /etc/exim{4,}
I see only few DD uses this yet.
Yes, we should use this more.
Update of wiki page will be appreciated.
Osamu
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: new tar behavior and --wildcards (proposed middle ground)
Both sides in this discussion seem to have valid concerns: FOR making --wildcard the default - compatibility with upstream - compatibility with standards - compatibility with other distributions - whatever reasons POSIX had for this were probably sensible - upstream's judgement on this is likely to be correct AGAINST making --wildcard the default - very difficult to figure out what scrips are affected - mysterious breakage far into the future - requires grubbing around and inserting --wildcard in many places As a compromise that addresses some of the issues I would suggest the following: go with upstream, but add some convenience code, to whit: (1) Hot-wire tar to check an environment variable TAR_WILDCARD_DEFAULT and activate the --wildcard option if set. (2) Hot-wire tar to print a warning message to stderr if it (a) is defaulting to the --no-wildcard behaviour and, (b) it notices a filename that, had tar instead been in --wildcards mode, would have been expanded. If stderr is not hooked up, the warning could reasonably be sent using syslog() instead. I wouldn't bother adding any mechanism to shut these warnings off; if one really wants them shut off, use either --wildcards or --no-wildcards. Point (1) would allow people to easily check if some breakage is caused by this change, or to use old scripts/sources without contortions; and point (2) would serve to catch problems in scripts that might otherwise elude us. Point (2) would be especially helpful given that we're coming up on a release, so it would be nice to find and correct all affected scripts as rapidly as possible. -- Barak A. Pearlmutter Hamilton Institute & Dept Comp Sci, NUI Maynooth, Co. Kildare, Ireland http://www.bcl.hamilton.ie/~barak/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 12:01:31PM +0200, Adam Borowski wrote: > On Wed, Jun 28, 2006 at 02:06:26AM -0700, Don Armstrong wrote: > > This has the disadvantage of not automatically using -j for every > > package and requiring maintainer buy in to see results... but > > presumably those packages where it actually makes a difference will > > adopt it just so maintainer builds don't take as long. > > Why would you restrict this just to maintainer builds? Speeding up > buildds and user builds would be a worthy goal, too. "Speeding up" buildds through uninvited make -j is certainly _not_ a worty goal, simply because you can't do that properly. SMP buildd systems currently run multiple instances of buildd at the same time, rather than expecting a package to specify make -j itself. Having three packages that specify 'make -j 4' on a multiprocessor buildd host that _already_ runs six builds in parallel is going to be devastating. While parallel builds can indeed help speed up things even on uniprocessor systems, they do this at a price; they require a _lot_ more RAM than non-parallel builds. A uniprocessor parallel build will go faster for as long as your buffercaches can keep make, the compiler, and the most often used header files in memory; as soon as those need to be swapped out, your performance will go down dramatically. Since not all machines have the same amount of RAM, it is impossible to say in a generic way when this point is going to be. In any case, I have nothing against doing parallel builds by default (at least, not as long as it can be properly disabled and regulated), but do not claim that it's a good thing "also for the buildd's", because you are sadly mistaken. -- Fun will now commence -- Seven Of Nine, "Ashes to Ashes", stardate 53679.4 signature.asc Description: Digital signature
problem with mdadm 2.5.2-1 / initramfs-tools <<0.65 (was: RFT: please test mdadm/experimental)
Hi again, I am sorry, but there is a stupid, minor glitch in the README.experimental file, which may lead to an unbootable system with initramfs-tools << 0.65. The cause of the problem is the sed call: sed -i -e 's,^PREREQ=\"md\"$,PREREQ=\"mdadm\"$,' \ /usr/share/initramfs-tools/scripts/local-top/lvm See that second '$'? It should not be there. If it is, the local-top/lvm script declares a prerequisite on mdadm$, and due to #369617, this will lead to an error during the initramfs initialisation. Fortunately, you all scrutinised the commands before pasting and running them as root, so the error should not affect anyone. :) The solution if you have not yet rebooted is to change the line PREREQS="mdadm"$ to PREREQS="mdadm" in /usr/share/initramfs-tools/scripts/local-top/lvm, and to rerun update-initramfs -u -k$(uname -r) If you have rebooted and your system won't come back up, reset it, append break=mount to the kernel command line with your boot loader and make the above change as soon as you have a shell in /scripts/local-top/lvm. Then hit ctrl-d and the system should boot regularly. Don't forget to make the change in /usr/share/initramfs-tools/scripts/local-top/lvm as well! Sorry for the inconvenience. -- Please do not send copies of list mail to me; I read the list! .''`. martin f. krafft <[EMAIL PROTECTED]> : :' :proud Debian developer and author: http://debiansystem.info `. `'` `- Debian - when you have better things to do than fixing a system quidquid latine dictum sit, altum viditur. signature.asc Description: Digital signature (GPG/PGP)
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 03:22:35PM +0200, Ingo Juergensmann wrote: > On Wed, Jun 28, 2006 at 01:37:37PM +0200, Adam Borowski wrote: > > The benefits on UP are small (~10%), but except for huge working > > sets, non-negative. And the maintainer knows if the package handles > > huge chunks at once or not. > Right. The maintainer should know it. But when the benefits are so small, > one can argue if it's worth the work to change the build system? ;) > Therefore I think it's best to first go with opt-in before make opt-out the > default, let's say when etch+2 is out. What I mean is opt-in for the maintainer, opt-out for the builder. And the benefits are small only on UP. > > > I would like to see a method to allow usage of other compile accelerators > > > as > > > well (distcc (for using crosscc, but testing locally), ccache, etc). > > In theory, you would set CC, but not everything obeys it. I > > personally tend to mess with what "gcc" points to. Usually with more > > than one levels of indirection, too -- colorgcc is a nice thing for > > when the output goes to a terminal. > Of course, but my point is: when we make a decision now about -j flag, we > can have some thougts about such accelerators as well and implement both in > one go instead of having two separate changes to the build system. "changes to the build system"? Where? Note that in my proposal there are exactly no changes to the global build system, except for a flag that can be set to force a package to go down to -j 1. No legislation needed. And with the guessing code in my debian/rules (package kbtin on m.d.n, as per the sibling branch of the thread), even that flag is not needed. On a small box, the package will go down to -j 1 without any admin intervention. I'm not talking about etch+2, I'm talking about something that can go into the archive right now. Just opt-in by including in debian/rules the code of which v0.1 I uploaded to mentors. Whee? -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 01:37:37PM +0200, Adam Borowski wrote: > > Well, make -jX is not everywhere faster on UPs. It depends on other factors > > as well. If you specify -j2 and the second make is causing lots of swapping, > > you won't gain much if anything at all. > Exactly, just like I said: it depends on the memory. Not only on memory. Take into calculation such values like context switch overhead, task scheduling overhead, etc. On SMP systems there's is even some gain in parallel processing, when you can nail certain process onto certain CPUs to prevent their migration across CPU bounds. But that's another story. > The benefits on UP are small (~10%), but except for huge working > sets, non-negative. And the maintainer knows if the package handles > huge chunks at once or not. Right. The maintainer should know it. But when the benefits are so small, one can argue if it's worth the work to change the build system? ;) Therefore I think it's best to first go with opt-in before make opt-out the default, let's say when etch+2 is out. > > I don't think it's good to define an opt-out variable (*_NON_PARALLEL). > > Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even > > better it would be to use something existing: CONCURRENCY_LEVEL as Don > > Amstrong suggests. > > If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer > > can make use of -jX. > Indeed, using the existing name is better. > But, did you really want to say -z? It's what I want, and you seem > to argue against me :p *cough* missing ! or -n instead of -z of course ;) > However, it's very unlikely someone would set a random env var unless > the person knows about it. Defaults are what a vast majority of > users use, so I would go for sensible ones. The most sensible value, that should work for almost every user would be: make without -j option set. That's the default now, isn't it? ;) > > Keep in mind that there are archs that don't have that much resources to > > build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M > > RAM, but for example armeb which has only 32M. > Good point, but such machines are RARE. I would rather go for either > have the admin opt out or them, or, have the packages detect low-mem > conditions and optimize their builds accordingly. I don't think that those machines are rare. OTOH, almost everything is rare when compared to x86 boxes. But let some porters/policy people decide whether to use opt-out or opt-in. > > I would like to see a method to allow usage of other compile accelerators as > > well (distcc (for using crosscc, but testing locally), ccache, etc). > In theory, you would set CC, but not everything obeys it. I > personally tend to mess with what "gcc" points to. Usually with more > than one levels of indirection, too -- colorgcc is a nice thing for > when the output goes to a terminal. Of course, but my point is: when we make a decision now about -j flag, we can have some thougts about such accelerators as well and implement both in one go instead of having two separate changes to the build system. -- Ciao...//Fon: 0381-2744150 Ingo \X/ SIP: [EMAIL PROTECTED] gpg pubkey: http://www.juergensmann.de/ij/public_key.asc -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 12:38:51PM +0200, Ingo Juergensmann wrote: > I don't think it's good to define an opt-out variable (*_NON_PARALLEL). > Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even > better it would be to use something existing: CONCURRENCY_LEVEL as Don > Amstrong suggests. > If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer > can make use of -jX. > > Keep in mind that there are archs that don't have that much resources to > build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M > RAM, but for example armeb which has only 32M. Ok, so let's try some actual code. I've used my usual playground: package "kbtin" on mentors.debian.net Take a look at the marked ###\\\### box: If CONCURRENCY_LEVEL is set, it will be used if it looks plausible. If it's not there, it will be set to 2 if the box has at least 128MB memory. If any problem happens, the build continues with make -j 1. How do you like this, Ingo? Cheers and schtuff, -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Is OSS only support to be considered a bug?
On Tue, 27 Jun 2006 18:25:20 +0200 Wouter Verhelst <[EMAIL PROTECTED]> wrote: > On Tue, Jun 27, 2006 at 02:08:57PM +0200, Michal Čihař wrote: > > On Mon, 26 Jun 2006 21:35:19 +0200 > > Wouter Verhelst <[EMAIL PROTECTED]> wrote: > > > > > No. There is snd-pcm-oss.ko, which provides working OSS sound, even if > > > you don't use aoss. Just make sure to load the proper module. > > > > This won't provide working sound in case you're using dmix or something > > similar (on sound card without hardware mixing) and other program is > > using sound. > > That isn't a regression from "regular" (i.e., not emulated) OSS sound, > so... Yes it is not, I just wanted to point out that in kernel OSS emulation won't fix everything. -- Michal Čihař | http://cihar.com | http://blog.cihar.com signature.asc Description: PGP signature
Re: [Yaird-devel] RFC: swap on a LVM volume in debian-installer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 22 Jun 2006 22:46:59 +0200 David Härdeman wrote: > in debian-installer, there is a package - partman-auto-lvm - which > can setup an entire disk to be used for the debian installation with > the use of lvm for most partitions. > > Currently it sets up one boot partition, one swap partition and one > lvm PV which is used for the rest of the partitions (usually root and > possibly home depending on the recipe used). > > I'm currently considering whether to change partman-auto-lvm so that > the swap partition is created as a lvm lv rather than a separate > partition, and I'd like to ask for some comments and feedback before > doing so. I believe newest release of yaird properly supports suspend-to-disk, including possibly different driver requirements than for the rootfs. I do not use suspend-to-disk myself, so welcome any feedback, positive and negative, on wether it actually works in all possible scenarios. If I somehow misunderstood the intend of this thread (and yaird being included in it), I apologize: Please then spell it out more clearly for me :-) You can update info directly[1] or report your findings to [EMAIL PROTECTED] Hope that this helps, - Jonas [1] http://wiki.debian.org/InitrdReplacementOptions - -- * Jonas Smedegaard - idealist og Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ - Enden er nær: http://www.shibumi.org/eoti.htm -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFEomatn7DbMsAkQLgRAkAGAJ9lt0+uhEsLKIRXRSn+0vSbtQ56OQCfe5yL L7CY7fyvIj2AQQKLO33Tgyc= =COzl -END PGP SIGNATURE-
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 12:38:51PM +0200, Ingo Juergensmann wrote: > On Wed, Jun 28, 2006 at 10:31:54AM +0200, Adam Borowski wrote: > > On the other hand, making builds significantly faster is not > > something that you can shake a stick at. Typically make -jX is faster > > even on uniprocessor, and I don't need to tell you why it's much > > faster on SMP. > > Well, make -jX is not everywhere faster on UPs. It depends on other factors > as well. If you specify -j2 and the second make is causing lots of swapping, > you won't gain much if anything at all. Exactly, just like I said: it depends on the memory. The benefits on UP are small (~10%), but except for huge working sets, non-negative. And the maintainer knows if the package handles huge chunks at once or not. > > Nearly every buildd and nearly every user building the packages > > on his own will benefit from -j2 [2], even on non-SMP. Unless > > it's a piece of heavily-templated code, any modern box will have > > enough memory to handle it. The maintainers know whether the > > code is heavily templated or not. > > > > Giving the choice to someone who doesn't know the package > > (upstream and/or the maintainer) will lead to _random_ FTBFSes. > > Therefore a double opt-in would be nice (machine admin and package > maintainer saying it's OK). > > > Having DEB_MAKE_NON_PARALLEL disable this behavior would make > > m68k happy, while giving a reasonable default for everyone else. > > I don't think it's good to define an opt-out variable (*_NON_PARALLEL). > Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even > better it would be to use something existing: CONCURRENCY_LEVEL as Don > Amstrong suggests. > If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer > can make use of -jX. Indeed, using the existing name is better. But, did you really want to say -z? It's what I want, and you seem to argue against me :p However, it's very unlikely someone would set a random env var unless the person knows about it. Defaults are what a vast majority of users use, so I would go for sensible ones. > Keep in mind that there are archs that don't have that much resources to > build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M > RAM, but for example armeb which has only 32M. Good point, but such machines are RARE. I would rather go for either have the admin opt out or them, or, have the packages detect low-mem conditions and optimize their builds accordingly. > I would like to see a method to allow usage of other compile accelerators as > well (distcc (for using crosscc, but testing locally), ccache, etc). In theory, you would set CC, but not everything obeys it. I personally tend to mess with what "gcc" points to. Usually with more than one levels of indirection, too -- colorgcc is a nice thing for when the output goes to a terminal. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: proposed mozilla-firefox security update, needs testing!
On Wed, Jun 28, 2006 at 12:44:50PM +0200, Alexander Sack wrote:
> On Wed, Jun 28, 2006 at 12:10:45PM +0200, Alexander Sack wrote:
> > On Tue, Jun 27, 2006 at 10:54:47PM -0400, Eric Dorland wrote:
> > > > >
> > > > > Could you send me a copy of your bookmarks.html file privately?
> > > >
> > > > Eric, was this reproducible?
> > >
> > > Unfortunately it was very reproducible :( I'm not sure how to
> > > proceed. Is anyone else using these patches that might be able to help
> > > figure out where the problem lies?
> > >
> >
> > No ... upstream does not do official rc testing for us. Though, if we can
> > track
> > down which specific patch causes this regression, they will help to figure
> > it
> > out.
> >
> > Did you try to track down which patch(es) cause these troubles?
>
> I tracked it. The evil patch is:
>
>0005-mfsa2006-35-329677.txt
>
> ... reverting this fixes this resolves this regression ... I will investigate.
>
And here comes the fix. In addition I attached a fix for another regression.
Please respin and test with these two patches added.
- Alexander
--
GPG messages preferred.| .''`. ** Debian GNU/Linux **
Alexander Sack | : :' : The universal
[EMAIL PROTECTED]| `. `' Operating System
http://www.asoftsite.org/ | `-http://www.debian.org/
>From nobody Mon Sep 17 00:00:00 2001
From: Alexander Sack <[EMAIL PROTECTED]>
Date: Wed, 28 Jun 2006 12:53:56 +0200
Subject: [PATCH] mfsa2006-35; 335142; regression 1/2 for 329677
---
content/xul/templates/src/nsXULContentUtils.cpp | 11 +--
1 files changed, 9 insertions(+), 2 deletions(-)
463bd4875866b9ac52c198cc38dfb26722cfb56f
diff --git a/content/xul/templates/src/nsXULContentUtils.cpp
b/content/xul/templates/src/nsXULContentUtils.cpp
index 15d217b..bfbf602 100644
--- a/content/xul/templates/src/nsXULContentUtils.cpp
+++ b/content/xul/templates/src/nsXULContentUtils.cpp
@@ -70,7 +70,7 @@ #include "nsContentUtils.h"
#include "nsIDateTimeFormat.h"
#include "nsDateTimeFormatCID.h"
#include "nsIScriptableDateFormat.h"
-
+#include "nsIDOMXULElement.h"
static NS_DEFINE_CID(kDateTimeFormatCID,NS_DATETIMEFORMAT_CID);
static NS_DEFINE_CID(kRDFServiceCID,NS_RDFSERVICE_CID);
@@ -226,6 +226,10 @@ nsXULContentUtils::GetElementRefResource
NS_ASSERTION(NS_SUCCEEDED(rv), "severe error retrieving attribute");
if (NS_FAILED(rv)) return rv;
+if (rv != NS_CONTENT_ATTR_HAS_VALUE) {
+rv = aElement->GetAttr(kNameSpaceID_None, nsXULAtoms::id, uri);
+}
+
if (rv == NS_CONTENT_ATTR_HAS_VALUE) {
// We'll use rdf_MakeAbsolute() to translate this to a URL.
nsCOMPtr doc = aElement->GetDocument();
@@ -242,7 +246,10 @@ nsXULContentUtils::GetElementRefResource
rv = gRDF->GetUnicodeResource(uri, aResult);
}
else {
-rv = GetElementResource(aElement, aResult);
+nsCOMPtr xulElem(do_QueryInterface(aElement, &rv));
+if (xulElem) {
+rv = xulElem->GetResource(aResult);
+}
}
return rv;
--
1.3.3
>From nobody Mon Sep 17 00:00:00 2001
From: Alexander Sack <[EMAIL PROTECTED]>
Date: Wed, 28 Jun 2006 13:04:30 +0200
Subject: [PATCH] mfsa2006-35, 337841 - regression part 2/2 for 329677
---
content/xul/templates/src/nsXULSortService.cpp |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
96a0d1579e945ea50e0c31fc2e0b27638d751a7b
diff --git a/content/xul/templates/src/nsXULSortService.cpp
b/content/xul/templates/src/nsXULSortService.cpp
index 67de503..f9223c2 100644
--- a/content/xul/templates/src/nsXULSortService.cpp
+++ b/content/xul/templates/src/nsXULSortService.cpp
@@ -1403,7 +1403,7 @@ XULSortServiceImpl::InsertContainerNode(
if (!id.IsEmpty()) {
nsCOMPtr containerRes;
-gRDFService->GetUnicodeResource(id, getter_AddRefs(containerRes));
+rv = gRDFService->GetUnicodeResource(id, getter_AddRefs(containerRes));
if (NS_SUCCEEDED(rv))
rv = gRDFC->IsSeq(sortInfo.db, containerRes, &isContainerRDFSeq);
}
--
1.3.3
Re: proposed mozilla-firefox security update, needs testing!
On Wed, Jun 28, 2006 at 12:10:45PM +0200, Alexander Sack wrote: > On Tue, Jun 27, 2006 at 10:54:47PM -0400, Eric Dorland wrote: > > > > > > > > Could you send me a copy of your bookmarks.html file privately? > > > > > > Eric, was this reproducible? > > > > Unfortunately it was very reproducible :( I'm not sure how to > > proceed. Is anyone else using these patches that might be able to help > > figure out where the problem lies? > > > > No ... upstream does not do official rc testing for us. Though, if we can > track > down which specific patch causes this regression, they will help to figure it > out. > > Did you try to track down which patch(es) cause these troubles? I tracked it. The evil patch is: 0005-mfsa2006-35-329677.txt ... reverting this fixes this resolves this regression ... I will investigate. - Alexander -- GPG messages preferred.| .''`. ** Debian GNU/Linux ** Alexander Sack | : :' : The universal [EMAIL PROTECTED]| `. `' Operating System http://www.asoftsite.org/ | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 02:06:26AM -0700, Don Armstrong wrote: > Why not just have some ENV variable (CONCURRENCY_LEVEL?) which > specifies the maximum -j; the package maintainer is free to choose any > level equal to or below that. > [...] > This has the disadvantage of not automatically using -j for every > package and requiring maintainer buy in to see results... but > presumably those packages where it actually makes a difference will > adopt it just so maintainer builds don't take as long. Yes, makes sense to me. When CONCURRENCY_LEVEL already works in reality, we could adopt it and make it a standard env var. -- Ciao...//Fon: 0381-2744150 Ingo \X/ SIP: [EMAIL PROTECTED] gpg pubkey: http://www.juergensmann.de/ij/public_key.asc -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 10:31:54AM +0200, Adam Borowski wrote: > On the other hand, making builds significantly faster is not > something that you can shake a stick at. Typically make -jX is faster > even on uniprocessor, and I don't need to tell you why it's much > faster on SMP. Well, make -jX is not everywhere faster on UPs. It depends on other factors as well. If you specify -j2 and the second make is causing lots of swapping, you won't gain much if anything at all. > Too bad, a C++ build where every file takes 1GB memory obviously > should not be parallelized. Also, no one but the maintainer knows > whether a package is SMP-clean or not. You cannot guess this in an > automated way. Correct. But you should extend the decision whether a package should be built in a parallel make to the machine admin as well. When both say it's ok to make use of -jX, then go for it. If one of these two have any objections, then leave out the -j option. > Thus, my counter-proposal: > Let's allow maintainers to use make -jX according to their common > sense, requiring obeying an env variable to opt out. I would prefer an opt-in variable. > Rationale: > Nearly every buildd and nearly every user building the packages on > his own will benefit from -j2 [2], even on non-SMP. Unless it's a > piece of heavily-templated code, any modern box will have enough > memory to handle it. The maintainer know whether the code is heavily > templated or not. > > Giving the choice to someone who doesn't know the package (upstream > and/or the maintainer) will lead to _random_ FTBFSes. Therefore a double opt-in would be nice (machine admin and package maintainer saying it's OK). > Having DEB_MAKE_NON_PARALLEL [3] disable this behavior would make > m68k happy, while giving a reasonable default for everyone else. I don't think it's good to define an opt-out variable (*_NON_PARALLEL). Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even better it would be to use something existing: CONCURRENCY_LEVEL as Don Amstrong suggests. If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer can make use of -jX. Keep in mind that there are archs that don't have that much resources to build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M RAM, but for example armeb which has only 32M. I would like to see a method to allow usage of other compile accelerators as well (distcc (for using crosscc, but testing locally), ccache, etc). -- Ciao...//Fon: 0381-2744150 Ingo \X/ SIP: [EMAIL PROTECTED] gpg pubkey: http://www.juergensmann.de/ij/public_key.asc -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: proposed mozilla-firefox security update, needs testing!
On Tue, Jun 27, 2006 at 10:54:47PM -0400, Eric Dorland wrote: > > > > > > Could you send me a copy of your bookmarks.html file privately? > > > > Eric, was this reproducible? > > Unfortunately it was very reproducible :( I'm not sure how to > proceed. Is anyone else using these patches that might be able to help > figure out where the problem lies? > No ... upstream does not do official rc testing for us. Though, if we can track down which specific patch causes this regression, they will help to figure it out. Did you try to track down which patch(es) cause these troubles? - Alexander -- GPG messages preferred.| .''`. ** Debian GNU/Linux ** Alexander Sack | : :' : The universal [EMAIL PROTECTED]| `. `' Operating System http://www.asoftsite.org/ | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 02:06:26AM -0700, Don Armstrong wrote: > On Wed, 28 Jun 2006, Adam Borowski wrote: > > Let's allow maintainers to use make -jX according to their common > > sense, requiring obeying an env variable to opt out. > > Why not just have some ENV variable (CONCURRENCY_LEVEL?) which > specifies the maximum -j; the package maintainer is free to choose any > level equal to or below that. > > If CONCURRENCY_LEVEL is not set, the rules file must assume > CONCURRENCY_LEVEL=1 (or not specify -j?); this will keep what should > be the current status quo the same, and allow buildd's and other > builders to set this variable to a reasonable level for their system. I would rather have it default to "a reasonable value on an average uniprocessor box, according to the maintainer's common sense". Otherwise, nearly no one will have this variable set and thus the whole benefit will be lost. And cutting the build time by half or more is a GREAT thing to have. > (or not specify -j?) no -j => 1 -j with no value => INFINITY This is certainly not what you want for a package build; unlimited -j is useful for other uses of make. -jX (or -j X, the space is BAD for an option with an optional argument) => X processes > This has the disadvantage of not automatically using -j for every > package and requiring maintainer buy in to see results... but > presumably those packages where it actually makes a difference will > adopt it just so maintainer builds don't take as long. Why would you restrict this just to maintainer builds? Speeding up buildds and user builds would be a worthy goal, too. An obscure env var hardly anyone knows about means that hardly anyone will be affected. The m68k buildd admins are here, listening, so they can add this line. Or, another approach: if CONCURRENCY_LEVEL is unset, debian/rules can check the amount of memory present and/or the number of CPUs and make a guess. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, 28 Jun 2006, Adam Borowski wrote: > Let's allow maintainers to use make -jX according to their common > sense, requiring obeying an env variable to opt out. Why not just have some ENV variable (CONCURRENCY_LEVEL?) which specifies the maximum -j; the package maintainer is free to choose any level equal to or below that. If CONCURRENCY_LEVEL is not set, the rules file must assume CONCURRENCY_LEVEL=1 (or not specify -j?); this will keep what should be the current status quo the same, and allow buildd's and other builders to set this variable to a reasonable level for their system. This has the disadvantage of not automatically using -j for every package and requiring maintainer buy in to see results... but presumably those packages where it actually makes a difference will adopt it just so maintainer builds don't take as long. Don Armstrong -- There is no mechanical problem so difficult that it cannot be solved by brute strength and ignorance. -- William's Law http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#375809: ITP: python-chardet -- character encoding auto-detection in Python
Package: wnpp Severity: wishlist Owner: Piotr Ozarowski <[EMAIL PROTECTED]> * Package name: python-chardet Version : 1.0 Upstream Author : Mark Pilgrim <[EMAIL PROTECTED]> * URL : http://chardet.feedparser.org/ * License : GPL Programming Lang: Python Description : character encoding auto-detection in Python It takes a sequence of bytes in an unknown character encoding, and attempts to determine the encoding. . This library is a port of the auto-detection code in Mozilla. -- System Information: Debian Release: testing/unstable APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: i386 (i686) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.16.19-grsec Locale: LANG=pl_PL, LC_CTYPE=pl_PL (charmap=ISO-8859-2) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RFT: please test mdadm/experimental
Meep. I have done some major work on mdadm in experimental, which is now at version 2.5.2-1. I still have not received enough success reports to make me comfortable enough to upload to unstable, so here's another request for testing (or else etch will ship with 2.4.1). I am 95% confident that it's a smooth upgrade, though it helps if you make sure that the output of the script /usr/share/mdadm/mkconf looks okay before you reboot. Come and find me on IRC if you need reconfirmation (see below). Previously, mdadm required a newer version of initramfs-tools, one which did not exist yet. I have addressed this issue with this version, meaning that you can install it with *any* initramfs-tools out there, and it will just do what it can do. Details are in http://madduck.net/~madduck/scratch/mdadm.README.experimental, which is a little more up to date than the included /usr/share/doc/mdadm/README.experimental . Please test mdadm 2.5.2-1 and drop me a line if it works, or if it does not! The package is in experimental or available here: http://debian.madduck.net/repo/dists/experimental/main/binary-i386/admin/ http://debian.madduck.net/repo/dists/experimental/main/binary-amd64/admin/ Again, do not hesitate to bug me on IRC (#debian-devel/irc.debian.org) if you have questions or need assistance. Thanks, -- Please do not send copies of list mail to me; I read the list! .''`. martin f. krafft <[EMAIL PROTECTED]> : :' :proud Debian developer and author: http://debiansystem.info `. `'` `- Debian - when you have better things to do than fixing a system "the vast majority of our imports come from outside the country." - george w. bush signature.asc Description: Digital signature (GPG/PGP)
Re: new tar behavior and --wildcards
On Wed, Jun 28, 2006 at 07:02:15AM +0200, Christian Perrier wrote: > > Debian still has to provide an upgrade path for users upgrading from Sarge. > > We cannot blindly break users scripts. > > Here, the only way seems to be putting an entry in NEWS.Debian (for > users script, ie things not under our control). In addition, I would suggest we reinstate the previous behaviour, but display a warning when wildcards are used but --wildcards is not set. The warning would tell people about the migration and explains they must fix their scripts to use --wildcards before upgrading to etch+1. This way it is easy to find problematic scripts. Cheers, -- Bill. <[EMAIL PROTECTED]> Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 03:17:27AM +0200, Henning Makholm wrote:
> > If package maintainer wants to build it faster on their own machine, I
> > would imagine that checking for an environment variable (DEB_MAKE_OPTS
> > or something, perhaps?) and using that would be the way to go. By
> > default, build with a single processor.
This would affect every single package, and you can't do that. While
a vast majority of C code will build correctly, not every package is
SMP-safe. [1]
> If I understand the problem correctly, it is not even necessary to
> modify debian/rules to get this behavior. If the interdependencies
> are properly declared,
>
> $ debian/rules -j42 binary
>
> should do the trick, as GNU make is smart enough to pass the option
> down to sub-makes that it starts.
Actually, this is a bad idea; debian/rules are specifically the kind
of makefiles that typically rely on the order in which dependencies
are built. This is a bug as it breaks make -k, but as -k hardly ever
makes sense with regard to debian/rules, this is nearly totally
untested.
On the other hand, making builds significantly faster is not
something that you can shake a stick at. Typically make -jX is faster
even on uniprocessor, and I don't need to tell you why it's much
faster on SMP.
Too bad, a C++ build where every file takes 1GB memory obviously
should not be parallelized. Also, no one but the maintainer knows
whether a package is SMP-clean or not. You cannot guess this in an
automated way.
Thus, my counter-proposal:
Let's allow maintainers to use make -jX according to their common
sense, requiring obeying an env variable to opt out.
Rationale:
Nearly every buildd and nearly every user building the packages on
his own will benefit from -j2 [2], even on non-SMP. Unless it's a
piece of heavily-templated code, any modern box will have enough
memory to handle it. The maintainer know whether the code is heavily
templated or not.
Giving the choice to someone who doesn't know the package (upstream
and/or the maintainer) will lead to _random_ FTBFSes.
Having DEB_MAKE_NON_PARALLEL [3] disable this behavior would make
m68k happy, while giving a reasonable default for everyone else.
And yeah, I'm guilty of using unconditional make -j4 in one of my
packages (not in the archive yet), too.
[1] A real-world example:
[ make_commands ]
#!/bin/sh
rm -f commands.h
rm -f load_commands.h
ARGS="_command(char *arg, struct session *ses)"
while read FUNC
do
if [ -z "$FUNC" ]
then
continue
fi
case $FUNC in
\;*);;
\#*)echo "$FUNC" >>commands.h;echo "$FUNC" >>load_commands.h
;;
\**)FUNC=`echo "$FUNC"|sed 's/^\*//'`
echo "extern struct session *$FUNC$ARGS;" >>commands.h
echo "add_command(c_commands, \"$FUNC\",
(t_command)${FUNC}_command);" >>load_commands.h
;;
*) echo "extern void$FUNC$ARGS;" >>commands.h
echo "add_command(commands, \"$FUNC\", ${FUNC}_command);"
>>load_commands.h
;;
esac
done
==
This script produces two files, commands.h and load_commands.h, both
of which should be regenerated every time the input data changes.
A naive Makefile will just have both of these depend on an
invocation of make_commands. This will work correctly and be
unnoticed for years until someone gives "-j" to make.
Would you bet that every single of your upstreams knows to avoid such
subtle errors?
[2] I benchmarked this with an old gcc several years ago on an
uniprocessor, and -j4 (!) gave the best results. Today, with gcc-4.2
-j2 won.
[3] This name sucks. Give a better one if you care :p
--
1KB // Microsoft corollary to Hanlon's razor:
// Never attribute to stupidity what can be
// adequately explained by malice.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#375797: ITP: zope-compositepack -- allows the Plone Manager to build composite pages
Package: wnpp Severity: wishlist Owner: alex bodnaru <[EMAIL PROTECTED]> * Package name: zope-compositepack Version : 1.0 Upstream Author : Godefroid Chapelle ([EMAIL PROTECTED]) * URL : http://plone.org/products/compositepack * License : Zope Public License (ZPL) Version 2.1 Programming Lang: Python Description : allows the Plone Manager to build composite pages (Include the long description here.) CompositePack is a product that allows the Plone Manager to build composite pages by manually aggregating archetype content from his site. . Composition of content is made through a pseudo WYSIWYG user interface: the design view. A composite page has a layout which defines its structure. Composite elements are displayed through viewlets. . Both layouts and viewlets are acquired from the skin, which implies they are customizable. . Layouts and viewlets are registered through the composite_tool in ZMI. -- System Information: Debian Release: testing/unstable APT prefers testing APT policy: (500, 'testing') Architecture: i386 (i686) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.16-skas3-v8.2 Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#375801: ITP: zope-cmfcompositepage -- visually combine page fragments into complete pages
Package: wnpp Severity: wishlist Owner: alex bodnaru <[EMAIL PROTECTED]> * Package name: zope-cmfcompositepage Version : 0.2 Upstream Author : Shane Hathaway, Zope Corporation, <[EMAIL PROTECTED]> * URL : http://hathawaymix.org/Software/CompositePage * License : Zope Public License (ZPL) Version 2.1 Programming Lang: Python Description : visually combine page fragments into complete pages (Include the long description here.) CompositePage is a new way to assemble pages for the World Wide Web. Through the use of Zope technology, browser-based drag and drop, and custom context menus, CompositePage makes it easy to visually combine page fragments into complete pages. -- System Information: Debian Release: testing/unstable APT prefers testing APT policy: (500, 'testing') Architecture: i386 (i686) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.16-skas3-v8.2 Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 03:17:27AM +0200, Henning Makholm wrote: > Scripsit Lars Wirzenius <[EMAIL PROTECTED]> > > > If package maintainer wants to build it faster on their own machine, I > > would imagine that checking for an environment variable (DEB_MAKE_OPTS > > or something, perhaps?) and using that would be the way to go. By > > default, build with a single processor. > > If I understand the problem correctly, it is not even necessary to > modify debian/rules to get this behavior. If the interdependencies > are properly declared, > > $ debian/rules -j42 binary > > should do the trick, as GNU make is smart enough to pass the option > down to sub-makes that it starts. Yes and no. * It only works if debian/rules calls make as `$(MAKE)'; not if it calls it as `make' or even `/usr/bin/make'. In the latter case, it just considers it some random program (rather than a sub-make) and will not pass the flags it received down to it. * It only works if the build system of the package does indeed use GNU make, and not something else, like pmake, smake, or SCons. -- Fun will now commence -- Seven Of Nine, "Ashes to Ashes", stardate 53679.4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
