Re: User extensions
On Sat, Oct 23, 2010 at 4:05 PM, Stefano Lattarini
stefano.lattar...@gmail.com wrote:
Hi Valentin. Thank you very much for your patch implementing this
much-desired automake feature. Here is a preliminary review...
On Friday 03 September 2010, Valentin David wrote:
* The lang_*_rewrite are added to the Language structure. The default
is lang_sub_obj. They do not return anymore the object extension
because the field 'output_extensions' already computes it.
Hmm... looking at the git diffs, this seems like a useful refactoring
by itself. Could this be done in separate, preparatory patch preceding
the implementation of the user extensions support proper?
Sure, I can make a series of patches like that. But there might be
also a lot of refactoring afterwards as well. For example language
dependent code should probably all move in a different file and be
imported via the extension thing.
* Compilers that generate source files might still generate dependency files.
* --libdir= can be called several times, the arguments can also have a
list of paths separated by a colon.
We should really use the system PATH separator here, for better portability
(such PATH separator is already computed and AC_SUBST'd by configure as
`...@path_separator@').
If you say so. Is it normal to have different syntax depending from
which shell you call automake?
Empty paths correspond to the original libdir.
I guess this is done so that the user can override the previously-passed
custom libdirs with the default one, without having to know its complete
path -- right? I think this is useful and smart, but couldn't we find a
better speacial value than empty to indicate the default libdir?
I thought I have seen it somewhere else, but I do not remember which
software. If you have any suggestion instead of empty, then shoot.
The test tests/user_def_lang.test should show how the feature works.
We'll need much deeper test than the one provided by your patch (I can help
writing them if you want).
I do agree. I did more modifications since that patch, and I
understood it was not good.
I can understand that loading perl files is not really nice because
there easily can be problems of backward compatibility.
But it's surely better than the present situation IMHO.
It also mean that an API documentation will be required in the future.
+ my $r
+ = {$lang-rewrite} ($directory, $base, $extension,
$nonansi_obj, $have_per_exec_flags, $var);
What about a saner indentation here? As in:
my $r = {$lang-rewrite} ($directory, $base, $extension,
$nonansi_obj, $have_per_exec_flags,
$var);
[...]
The function implementation below doesn't respect the GCS (GNU coding
standards) w.r.t. use of white spaces, indentation and brackets placement.
And while I dislike what the GCS mandate on these matters, we should
strive to be consistent anyway.
I trusted emacs...
+sub libfile ($)
+{
+ my ($f) = @_;
+ foreach (@userlibdir) {
+ return $_/$f
+ if -r $_/$f;
Wouldn't a `-f $_/$f' be more correct here?
Is there a case we want to know if a file exists but fail reading it?
Should we continue to search for another file? For me both are OK. But
I can change to -f
+ }
+ return $userlibdir[0] . / . $f;
Shouldn't this be `$libdir' instead of `$userlibdir[0]'? Otherwise,
No. $userlibdir[0] should contain the first directory in $libdir. If
it was not like that, then there is a problem.
Automake even fails to bootstrap itself, giving the error message:
Use of uninitialized value $userlibdir[0] in concatenation (.) or string at
./automake.tmp line 7088.
automake.tmp: error: cannot open /am/header-vars.am: No such file or
directory
I am not sure how you got this one.
With the change I'm proposing, Automake bootstrap itself *and* passes
the whole testsuite.
Is not it make distcheck enough? Because in my case it was working.
+ 'libdir=s' = sub { push @userlibdir,
+ (map { ($_ eq '') ? $libdir : $_ }
+ split (':', $_[1])); },
We should use $PATH_SEPARATOR here, not ':'.
OK
+ foreach (@userlibdir) {
+ my $dir = $_;
+ if (-d $dir/lang) {
+ opendir(LANG_DIR, $dir/lang) || next;
+ my @files=readdir(LANG_DIR);
+ foreach (@files) {
+ do $dir/lang/$_
+ if (-r $dir/lang/$_ $_ =~ /\.pm$/);
Wouldn't a `-f $dir/lang/$_' be more correct here?
To me, that is the same.
+user_def_language.test \
Typo; should be `user_def_lang.test'.
I am wondering if I sent you the right patch...
diff --git a/tests/user_def_lang.test b/tests/user_def_lang.test
new file mode 100755
index 000..9b015da
--- /dev/null
+++ b/tests/user_def_lang.test
This test is missing the #! /bin/sh shebang line, the copyright header, and
a brief comment explaining what is being tested.
+cat
Re: using autoconf with ppuxlc++,spuxlc++
Hello John, * Eric Blake wrote on Wed, Oct 27, 2010 at 01:18:59AM CEST: On 10/26/2010 04:31 PM, John Wohlbier wrote: We're trying to use autoconf in a hybrid environment that includes the CellBE. I have a configure.ac for the CellBE PPE core and one for the SPE core. I also have some macros for setting up the compilers and their flags since it appears that AC_PROG_CC and AC_PROG_CXX do not support these compilers. I have what I feel are valid compiler flags for, in this case, ppuxlc++. However, I think I have a problem with the interaction of autoconf/libtool and cannot get my sources to compile. Below is what I see on output. To try and narrow down on the error ppuxlc++: 1501-208 (S) command option D is missing a subargument I went through by hand and removed all of the -D arguments, and it still failed. Finally, I removed the -MD argument and it compiles. Unfortunately I don't know enough about autoconf/libtool to know what the -MD -MP -MF arguments are or where they come from. Can anyone provide any hints or suggestions on how to figure out what is going on? How exactly do you invoke configure, what does configure output as dependency mechanism for these two compilers, where are the manpages for the compilers? Thanks, Ralf
Re: User extensions
On Sat, Oct 23, 2010 at 5:27 PM, Ralf Wildenhues ralf.wildenh...@gmx.de wrote: I haven't looked at the patch in detail yet, but will, now that the assignment papers are done (thanks!). It is done. -- Valentin David valentin.da...@gmail.com
Re: Strangeness with m4_include and aclocal.
On 2010-10-23 15:23 +0200, Stefano Lattarini wrote: Now I do. This happens because `aclocal', to find the includes in an input file, *grep* the file's content, looking for literal `m4_include' (and similar), without analyzing m4 traces (not sure why aclocal acts this way, but I suspect there is a good reason behind this behaviour). So it seems that there is a strange dichotomy between automake and aclocal: automake apparently uses the traces to figure out the aclocal dependencies (also for 'make dist'), but aclocal _doesn't_ use the traces to actually do things. So I think your first problem is just an aclocal limitation we should resign to live with. But I also think that such a limitation should be documented explicitly, since it's by no means obvious. It's especially non-obvious because the other autotools handle this just fine. so it believes that aclocal.m4 is out of date when foo.m4 is updated. And this is right, because configure.ac includes foo.m4, and since aclocal.m4 depends on all the content of configure.ac, *also the parts m4_included from other files*, aclocal.m4 must be considered out-of-date when it's older than foo.m4. What is wrong is that aclocal fails to update the aclocal.m4 timestamp, so that the rules to rebuild it are uselessly called over and over ... Ah right. Foo.m4 might call AC_DEFUN or another macro defined in this way, and aclocal needs to know about these things. In my instance, foo.m4 _doesn't_ do either of these things so I didn't notice this. If I'm understanding correctly, then, aclocal _isn't_ going to find this information from foo.m4 because it doesn't _know_ about foo.m4. I haven't tested this, but I suspect it also means that my workaround isn't going to cut it in the general case where foo.m4 uses or defines macros that aclocal needs to find. Nevertheless, if both aclocal and automake agreed on when to update aclocal.m4, then the workaround _would_ solve my specific problem. Thanks for your time on this matter, -- Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)
Re: Using -MMD instead of -MD for depndency generation
Hello Holger, * Holger Freyther wrote on Mon, Oct 25, 2010 at 09:39:29AM CEST: On 10/25/2010 08:09 AM, Ralf Wildenhues wrote: * Holger Freyther wrote on Sat, Oct 23, 2010 at 10:47:47PM CEST: when typing make in the build directory of WebKit/GTK+ it takes multiple minutes until the first file is compiled. Can you go into (a lot) more detail here? What happens during those minutes, which processes are running (look at ps output etc.)? I am talking about executing make after the source has been built once. E.g. there is only one file changed and I want to rebuild the library with this single new file. My benchmark is to type 'time make' on unchanged source. From my expectation it will parse the GNUmakefile and all the files (the .Plo, Po with the dependencies) included by it, then it needs to verify the dependencies of all these rules for the 'all' target and then exits. Right now this takes 5:42m usertime (+- 10 seconds). OK. That's a lot. My half-stable-half-testing system needs some love to build webkit OOTB so I can't provide comparison data yet. Is this cold-cache or hot-cache? Only the first time the code is compiled, or the build tree is used, i.e., does it happen again after 'make clean'? Or are you talking about configure (-q?) run time here? My system is a MSI X340 with a U7300 @ 1.30GHz and had about 2GB of RAM available, it is running 2.6.36rc8 (but all kernels have shown this behavior) on a Fedora F14 installation. GNU make is of version 3.82, automake is 1.11.1. This is with a 'hot-cache' but with only 2GB of RAM not every file will fit into the page cache. The disk speed is quite okay, slowed down a bit due dm-crypt. The system is not loaded at all, it runs a wm, some gnome-terminals with remote ssh sessions, when executing make it will be the process that takes 99% of the CPU. Have you tried moving the build tree to a ramdisk (/dev/shm)? Do you use the noatime or relatime mount option? If not, please seriously consider doing so on mount points where you keep temporary data (not your mail folders!). For most effective edit-compile-test cycle, I typically move to a big system with lots of RAM and use make -jN with the build tree (not necessarily the source tree) on ramdisk and optionally also use ccache and/or distcc (if the system is not that big). I've found the ramdisk to make file system accesses fairly irrelevant. Eventually, the right answer to lots of file accesses for builds this size or larger is to use a beta build system. I hope somebody ports tup to GNU make syntax, or extends GNU make to have tup-like functionality. If the above all doesn't help make your problem irrelevant, I might be willing to look into optionally going for -MMD instead of -MD, but I do think that from a correctness POV it is not the right thing. If your installed software changed, then you need to also rebuild. Cheers, Ralf
Re: Using -MMD instead of -MD for depndency generation
On Wed, 2010-10-27 at 20:51 +0200, Ralf Wildenhues wrote: I am talking about executing make after the source has been built once. E.g. there is only one file changed and I want to rebuild the library with this single new file. My benchmark is to type 'time make' on unchanged source. From my expectation it will parse the GNUmakefile and all the files (the .Plo, Po with the dependencies) included by it, then it needs to verify the dependencies of all these rules for the 'all' target and then exits. Right now this takes 5:42m usertime (+- 10 seconds). I saw this come across my inbox and thought I'd give it a tree; 5:42 seems like a very, very long time. I downloaded the tarball you linked to, to my system but wasn't able to build; it configured and built about half the code, then failed: CXX WebKit/gtk/WebCoreSupport/libwebkitgtk_1_0_la-FullscreenVideoController.lo WebKit/gtk/WebCoreSupport/FullscreenVideoController.cpp:32: fatal error: gst/gst.h: No such file or directory But I do have libgstreamer dev packages installed: webkit-1.3.5$ locate gst.h /usr/include/gstreamer-0.10/gst/gst.h Do I need to do something else? Anyway, with half the tree built (I think): webkit-1.3.5$ find -name \*.o | wc -l 1437 make 3.81 takes about 8 seconds on my system to get back to trying to build the above file, and failing. I do notice that make 3.82 takes a lot longer (23s !!) so there's definitely something I need to check out in that respect. But 5m seems pretty crazy. FWIW, I have a Intel Core2 6600 at 2.4GHz with 2G RAM, running 64bit 2.6.35+ kernel on an ext4 partition.
Re: Using -MMD instead of -MD for depndency generation
Hello Paul, * Paul Smith wrote on Wed, Oct 27, 2010 at 10:25:03PM CEST: I downloaded the tarball you linked to, to my system but wasn't able to build; it configured and built about half the code, then failed: CXX WebKit/gtk/WebCoreSupport/libwebkitgtk_1_0_la-FullscreenVideoController.lo WebKit/gtk/WebCoreSupport/FullscreenVideoController.cpp:32: fatal error: gst/gst.h: No such file or directory Rerun with 'make V=1' to see the command that causeed the failure. Anyway, with half the tree built (I think): webkit-1.3.5$ find -name \*.o | wc -l 1437 make 3.81 takes about 8 seconds on my system to get back to trying to build the above file, and failing. I do notice that make 3.82 takes a lot longer (23s !!) so there's definitely something I need to check out in that respect. But 5m seems pretty crazy. Agreed. Thanks for sharing this data! Ralf
Re: Using -MMD instead of -MD for depndency generation
On 10/27/2010 10:25 PM, Paul Smith wrote: On Wed, 2010-10-27 at 20:51 +0200, Ralf Wildenhues wrote: But I do have libgstreamer dev packages installed: webkit-1.3.5$ locate gst.h /usr/include/gstreamer-0.10/gst/gst.h hmm. Could you check the config.log if video was enabled? Maybe the configure decided that video was disabled but we still build that file... Do I need to do something else? Well, the problem is more with the .Po and .Plo files. If half the files contain '#dummy' it will probably still be a lot faster, if all of them contain the dependencies it will be a lot slower (at least for me). FWIW, I have a Intel Core2 6600 at 2.4GHz with 2G RAM, running 64bit 2.6.35+ kernel on an ext4 partition. My desktop is still in a container on some boat and I am stuck with my lower end laptop. Is there anything I can help with? bisect GNU make 3.81 - 3.82 to see when it got slower? Or anything on our GNU make/automake usage? E.g. we use a non recursive makefile
Question about automake fails to pass the correct parameters to pax when using large UID/GID
Hi, I'm a developer from WindRiver, and I have a question about automake. When I using a large UID/GID(2097151) to configure and build a linux project, gnu tar fails and automake select pax as a replacement. However, automake seems fail to pass the correct parameters to pax(we are using ustar as archive format), the build hangs at: --- Configuring: rpm --- configure: creating cache ldat_config_cache Configuring RPM Package Manager (RPM), Version 5.1.9 === INITIALIZATION === checking build system type... i686-wrs-linux-gnu checking host system type... i686-wrs-linux-gnu checking target system type... i686-wrs-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking how to create a ustar tar archive... ATTENTION! pax archive volume change required. Ready for archive volume: 1 Input archive name or . to quit pax. Archive name I think the problem is caused by the maximum limitation of UID/GID for ustar archive format (Maximum value of UID/GID is 2097151). So if it is proper to increase the limitation for UID/GID in ustar? I don't know if there is any considerations for this limitation? Thanks, Xufeng Zhang
