make -j1 fails

[Pippijn van Steenhoven]

Hi,

I am using the FreeBSD make utility to build an automake based
distribution. When I run make with a -j option (even -j1), it fails.
Running make with a -j option makes the utility behave differently. The
issue only arises with built sources (in my case, yacc generated C code),
since those trigger the inference rule ".c.lo". Perhaps a solution would
be to also generate explicit rules for built sources.

I have attached the output of running make with and without -j1.

Regards,
Pippijn van Steenhoven
> make V=1 axl.lo 
/bin/sh ../autoconf/ylwrap `test -f 'src/compiler/phases/parse/axl.y' || echo 
'../'`src/compiler/phases/parse/axl.y y.tab.c axl.c y.tab.h axl.h y.output 
axl.output -- byacc  -d
axl.h is unchanged
/bin/sh ./libtool  --tag=CC--mode=compile gcc -DHAVE_CONFIG_H  -I. -I.. 
-I./include/config  -I../include -I./include  -I../include -I./include -D_DEBUG 
 -DPREFIX='"/usr/local"' -I../`echo axl.c | grep -o  
'src/library/\w\+'`/include  -g -O2 -pipe -ggdb3 -pedantic -ansi 
-fvisibility=hidden -MT axl.lo -MD -MP -MF .deps/axl.Tpo -c -o axl.lo axl.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I./include/config -I../include 
-I./include -I../include -I./include -D_DEBUG -DPREFIX=\"/usr/local\" 
-I..//include -g -O2 -pipe -ggdb3 -pedantic -ansi -fvisibility=hidden -MT 
axl.lo -MD -MP -MF .deps/axl.Tpo -c axl.c  -fPIC -DPIC -o .libs/axl.o
mv -f .deps/axl.Tpo .deps/axl.Plo
> 
> make V=1 axl.lo -j1
/bin/sh ../autoconf/ylwrap `test -f 'src/compiler/phases/parse/axl.y' || echo 
'../'`src/compiler/phases/parse/axl.y y.tab.c axl.c y.tab.h axl.h y.output 
axl.output -- byacc  -d
axl.h is unchanged
/bin/sh ./libtool  --tag=CC--mode=compile gcc -DHAVE_CONFIG_H  -I. -I.. 
-I./include/config  -I../include -I./include  -I../include -I./include -D_DEBUG 
 -DPREFIX='"/usr/local"' -I../`echo axl.c | grep -o  
'src/library/\w\+'`/include  -g -O2 -pipe -ggdb3 -pedantic -ansi 
-fvisibility=hidden -MT axl.lo -MD -MP -MF 
.deps/src/compiler/phases/parse/axl.Tpo -c -o axl.lo axl.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I./include/config -I../include 
-I./include -I../include -I./include -D_DEBUG -DPREFIX=\"/usr/local\" 
-I..//include -g -O2 -pipe -ggdb3 -pedantic -ansi -fvisibility=hidden -MT 
axl.lo -MD -MP -MF .deps/src/compiler/phases/parse/axl.Tpo -c axl.c  -fPIC 
-DPIC -o .libs/axl.o
axl.c:3626: fatal error: opening dependency file 
.deps/src/compiler/phases/parse/axl.Tpo: No such file or directory
compilation terminated.
*** Error code 1
1 error
> 


signature.asc
Description: Digital signature

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Stefano Lattarini wrote on Wed, Jan 12, 2011 at 09:58:36PM CET:
> On Wednesday 12 January 2011, Ralf Wildenhues wrote:
> > The upsides are obvious.
> >
> If I'm not deluding myself, most of the contents of my proposal were
> aimed at showing why I believe that requiring GNU make is a reasonable
> and sensible policy, not to show what the advantages of such a policy
> would be -- quoting myself:
> 
>  `` And the gains in terms of maintainability, testability, and
> possibility of optimization are obvious. ''

Sure.  You discuss the upsides of the strategy.  What I was asking for
is a discussion of the downsides of requiring GNU make, and arguing why
they are not a problem *for our users*.  *That* is what is needed to
convince those who do consider requiring GNU make outright a regression.

> > I was planing to introduce optional GNU make-specific code, and allowing
> > to let the user specify "my project requires GNU make anyway", which
> > would enable Automake to emit better code.  Arguably more complex than
> > requiring GNU make outright, but it wouldn't throw away all the make
> > portability work that exists in Automake.
> >
> Hmm... so you're telling me that "This code should be kept because it
> had been difficult to write it" is an acceptable rationale? ;->

No.  I'm saying that that portability is a *feature*.  You want to
remove that feature.  You need to bring solid reasons why the feature
is worthless, used by nobody, and nobody is interested in it.  And, of
course, why we should abandon the GNU Coding Standards mandate of only
requiring a small fixed set of Posix tools.

This thread has already shown that to not be true, judging from a couple
of the other answers.

> (BTW, that portability-related work has already and definitely served
> the purpose of making automake usage more widespread, so I don't think
> the efforts that went into it would ever be "wasted", even if the code
> they produced is going to be removed).

OK, so you're acknowledging that the feature serves a useful purpose.
Why should we then remove it?

> > But let me rephrase the critique in a poignant way: if you want to
> > require GNU make anyway, what is your rational to not use quagmire
> > instead of Automake?
> >
> You mean this?
>  

Yes.  Please drop ridiculing in an otherwise sensible discussion, it
weakens your argumentation.  (It's also not the first time we've had
this discussion on this list.)

Tom Tromey (who, should you not know, has done more work on Automake
than anybody else), started quagmire, as a response to the idea that
GNU make should be required and exploited.  It is a rational answer
given the following thoughts: when you require GNU make, you

- want to allow your users to use GNU make to its fullest extent,
- can fix the problem of non-parallel configure tests,
- can avoid the slow m4+perl preprocessing steps.

* Stefano Lattarini wrote on Wed, Jan 12, 2011 at 10:12:37PM CET:
> On Wednesday 12 January 2011, Ralf Wildenhues wrote:
> > Apart from that, if Automake requires GNU make, its users would rightly
> > demand that Automake ought to understand GNU make-specific constructs.
> >
> In which sense exactly?  If you mean in something like:
> 
>  bin_PROGRAMS = $(call my-macro,foo,bar)

Well, more likely, things like

foo_SOURCES = $(wildcard *.cc *.h)

which are asked for every few months.  We have a FAQ entry about it.

> then they should resign not to have it working for quite a long
> time I fear.

But that's something that would make me cringe if I had to use GNU make
anyway.

> After all, automake is just a pre-processor, not a GNU make superset.

But that mis-feature would be bug number One in this case.

> I stress this because I think that the only way not to have the
> hypotetical "automake2" end up as vaporware would be to start from
> the current automake implementation, ensuring we don't break the
> API nor regress the testsuite while we convert to gmake-only output.

* Stefano Lattarini wrote on Wed, Jan 12, 2011 at 10:36:52PM CET:
> The "answer" I was speaking about would have been concerned with why I think
> that the quagmire *design* and *roadmap* are broken (even ignoring its "less
> than excellent" developement status).

Actually, I consider the design not so bad, but that's fairly irrelevant
to this discussion.

> Okay, at this point I can as well write that answer out summarily:
> 
>  - I want something that is backward-compatible with automake 1.11 as
>much as possible (I mean 98/99% compatible), and that works from
>"day 0".  Otherwise it won't stand any real chance of being used
>by real-worls projects.

Well, there are counter examples (CMake, a couple of others) that show
that it is possible.  But it would be a large, not a small, effort, and
it is not clear that a new contester would stand a chance now.  One
cannot tell without trying though.

You are probably right in that many alternative build tools have not
taken of

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Юрий Пухальский]

Good day.

I'll throw my humble userish $0.02 in here.

First of all, i like the idea of autotools working in any POSIX
enviroment, requiring a set of tools that is everywhere by definition.
Because when for example i try to compile my project in hostile
environment with suspicious admins that watch every my step -
requiring GNU make would be a hindrance. There are very many
*seriously* old systems out there, at least here, in poor Russia.

Also, if we step away from requiring only a standard set of tools,
that makes a precedent. Next step would easily be to require bash
(which, by the way, i believe to be much more ubiquitous that
gnu/bsd-make). That will surely clarify the utilities' code, if we
drop the support of them old shells. So the same reasoning will apply
to this also. So the trend, i'm afraid, will be like this - stick to
GNU software. I have nothing against it, but that is a thing to
consider - a balance between developers' lazyness and users' comfort.
(I already don't like that the autotools' set requires gnu M4).

2011/1/13 Stefano Lattarini :
> On Wednesday 12 January 2011, Ralf Wildenhues wrote:
>> > * Stefano Lattarini wrote on Wed, Jan 12, 2011 at 07:01:47PM CET:
>> >
>> > > And more than this -- brace yourself -- I'm starting to think that
>> > > automake should *really* start supporting *only* GNU make (at least
>> > > from version 3.75 or so).
>> >
>> > If you want support for this, then you need to discuss away the
>> > downsides (i.e., convince those _opposed_ to the idea, not the
>> > rest).  The upsides are obvious.
>>
>> Apart from that, if Automake requires GNU make, its users would rightly
>> demand that Automake ought to understand GNU make-specific constructs.
>>
> In which sense exactly?  If you mean in something like:
>
>  bin_PROGRAMS = $(call my-macro,foo,bar)
>
> then they should resign not to have it working for quite a long
> time I fear.
>
> After all, automake is just a pre-processor, not a GNU make superset.
>
> I stress this because I think that the only way not to have the
> hypotetical "automake2" end up as vaporware would be to start from
> the current automake implementation, ensuring we don't break the
> API nor regress the testsuite while we convert to gmake-only output.
>
>> I don't consider that a feasible endeavor, short of reimplementing
>> GNU make inside Automake.
>>
>
> Regards,
>   Stefano
>



-- 
«The good thing about standards is there are so many to choose from.»

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Steffen Dettmer]

On Thu, Jan 13, 2011 at 3:39 AM, Bob Friesenhahn
 wrote:
> While GNU make is a really good 'make' program, I think that 'make' in
> general is a flawed concept.

Could you please explain this a bit?

I like the `make' concept; in some situations I even love it.
One example of such a situation is to create statistics from
productive log files.
This is an incremental, time consuming process. I wrote the stats
script in Makefile language (starting with a shebang line).
Intermediate results are stored as files (an automatic cache).
The process is very time consuming but thanks to make, it can be
aborted and restarted at any time (just the current file action
is lost). When restarted later, only the new files are processed
etc. The Makefile is very simple, it just has the rules for each
step, no need to worry how to get this in the right order, no
need to worry about parallelization or where to continue after
interruption. It just works.
(this has no relation to automake of course).

> If there was going to be a revolutionary change, then I would
> like to see a small embedded build engine be included which
> accomplishes the few functionalities needed from make, but also
> avoids needing additional programs.

I think the need for small dedicated additional programs often is
very useful, because by using the simple building blocks, quickly
complex solutions can be constructed, even if no one ever had a
related requirement ever.

In contrast to let's say "ant", a build tool popular in Java
world, does this differently. This is good for
platform-independent java building, but has IMHO serious
disadvantages and IMHO the big problem that it can do only "what
was built-in in advance". You can create .jar files but you
cannot use source code generation or even use some other packager
to bzip2 some output result in the end. Even compiling-in a
version number can become tricky.
(Trying to build C code with ant seems to be a bad idea anyway).
Surely I'm not capable to use ant correctly.

Maybe a future version of automake creates an efficient
GNUMakefile and a less efficient but more portable Makefile at
the same time, leaving it up to the user to select which one to use.

> A little bit of analysis will reveal that Automake really does
> not require much functionality from 'make'.  Probably 5% of
> what GNU 'make' is capable of.  I don't like it that 'make'
> depends on file timestamps, and that it is unable to cache what
> was used to create the current binary, and that dependency
> information from the build system needs to be cached in 'make'
> include files.

What is the problem with timestamps?
I don't like that ant does not depend on timestamps (leading to
rebuilt jars, resigning, which takes time and needs entropy etc).

Where else should dependency information be stored? Of course, if
you would have a special make-replacement dedicated (limited?) to
C building, it could be "smart", but I'm afraid the "smart"
things fail more often that the "simple" things.
I think the dependency issue arises from a kind of lazyness
(generate the deps automatically instead of writing them
explicitely). To save that for C code, there are tools for it
(gcc -M).

Do you mean that make is a good multi-purpose tool but not well
suited for C building, especially in automake environments?

oki,

Steffen

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Steffen Dettmer]

On Wed, Jan 12, 2011 at 10:36 PM,  wrote:
>  - I think that keeping configuration and build steps separated is
>   a very good idea.

Do you mean this is a good idea in the context of todays systems
- or -
Do you mean this is good idea in general and could be a design
criteria for future build environments?

I think I agree to the first (mostly because I assume if the
autotools developers and experts separate those steps, they do it
for a good reason), but I don't understand my this could be a
requirement in future systems.

Wouldn't it be great to type "make" which automatically knows by
depedencies that some configuration rules have to be executed
(i.e. to determine facts about the environment if they are not
available in form of small .h files or alike)?

If, for example, Makefiles would have rules to check for the
libraries as soon as needed etc, wouldn't this be good? Tests
that are not needed for the configuration to be built would not
even be executed (saving time).

What important points did I miss in my consideration?

oki,

Steffen

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Stefano Lattarini]

On Thursday 13 January 2011, Юрий Пухальский wrote:
> Good day.
> 
> I'll throw my humble userish $0.02 in here.
> 
> First of all, i like the idea of autotools working in any POSIX
> enviroment, requiring a set of tools that is everywhere by definition.
> Because when for example i try to compile my project in hostile
> environment with suspicious admins that watch every my step -
> requiring GNU make would be a hindrance. There are very many
> seriously old systems out there, at least here, in poor Russia.
>
OK, so you're telling me that asking to install just GNU make is
perceived by some admins as an unreasonable, suspicious request.
I'd never guessed by myself that this could be the case, really.
That's bad news for me, but I'm glad that I'm now aware of it at
least.

> Also, if we step away from requiring only a standard set of tools,
> that makes a precedent. Next step would easily be to require bash
> (which, by the way, i believe to be much more ubiquitous that
> gnu/bsd-make). That will surely clarify the utilities' code, if we
> drop the support of them old shells. So the same reasoning will apply
> to this also. So the trend, i'm afraid, will be like this - stick to
> GNU software. I have nothing against it, but that is a thing to
> consider - a balance between developers' lazyness and users' comfort.
> (I already don't like that the autotools' set requires gnu M4).
> 

Thanks for your contribution.  It's always useful to hear the
opinions and experiences of a real user.

Regards,
  Stefano

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Юрий Пухальский]

2011/1/13 Stefano Lattarini :
> On Thursday 13 January 2011, Юрий Пухальский wrote:
>> Good day.
>>
>> I'll throw my humble userish $0.02 in here.
>>
>> First of all, i like the idea of autotools working in any POSIX
>> enviroment, requiring a set of tools that is everywhere by definition.
>> Because when for example i try to compile my project in hostile
>> environment with suspicious admins that watch every my step -
>> requiring GNU make would be a hindrance. There are very many
>> seriously old systems out there, at least here, in poor Russia.
>>
> OK, so you're telling me that asking to install just GNU make is
> perceived by some admins as an unreasonable, suspicious request.
> I'd never guessed by myself that this could be the case, really.
> That's bad news for me, but I'm glad that I'm now aware of it at
> least.
It's not that they don't like GNU make, but - i work in the company
that makes card processing solutions - the banks are suspicious about
stuff very often. It's normal. It's not entirely impossible to have a
local copy of a gmake there to build the project, but these are the
questions one might need to answer: what's this, and what does it do,
what do you need it for... So i repeat, it's basically - as to my
experience - not a real problem, but merely a hindrance. Another thing
might be - and with me it was - to explain admin what i need. Because
sometimes they don't know how to install stuff from sources, or maybe
know how, but don't want to take responsibility (it might be the
reason in most cases).
The systems there very often lack gmake and stuff, probably because
they're installed it out of a box, then added oracle and there's all
to it, the system's ready for the tasks we need it for.
This i tell from my own experience with hostile environments i must
sometimes compile in:)
So yes, dependance on gmake will be seen as a regress at least by me.

>
>> Also, if we step away from requiring only a standard set of tools,
>> that makes a precedent. Next step would easily be to require bash
>> (which, by the way, i believe to be much more ubiquitous that
>> gnu/bsd-make). That will surely clarify the utilities' code, if we
>> drop the support of them old shells. So the same reasoning will apply
>> to this also. So the trend, i'm afraid, will be like this - stick to
>> GNU software. I have nothing against it, but that is a thing to
>> consider - a balance between developers' lazyness and users' comfort.
>> (I already don't like that the autotools' set requires gnu M4).
>>
>
> Thanks for your contribution.  It's always useful to hear the
> opinions and experiences of a real user.
>
> Regards,
>  Stefano
>



-- 
«The good thing about standards is there are so many to choose from.»

Re: High-Precision NFS Timestamps

[Eric Reischer]

As I said, we do use NTP to keep our machines in synchronization, every 
hour.  However, there's no time client that will keep everything aligned 
up to the nanosecond level, which is what ext4 timestamps are in.  
Besides, any amount of network lag is going to introduce small errors in 
this anyway.  The only temporary solution I can think of is to force the 
clock on the NFS server to be ~1 second behind all of the clients, so 
timestamps will never appear to be in the future.  Not a clean 
workaround, but it appears there's no other stopgap solution.


On 01/12/2011 09:58 PM, Bob Friesenhahn wrote:

On Wed, 12 Jan 2011, Eric Reischer wrote:

I recently upgraded our NFS fileserver to an ext4 filesystem, and 
since then we've been having clock skew warnings from make (3.81).  
Because the ext3 filesystem that was previously running on the NFS 
server didn't support high-precision timestamps, we didn't have clock 
skew warnings because an hourly cron job kept the clocks in 
relatively close sync.  However, since ext4 does support sub-second 
timestamps, we're getting warnings about clock skews on the order of 
a few milliseconds.  I tried adding our target objects


Usually the solution to this is to install and run ntp (Network Time 
Protocol, as offered by 'xntp') on the machines on your network. This 
seems to be quite effective on real hardware, and sometimes less 
effective in virtual machines (e.g. VMWare or Virtualbox) which can 
only emulate the hardware clock.  I am not aware of a more effective 
method than using ntp.


Bob

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Stefano Lattarini]

On Thursday 13 January 2011, Bob Friesenhahn wrote:
> On Wed, 12 Jan 2011, Stefano Lattarini wrote:
> >
> > Those you list above are very good *practical* reasons not to use quagmire
> > (and the very fact that it took me 3 minutes to find that page with google
> > was IMHO already a clear indicator that the project is dead in practice).
> >
> > The "answer" I was speaking about would have been concerned with why I think
> > that the quagmire *design* and *roadmap* are broken (even ignoring its "less
> > than excellent" developement status).
> 
> We should give Quagmire some respect since it is written by the
> original author of Automake, who did quite a good job with Automake. 
> It is wrong to think that lack of popularity means that the software 
> is not in effective use somewhere or is woefully incompete.  Only 
> analysis or real-world testing can show how effective the package is.
>
Sorry, I really, really didn't mean to show a lack of respect to the
project.

Let me restate what my points are:

 [1] Quagmire is not being developed anymore.  And the very fact that
 it took me some time just to find its website with google likely
 implies that it's used by at most very few projects, and taken
 into consideration by at most very very few developers.  These are
 facts IMHO; and even if Quagmire were to already have a superb
 design and the most beatiful codebase in the world, well, that
 wouldn't change these facts (sadly and unfortunately).

 [2] *I think* (as a matter of personal opinion) that planning to
 conflate automake and libtool (and maybe later even autoconf) in
 a single project is an error.  This, and only this, was the
 "design error" I was speaking about (sorry for not having been
 clearer about this before, I really really should have been, so
 I apologize).  On the other hand, I find many of ideas outlined
 in the Quagmire README sensible and nice for what concerns the
 substitution, re-implementation or improvement over purely
 automake features.

 [3] IMVHO Automake have had its success because it was the first
 project to fill an existing gap (offering an higher-level
 make-based "language" with the GNU coding standards in mind),
 and because it was adopted from day 0 by some (and after a
 short time many) GNU packages.  Since Automake's niche is so
 well occupied by Automake itself, any potential competitor
 should strive to obtain a very very high degree of compatibility
 to stand any real chance "against" Automake, bacause people are
 (rightly I'd say) reluctant and wary of abandoning something
 well-knonw, well-tested, and that works well today, for something
 partially incompatible, more unstable, less widespread, and less
 documented (as any younger project invariably is), only because
 this new project is (or should be) cleaner and *could* end up
 working better (even much better) "in the future" (BTW, what
 doesn "future" means? One month? One year? Ten?).
 This is the "roadmap" error of Quagmire I was speaking about: if it
 doesn't start by being 99% automake-compatible from the beginning,
 *I think* it won't take off, no matter how much better and cleaner
 and easy to use it is.  That's a pity, and I don't like things
 being this way, but they are nonetheless.
 [ Note: even if motivated, this is just my opinion, and I don't
   claim it's an objective truth.  But the fact that Quagmire
   failed to take off despite being written by the person who's
   probably the most outstanding Automake contributor shows IMHO
   how that opinion deserves at least some credit ].

> However, it is not Automake.
>
Anyway, I think I could make amend of my careless (and apparently
down-putting, even if this wasn't intended) comments by downloading
and trying out Quagmire.  There's surely a lot to be learnt from it.

> Bob
> 

Regards,
   Stefano

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Bob Friesenhahn]

On Thu, 13 Jan 2011, Steffen Dettmer wrote:


On Thu, Jan 13, 2011 at 3:39 AM, Bob Friesenhahn
 wrote:

While GNU make is a really good 'make' program, I think that 'make' in
general is a flawed concept.


Could you please explain this a bit?


Make depends entirely on file timestamps and iteratively applying 
various 'rules' until it finds an incantation to transform something 
on disk into a target.  File timestamps are not reliable.  When used 
with something smart like Automake the magical incantations are not 
really necessary since they can be explicit from the start. Make does 
not have a way to store what it did other than the resulting output 
files.



What is the problem with timestamps?


Quite often, they don't work, and they are not a reliable indicator of 
change.  Timestamps are useful as an optimization, but a file MD5 is 
much more reliable.



Where else should dependency information be stored? Of course, if


Perhaps in a project database.  The only dependency information which 
is arbitrary (site/build specific) is dependency on files from outside 
of the project.



Do you mean that make is a good multi-purpose tool but not well
suited for C building, especially in automake environments?


Obviously, make usually works, but since automake writes the actual 
rules, automake is capable of writing optimum build rules for a given 
project and should not have to depend so much on implicit dependency 
logic.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Guido Draheim]

> 
> 
> [About GNU make]
> 
>  GNU make is a basic component of the GNU system, actively maintained and
>  developed, well documented, and required by other very important projects
>  (Linux Kernel and Git DVCS, for example).
> 
>  GNU make is very portable, easy to compile, and fully bootstrapping (its
>  build system does not require a pre-existing make installation).
>  
>  GNU make is the default make program on GNU/Linux (okay, we're in
>  full platitude land here, but I value completeness in this issue).
>  
>  GNU make is readily available in the FreeBSD port collection (and it's
>  required by many other ports to compile, see
>
>  for more info).
> 
>  GNU make is available as a NetBSD package, for many architectures
>  and versions; for example:
>   
> 
>   
> 
>   
> 
>  
> 
>  GNU make should also be available as a Solaris precompiled package:
>
>  or as an unofficial pre-compiled binary:
>.
> 
>  In conclusion, it's not unresonable to expect that a user ready to
>  compile a package from sources will also be ready and capable to obtain,
>  compile and install a non-ancient version of GNU make.
> 

Hi folks,

I haven't seen a reference to non-replace makefile-engines.

In reality all the previous makefile-systems (shipping along Solaris,
AIX, etc) have been subverted by GNU-Make - in almost any company that
I came across they had installed "gmake" in parallel with the native
makefile-system. And in the OpenSource world one can see many BSD packages
that have a build dependency on gmake these days.

However there are systems where gmake can not be run as a substitute,
this is atleast true for
* CmSynergy - ships objectmake (omake) integrated with the build system.
* ClearCase - ships clearmake integrated with the build systems.
and there are sure some other build systems that have a frontend to
the user allowing for makefiles as the backend.

These alternate makefile-systems had been modelled after standard
"make" plus some extensions which are usually compatible with GNU
Make. GNU Make has set the standard but GNU Make is NOT FREE to be
used as the backend of these non-GPL makefile-systems. Cough up a
free libgmake and you can assume its standard will be available
everywhere in a short time. But that's not how it is.

cheers, Guido

bug#7833: automake uses two different values for DejaGNU srcdir

[Ian Lance Taylor]

Stefano Lattarini  writes:

> Are you sure about this?  Why then is the attached testcase working
> (for me at least)?

I guess I'm not sure what point you are trying to make with your
example.  A relative srcdir will sometimes work and sometimes fail.  It
seemed to me an absolute srcdir should always work, but Ralf has pointed
out a somewhat obscure case where it too can fail.  Either way, it makes
no sense for automake to use two different values for srcdir.  It should
pick one and stick to it.

Ian

Re: make -j1 fails

[Ralf Wildenhues]

Hello Pippijn,

* Pippijn van Steenhoven wrote on Thu, Jan 13, 2011 at 09:24:42AM CET:
> I am using the FreeBSD make utility to build an automake based
> distribution. When I run make with a -j option (even -j1), it fails.
> Running make with a -j option makes the utility behave differently. The
> issue only arises with built sources (in my case, yacc generated C code),
> since those trigger the inference rule ".c.lo". Perhaps a solution would
> be to also generate explicit rules for built sources.
> 
> I have attached the output of running make with and without -j1.

Thanks for the bug report.  Which version of Automake are you using?
If older than v1.10b-62-g0e411a0, then please update to 1.11.1 and
retry.  It may be possible that we overlooked one case in the
above-mentioned commit, but I don't see where that should be.

If the failure persists, please post short configure.ac and
Makefile.am which expose the problem for you. You can start with
what I show below, and adjust that if it doesn't expose it.

Does the failure happen reliably, i.e., every time, or only sometimes?
You can try in a loop with something like

  for i in 1 2 3 4 5 6 7 8 9 10; do
rm -f axl.[clo]*
make V=1 -j1
  done

Does 'make -B -jN' work for you reliably?
Which exact FreeBSD version are you using?  I'm kind of suspecting a bug
in the make implementation, or we're overlooking something with
one-shell issues.

cat >configure.ac <<\END
AC_INIT([a], [1])
AC_CONFIG_AUX_DIR([autoconf])
AM_INIT_AUTOMAKE([foreign])
AC_CONFIG_FILES([Makefile])
AC_PROG_CC
AC_PROG_LIBTOOL
AC_PROG_YACC
AC_OUTPUT
END

cat >Makefile.am <<\END
lib_LTLIBRARIES = libfoo.la
libfoo_la_SOURCES = src/compiler/phases/parse/axl.y
END

Thanks,
Ralf

> /bin/sh ./libtool  --tag=CC--mode=compile gcc -DHAVE_CONFIG_H  -I. -I.. 
> -I./include/config  -I../include -I./include  -I../include -I./include 
> -D_DEBUG  -DPREFIX='"/usr/local"' -I../`echo axl.c | grep -o  
> 'src/library/\w\+'`/include  -g -O2 -pipe -ggdb3 -pedantic -ansi 
> -fvisibility=hidden -MT axl.lo -MD -MP -MF .deps/axl.Tpo -c -o axl.lo axl.c

> /bin/sh ./libtool  --tag=CC--mode=compile gcc -DHAVE_CONFIG_H  -I. -I.. 
> -I./include/config  -I../include -I./include  -I../include -I./include 
> -D_DEBUG  -DPREFIX='"/usr/local"' -I../`echo axl.c | grep -o  
> 'src/library/\w\+'`/include  -g -O2 -pipe -ggdb3 -pedantic -ansi 
> -fvisibility=hidden -MT axl.lo -MD -MP -MF 
> .deps/src/compiler/phases/parse/axl.Tpo -c -o axl.lo axl.c
> libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I./include/config 
> -I../include -I./include -I../include -I./include -D_DEBUG 
> -DPREFIX=\"/usr/local\" -I..//include -g -O2 -pipe -ggdb3 -pedantic -ansi 
> -fvisibility=hidden -MT axl.lo -MD -MP -MF 
> .deps/src/compiler/phases/parse/axl.Tpo -c axl.c  -fPIC -DPIC -o .libs/axl.o
> axl.c:3626: fatal error: opening dependency file 
> .deps/src/compiler/phases/parse/axl.Tpo: No such file or directory

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Bob Friesenhahn]

On Thu, 13 Jan 2011, Guido Draheim wrote:

* ClearCase - ships clearmake integrated with the build systems.
and there are sure some other build systems that have a frontend to
the user allowing for makefiles as the backend.


FYI, ClearCase's clearmake is actually based on GNU make source code. 
At least it was back when I was using it.


ClearCase is an example of a build system which does produce a build 
'inventory' recording exactly how the target was built.  It 
accomplishes that by studying the I/O requests from the build tools.


Regardless, 'make's use of timestamps based on simple "newer than" 
analsys is not very robust in our real world.  A build system which at 
least records the absolute time stamps (and preferably a checksum) for 
everything used to build a built target is much more capable of 
knowing what needs to be rebuilt if something is changed.  If a source 
file becomes "older" and its content has changed, then anything 
depending on it needs to be rebuilt (which existing make will not do). 
If the source file somehow becomes "older" but its content has not 
changed, then the associated timestamp in the build inventory can be 
updated and nothing needs to be rebuilt.  In this case, the timestamp 
is simply an optimization.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

bug#7670: About make implementations that don't chain implicit rules

[Ralf Wildenhues]

I'm merging bug 7824 with this one.

If we can find a way to safely emit additional dependencies to help
these make implementation, then I am in favor of doing it.  (We can
wrap them in 'if !GNU_MAKE' if need be.)

Thanks,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Bob Friesenhahn wrote on Thu, Jan 13, 2011 at 07:50:08PM CET:
> Regardless, 'make's use of timestamps based on simple "newer than"
> analsys is not very robust in our real world.

That may be true, but this issue is completely orthogonal to whether
Automake caters to non-GNU make implementations or not.

You can implement hash-based dependencies inside GNU make if you like.
Or another make.  Maybe someone has even done so already.  It has little
to do with Automake.


make is a bit flawed for real large projects because it always walks
the whole dependency graph, unlike beta build systems who use a notify
daemon and a database to only walk subgraphs known to be outdated.
I maintain that even that sort of functionality can be mostly hidden
below a Posix make-like interface.  Not relevant for the Automake layer
either, though.

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Stefano Lattarini]

On Thursday 13 January 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Wed, Jan 12, 2011 at 09:58:36PM CET:
> > On Wednesday 12 January 2011, Ralf Wildenhues wrote:
> > > The upsides are obvious.
> > >
> > If I'm not deluding myself, most of the contents of my proposal were
> > aimed at showing why I believe that requiring GNU make is a reasonable
> > and sensible policy, not to show what the advantages of such a policy
> > would be -- quoting myself:
> > 
> >  `` And the gains in terms of maintainability, testability, and
> > possibility of optimization are obvious. ''
> 
> Sure.  You discuss the upsides of the strategy.  What I was asking for
> is a discussion of the downsides of requiring GNU make, and arguing why
> they are not a problem *for our users*.
>
Well, I originally thought that the only real downside of having GNU
make as a requirement was that it should have to be obtained and
installed on the target system (unless it's already there in advance,
like on GNU/Linux) before an automake-using package could be compiled
and installed.

I thought I had dealt with this downside by showing how easy it is to
install GNU make -- by repeating some well-known truths ("GNU make is
a basic component of the GNU system, actively maintained and developed,
well documented, ...",  "GNU make is very portable, easy to compile,
and fully bootstrapping ..."), and by providing some IMO useful links
(e.g. Solaris, NetBSD and FreBSD packages providing GNU make).

Unfortunately, it turned out that ease of installation is not enough
for some users; see the excellent replies of Юрий Пухальский for more
information:
  
  

But after all this is the point of this discussion: probe the users
(well, at least those reading this mailing list) for problems and
contrary opinions I might not have thought of (or I might not have
even be *able* to think of!) by myself.

> *That* is what is needed to convince those who do consider requiring
> GNU make outright a regression.
>
But then, first I have to hear *why* they consider requiring GNU make
a regression (and who knows, I might even end up agreeing with them
at last, in some way or another).

> > > I was planing to introduce optional GNU make-specific code, and allowing
> > > to let the user specify "my project requires GNU make anyway", which
> > > would enable Automake to emit better code.
>
BTW, this is great news!  Sorry for not having told so before.

> > > Arguably more complex than requiring GNU make outright, but it
> > > wouldn't throw away all the make portability work that exists
> > > in Automake.
> > >
> > Hmm... so you're telling me that "This code should be kept because it
> > had been difficult to write it" is an acceptable rationale? ;->
> 
> No.  I'm saying that that portability is a *feature*.  You want to
> remove that feature.  You need to bring solid reasons why the feature
> is worthless,
>
I've never said it's worthless.  Please understand that my proposal is
all about "measuring" cost/benefit ratio of that feature: ct for the
automake developers of having to deal with make portability issues vs.
benefit for the users (and how much users, and for what reasons) of
having automake-generated Makefiles just work with generic vendor make
implementations).  The feature should be removed if that ratio is
deemed too high in the end.

> used by nobody, and nobody is interested in it.
>
IMHO the real questions are: How many people are interested in the
feature?  How much are they interested?  And especially, *why* are
they interested?

I found the replies of Юрий I linked above especially useful because
they answer the latter question in a clear, motivated way (at least
for what concerns him and the situations he have to deat with).

> And, of course, why we should abandon the GNU Coding Standards mandate
> of only requiring a small fixed set of Posix tools.
> 
> This thread has already shown that to not be true, judging from a couple
> of the other answers.
>
Currently, the only motivated reason against my proposal I've read is
that of Юрий (which might be enough in the end, if we can conclude that
a high-enough number of users have requirements similar to his ones).

And to clarify, no, I don't think the "this is a dangerous slope" kind
of motivation is a sensible one; see:
  
for more info.

> > (BTW, that portability-related work has already and definitely served
> > the purpose of making automake usage more widespread, so I don't think
> > the efforts that went into it would ever be "wasted", even if the code
> > they produced is going to be removed).
> 
> OK, so you're acknowledging that the feature serves a useful purpose.
> Why should we then remove it?
>
We should remove it if (and only if) the costs/benefits ratio is deemed
too high (and I'm not 

bug#7824: 1.11 doesn't add sources with nonstandard suffixes when making a binary

[ralf.wildenhues]

* Юрий Пухальский wrote on Wed, Jan 12, 2011 at 03:21:11PM CET:
> -
> SUFFIXES = .pc
> 
> .pc.lo:
>   cp $(srcdir)/$*.pc $(builddir)/$*.c
>   $(LTCOMPILE) -c $(builddir)/$*.c
>   rm -f $(builddir)/$*.c
> 
> noinst_PROGRAMS = 1 2
> 
> 1_SOURCES = 1.pc

Program sources are not compiled into .lo files, but into .$(OBJEXT)
files.  So you would need a .pc.$(OBJEXT) rule.  You can use $(COMPILE)
instead of $(LTCOMPILE).

> db_src = 1.pc
> 2_SOURCES = dummy.c
> 2_LDADD = $(db_src:.pc=.lo)
> -
> 1 is not built:
> 
> devfe:~/tmp/test>make
> /bin/sh ./libtool --tag=CC   --mode=link gcc  -g -O2   -o 1
> libtool: link: gcc -g -O2 -o 1
> gcc: no input files
> make: *** [1] Error 1

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Guido Draheim]

Am 12.01.2011 19:01, schrieb Stefano Lattarini:
> 
>  GNU make is a basic component of the GNU system, actively maintained and
>  GNU make is very portable, easy to compile, and fully bootstrapping
>  GNU make is the default make program on GNU/Linux (okay, we're in
>  GNU make is readily available in the FreeBSD port collection (and it's
>  GNU make is available as a NetBSD package, for many architectures
>  GNU make should also be available as a Solaris precompiled package:
> 
>  In conclusion, it's not unresonable to expect that a user ready to
>  compile a package from sources will also be ready and capable to obtain,
>  compile and install a non-ancient version of GNU make.
> 
>  I don't think that requiring GNU make (not a specific version, just
>  a less-than-fifteen-years old one) is gonna be an harsh or unreasonable
>  requirement.  And the gains in terms of maintainability, testability,
>  and possibility of optimization are obvous.
> 
> 

Sure GNU Make has subverted most other makefile-systems but there are places
where it CAN NOT be used as a drop-in replacement. The paramount examples are
in larger build-systems with an integrated makefile-system - some IDEs for
embedded targets may have it but I do more think of

* CmSynergy - ships with omake (object make) integrated with version control
* ClearCase - ships with clearmake integrated with its version control

That's more than a third of the version control systems used in the
corporate world. So unless automake would officially dump the support
for the corporate world (which some GNU roughnecks might argue for) it
would be good to NOT allow GNUMake features all around.

Note however that both the systems above do have some extensions over
standard make that are always compatible with GNU Make - simply because
GNU Make has set the de facto standard. So it would be very fair to
assume a specific set of features on top of standard make that are
prevalent in modern makefile-system all around.

But GNU Make on its own has a very large feature-set that is strong enough to
be a build-system on its own. You don't need much of autoconf/automake anymore!!
And this is not just theory because I have already created such a system for
a company - you can run configure-tests with make rules in GNUmake. That's 
because
you can include make-generated makefile-snippets at runtime which is impossible
for most other makefile systems. (originally that feature was meant to allow
"make deps" generating *.dep's and compile the final target in one make-call. 
Here
you can use it to print defines to a makefile-snippet and load it right away.)

cheers, Guido

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Bob Friesenhahn]

On Thu, 13 Jan 2011, Ralf Wildenhues wrote:


* Bob Friesenhahn wrote on Thu, Jan 13, 2011 at 07:50:08PM CET:

Regardless, 'make's use of timestamps based on simple "newer than"
analsys is not very robust in our real world.


That may be true, but this issue is completely orthogonal to whether
Automake caters to non-GNU make implementations or not.


It does have to do with whether GNU make adds sufficient benefit to 
outweigh portability, or if something other than 'make' (and which is 
portable) should be used entirely.



You can implement hash-based dependencies inside GNU make if you like.
Or another make.  Maybe someone has even done so already.  It has little
to do with Automake.


I have never heard of such a thing implemented in make.  It would 
surely bloat the makefile by more megabytes or be excruciatingly slow.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Guido Draheim wrote on Thu, Jan 13, 2011 at 08:25:05PM CET:
> But GNU Make on its own has a very large feature-set that is strong enough to
> be a build-system on its own. You don't need much of autoconf/automake 
> anymore!!

Exactly.  quagmire is one (other) implementation of that.  I don't
understand why people don't just use that if they want to.

(The fact that the project appears stale is irrelevant.  This is free
software, you have the freedom to revive even the ugliest of zombies
... ;-)

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* Bob Friesenhahn wrote on Thu, Jan 13, 2011 at 08:32:01PM CET:
> On Thu, 13 Jan 2011, Ralf Wildenhues wrote:
> >You can implement hash-based dependencies inside GNU make if you like.
> >Or another make.  Maybe someone has even done so already.  It has little
> >to do with Automake.
> 
> I have never heard of such a thing implemented in make.  It would
> surely bloat the makefile by more megabytes or be excruciatingly
> slow.

That's not what I meant.  You can hack the source code of GNU make
with the goal to (optionally) replace timestamp-based dependencies
with file-hash-based depdendencies.  No changes at all in any user
makefiles.  It's not a large change even.  IIRC somebody has done
this already, it just hasn't been merged upstream.

BTW, Nathanael's propositions in this thread go in the same direction:
http://gcc.gnu.org/ml/gcc-patches/2010-11/msg01504.html

Cheers,
Ralf

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Stefano Lattarini]

On Thursday 13 January 2011, Steffen Dettmer wrote:
> On Wed, Jan 12, 2011 at 10:36 PM,  wrote:
> >  - I think that keeping configuration and build steps separated is
> >   a very good idea.
> 
> Do you mean this is a good idea in the context of todays systems
> - or -
> Do you mean this is good idea in general and could be a design
> criteria for future build environments?
>
The second one.  But TBH I don't have any objective justification
for this, only "gut feelings".  So I should better have said:

 ``I truly dislike the idea of not keeping configuration and build
   steps separated.''

Maybe I'd just like a system that *allows* me to keep configuration
and build steps clerarly distinct if I want to.  Yes, that would
be enough for me I guess.

> I think I agree to the first (mostly because I assume if the
> autotools developers and experts separate those steps, they do it
> for a good reason), but I don't understand my this could be a
> requirement in future systems.
> 
> Wouldn't it be great to type "make" which automatically knows by
> depedencies that some configuration rules have to be executed
> (i.e. to determine facts about the environment if they are not
> available in form of small .h files or alike)?
>
Yes, but then, this could be implemented by having the build system
call the configuration system properly, no?  More or less like is
done by automake-generated rebuild rules, just "on steroid" I guess.

> If, for example, Makefiles would have rules to check for the
> libraries as soon as needed etc, wouldn't this be good?
> Tests that are not needed for the configuration to be built
> would not even be executed (saving time).
>
What do you mean exactly by this?  I might appear dumb, but
I'm having some difficulties in following you here.

> What important points did I miss in my consideration?
> 
> oki,
> 
> Steffen
> 

Regards,
  Stefano

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Guido Draheim]

Am 13.01.2011 19:50, schrieb Bob Friesenhahn:
> On Thu, 13 Jan 2011, Guido Draheim wrote:
>> * ClearCase - ships clearmake integrated with the build systems.
>> and there are sure some other build systems that have a frontend to
>> the user allowing for makefiles as the backend.
> 
> FYI, ClearCase's clearmake is actually based on GNU make source code. At
> least it was back when I was using it.

Well, I had years to study the differences and there are many. What you
do think of is "-C gnu" which disables any GNU-incompatible features, it
defines some GNU-Make compatible default-rules and understands some more
GNU-Make-like options. But I would think clearmake is GNU-make based.
Sure however I do not know how the "gnu emulation" is implemented :-/

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Paul Smith]

On Thu, 2011-01-13 at 12:50 -0600, Bob Friesenhahn wrote:
> FYI, ClearCase's clearmake is actually based on GNU make source code. 
> At least it was back when I was using it.

Not that it matters but this is not _quite_ true... otherwise clearmake
would fall under the GPL.

What clearmake used to do was ship with a patched version of GNU make
that added a -E flag, when that flag was given then GNU make would read
the makefiles but instead of actually building anything it would output
an "elaborated" makefile with much of the extra functionality of GNU
make already expanded; for example ifdef etc. would be processed
already, etc.  Clearmake in GNU make mode would exec that patched
version of GNU make with the -E flag, then read the output to get a more
vanilla makefile.  This gave some limited support for GNU make syntax.

However, that model was ditched back in the 1990's sometime.  The last
time I used ClearCase (over 4 years ago now), clearmake simply
implemented some set of GNU make features natively without using any GNU
make preprocessor.  However, the set of features implemented was pretty
weak; something like GNU make 3.76 or so; not much beyond that.  And,
even some 3.76 features there were missing, like auto re-exec
capabilities etc.

bug#7824: 1.11 doesn't add sources with nonstandard suffixes when making a binary

[Юрий Пухальский]

Ah, sorry. That's so easy, and now i recollect you must have said this
before, but i've lost it...

2011/1/13 ralf.wildenhues :
> * Юрий Пухальский wrote on Wed, Jan 12, 2011 at 03:21:11PM CET:
>> -
>> SUFFIXES = .pc
>>
>> .pc.lo:
>>       cp $(srcdir)/$*.pc $(builddir)/$*.c
>>       $(LTCOMPILE) -c $(builddir)/$*.c
>>       rm -f $(builddir)/$*.c
>>
>> noinst_PROGRAMS = 1 2
>>
>> 1_SOURCES = 1.pc
>
> Program sources are not compiled into .lo files, but into .$(OBJEXT)
> files.  So you would need a .pc.$(OBJEXT) rule.  You can use $(COMPILE)
> instead of $(LTCOMPILE).
>
>> db_src = 1.pc
>> 2_SOURCES = dummy.c
>> 2_LDADD = $(db_src:.pc=.lo)
>> -
>> 1 is not built:
>> 
>> devfe:~/tmp/test>make
>> /bin/sh ./libtool --tag=CC   --mode=link gcc  -g -O2   -o 1
>> libtool: link: gcc -g -O2 -o 1
>> gcc: no input files
>> make: *** [1] Error 1
>



-- 
«The good thing about standards is there are so many to choose from.»

bug#7833: automake uses two different values for DejaGNU srcdir

[Stefano Lattarini]

On Thursday 13 January 2011, Ian Lance wrote:
> Stefano Lattarini  writes:
> 
> > Are you sure about this?  Why then is the attached testcase working
> > (for me at least)?
> 
> I guess I'm not sure what point you are trying to make with your
> example.  A relative srcdir will sometimes work and sometimes fail.  It
> seemed to me an absolute srcdir should always work, but Ralf has pointed
> out a somewhat obscure case where it too can fail.  Either way, it makes
> no sense for automake to use two different values for srcdir.  It should
> pick one and stick to it.
> 
Yes, but my testcase *should* show (at least in my understanding)
that in the dejagnu tests both the TCL-level '$srcdir' variable and
the environment variable 'srcdir' contain an absolute path, even
when the Makefile variable $(srcdir) contains a relative path (in
this case, `.').

Regards,
  Stefano

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Roger Leigh]

On Wed, Jan 12, 2011 at 07:01:47PM +0100, Stefano Lattarini wrote:
> To be honest, I'm starting to agree with Ralf more and more on these
> issues; i.e., just " ... require a decent make ;-)".
> 
> And more than this -- brace yourself -- I'm starting to think that
> automake should *really* start supporting *only* GNU make (at least
> from version 3.75 or so).
[...]

I certainly agree that there are several advantages to requiring GNU
make.  But I don't think you touched on the most common practical
problem with the current situation.

In theory, automake is portable and targets any make providing the basic
POSIX/SUS-specified functionality.  In reality, many (most?) automake-
using projects fail to build with non-GNU make.  This is /not/ because
of any deficiencies with automake, it's because the Makefile.ams are
only ever tested with GNU make, and so any nonportable constructs are
not picked up on.  Without regular testing using a set of different
make variants, we end up supporting GNU make by default--because that's
what the overwhelming majority of developers and users are using.

In a number of projects I maintain the build infrastructure for, we
already mandate GNU make.  This is just reflecting a simple reality: we
don't have the resources to continually test for and fix portability
issues in the Makefiles.  And if we do take the time to do this, it's
rapidly broken again because testing with GNU make doesn't pick up any
regressions.  We also don't have /any/ feedback from non-GNU make users
telling us about portability issues, so GNU make becomes a /de facto/
requirement in any case.  Supporting non-GNU makes just isn't worth the
effort--we gain nothing from it.

Note I'm not suggesting that automake immediately stop being portable
to other makes.  But I don't believe it's serving a useful purpose for
the vast majority of automake users.  I would find it useful to make
use of more advanced GNU make-specific features on occasion, without
automake complaining!


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?   http://gutenprint.sourceforge.net/
   `-GPG Public Key: 0x25BFB848   Please GPG sign your mail.


signature.asc
Description: Digital signature

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Guido Draheim]

Am 13.01.2011 15:44, schrieb Stefano Lattarini:
> 
>  [3] IMVHO Automake have had its success because it was the first
>  project to fill an existing gap (offering an higher-level
>  make-based "language" with the GNU coding standards in mind),
>
>  doesn "future" means? One month? One year? Ten?).
>  This is the "roadmap" error of Quagmire I was speaking about: if it
>  doesn't start by being 99% automake-compatible from the beginning,
>  *I think* it won't take off, no matter how much better and cleaner
>  and easy to use it is.  That's a pity, and I don't like things

By the way, I would find it more interesting to see some
automake-replacement that allows GNU-make extensions and
compiles them down to standard make as far as possible...

On the other hand some automake-restrictions are really not needed
anymore. (variables with "." and "/" would be nice - along with
http://www.gnu.org/software/automake/manual/make/Computed-Names.html
it would easily allow a scheme to define generated compile-flags).

cheers, Guido

Re: High-Precision NFS Timestamps

[Harlan Stenn]

Eric wrote:
> As I said, we do use NTP to keep our machines in synchronization, every 
> hour. ...

If you are doing something other than running ntpd as a daemon, you are
likely not keeping your clocks sync's well enough to keep NFS timestamps
happy.

H

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Stefano Lattarini]

Hello Roger.

I take up on a tangential issue here ...

On Thursday 13 January 2011, Roger Leigh wrote:
>
> [MEGA-CUT]
>
> I would find it useful to make use of more advanced GNU
> make-specific features on occasion, without automake complaining!
>
For this, I think having a new `portability-make' warning category
you could turn off without having to turn off also all the other
portability warnings could help.  There's an open PR about this:
 

Regards,
   Stefano

bug#7833: automake uses two different values for DejaGNU srcdir

[Stefano Lattarini]

On Thursday 13 January 2011, Ian Lance wrote:
> Stefano Lattarini  writes:
> 
> > On Thursday 13 January 2011, Ian Lance wrote:
> >> Stefano Lattarini  writes:
> >> 
> >> > Are you sure about this?  Why then is the attached testcase working
> >> > (for me at least)?
> >> 
> >> I guess I'm not sure what point you are trying to make with your
> >> example.  A relative srcdir will sometimes work and sometimes fail.  It
> >> seemed to me an absolute srcdir should always work, but Ralf has pointed
> >> out a somewhat obscure case where it too can fail.  Either way, it makes
> >> no sense for automake to use two different values for srcdir.  It should
> >> pick one and stick to it.
> >> 
> > Yes, but my testcase *should* show (at least in my understanding)
> > that in the dejagnu tests both the TCL-level '$srcdir' variable and
> > the environment variable 'srcdir' contain an absolute path, even
> > when the Makefile variable $(srcdir) contains a relative path (in
> > this case, `.').
> 
> Take a look at the site.exp file and see what value is in there for
> srcdir: relative or absolute?
>
Relative (as I expected):

  $ cat dejagnu-srcdir-is-absolute.dir/site.exp 
  ## these variables are automatically generated by make ##
  # Do not edit here.  If you wish to override these values
  # edit the last section
  set srcdir .
  set objdir /home/stefano/src/automake/tests/dejagnu-srcdir-is-absolute.dir
  ## All variables above are generated by configure. Do Not Edit ##

But then, why the testcase suceeds?  I would expect it to fail!

Thanks,
  Stefano

bug#7833: automake uses two different values for DejaGNU srcdir

[Ian Lance Taylor]

Stefano Lattarini  writes:

> On Thursday 13 January 2011, Ian Lance wrote:
>> Stefano Lattarini  writes:
>> 
>> > Are you sure about this?  Why then is the attached testcase working
>> > (for me at least)?
>> 
>> I guess I'm not sure what point you are trying to make with your
>> example.  A relative srcdir will sometimes work and sometimes fail.  It
>> seemed to me an absolute srcdir should always work, but Ralf has pointed
>> out a somewhat obscure case where it too can fail.  Either way, it makes
>> no sense for automake to use two different values for srcdir.  It should
>> pick one and stick to it.
>> 
> Yes, but my testcase *should* show (at least in my understanding)
> that in the dejagnu tests both the TCL-level '$srcdir' variable and
> the environment variable 'srcdir' contain an absolute path, even
> when the Makefile variable $(srcdir) contains a relative path (in
> this case, `.').

Take a look at the site.exp file and see what value is in there for
srcdir: relative or absolute?

Ian

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Paul Smith]

On Thu, 2011-01-13 at 20:38 +0100, Ralf Wildenhues wrote:
> * Bob Friesenhahn wrote on Thu, Jan 13, 2011 at 08:32:01PM CET:
> > On Thu, 13 Jan 2011, Ralf Wildenhues wrote:
> > >You can implement hash-based dependencies inside GNU make if you like.
> > >Or another make.  Maybe someone has even done so already.  It has little
> > >to do with Automake.
> > 
> > I have never heard of such a thing implemented in make.  It would
> > surely bloat the makefile by more megabytes or be excruciatingly
> > slow.
> 
> That's not what I meant.  You can hack the source code of GNU make
> with the goal to (optionally) replace timestamp-based dependencies
> with file-hash-based depdendencies.  No changes at all in any user
> makefiles.  It's not a large change even.  IIRC somebody has done
> this already, it just hasn't been merged upstream.

It's not QUITE so simple.  The advantage to using timestamp-based
dependencies is that the filesystem is your database of
"out-of-dateness", and the kernel ensures that the behavior of that
database is well-defined.

If you want to use anything more sophisticated to determine
"out-of-dateness", then make has to keep its own database to remember
the previous state, so it can be compared.  Granted this is not rocket
science but there are some tricky bits to it: it needs to be pretty
portable to the various architectures make supports; it needs to be
well-behaved in the case of a missing database, and in the case of
multiple instances of make running in the same directory at the same
time; what happens if the database is corrupted or somehow contains the
wrong information; etc.

Making this robust enough to trust it with your builds requires some
thought and care.

> BTW, Nathanael's propositions in this thread go in the same direction:
> http://gcc.gnu.org/ml/gcc-patches/2010-11/msg01504.html

There was a set of work done to introduce "user-defined out-of-date
functions" into GNU make, a few years ago.  That didn't end up getting
merged because it seemed too complex to me, both from a make developer
and make user standpoint.

I agree with Nathanael, although I take serious issue with his statement
"the GNU make people were completely uninterested, since they didn't get
why it was a good idea"--that is simply not true.

What is true is that Nathanael posted a 10,000ft-level description of
what he wanted, not completely thought-out, and when I replied to his
message with some discussion of the idea and potential problems with it,
he never followed up.

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[NightStrike]

On 1/13/11, Ralf Wildenhues  wrote:
> make is a bit flawed for real large projects because it always walks
> the whole dependency graph, unlike beta build systems who use a notify
> daemon and a database to only walk subgraphs known to be outdated.

How big is real large?  GCC uses make, for instance, and it's the
biggest public project that I personally know about.  I worked on a
250ksloc project, but that was in Java and just used eclipse native
stuff (not even ant).  I worked on a 1m sloc project in ada... but for
that project, every build was a full complete 100% fresh build for
reasons outside of this thread.

At what magnitude does make break down, do you think?  And how/where
does it become flawed?

In retrospect, even when dealing with GCC, I never do a partial
rebuild.  Every patch for me no matter how simple involves a complete
fresh build (and because of the nature of what I do with GCC, that
also involves a complete fresh build of binutils and of our libc).  So
I really don't have any good context from which to extrapolate your
meaning.

Re: [CRAZY PROPOSAL] Automake should support only GNU make

[Ralf Wildenhues]

* NightStrike wrote on Fri, Jan 14, 2011 at 05:12:46AM CET:
> On 1/13/11, Ralf Wildenhues wrote:
> > make is a bit flawed for real large projects because it always walks
> > the whole dependency graph, unlike beta build systems who use a notify
> > daemon and a database to only walk subgraphs known to be outdated.
> 
> How big is real large?  GCC uses make, for instance, and it's the
> biggest public project that I personally know about.

Running 'make' in an up to date GCC build tree and hot file cache takes
on the order of 10s for me.  That may be tolerable, but only because any
changes causing recompiles lead to much longer build times, and because
the GCC build system provides lots of special-case make targets to only
rebuild parts of the tree (e.g., stage3-bubble, all-target-libgfortran)
which finish more quickly.  The GCC build system is fairly special
though, in that it is pretty complex and (ab?)uses recursive makefiles
also to allow ignoring big chunks of the dependency tree in some cases.

(That, by the way, is what nobody ever tells you when they point you to
the "Recursive Make Considered Harmful" paper: that walking the full
dependency tree is more expensive than walking a factorized one.  Sure,
you need to keep the toplevel deps up to date manually then.)

> At what magnitude does make break down, do you think?  And how/where
> does it become flawed?

A rough estimate is 100k files, even after quadratic scaling issues in
make are fixed.  The tup project (which is a prototype beta build
system) claims some lower numbers even[1], but their graphs indicate
that they've also measured some nonlinear effects in make, which should
all be fixable.

I'd guess that webkit would benefit slightly already though.

> In retrospect, even when dealing with GCC, I never do a partial
> rebuild.

For full rebuilds, this particular issue is irrelevant.  There, the
whole dependency tree needs to be walked no matter what build system.

Cheers,
Ralf

[1] http://gittup.org/tup/make_vs_tup.html