Re: [HACKERS] DTrace build dependency rules

2015-10-15 Thread Robert Haas
On Tue, Oct 13, 2015 at 3:47 PM, Robert Haas  wrote:
> On Tue, Aug 18, 2015 at 12:08 PM, Alvaro Herrera
>  wrote:
>> Robert Haas wrote:
>>> On Sat, Aug 15, 2015 at 6:45 PM, Mark Johnston  wrote:
>>
>>> > The bug is in src/backend/Makefile. probes.o, the dtrace(1)-generated
>>> > object file, depends on the objfiles.txt for each of the backend
>>> > subdirs. These files depend in turn on the object files themselves; if
>>> > objfiles.txt is out of date with respect to one of its object files, the
>>> > mtime of objfiles.txt is updated with "touch" (see backend/common.mk).
>>> > The problem is that dtrace -G, which runs at the end of the build,
>>> > modifies a number of object files (it overwrites their probe sites with
>>> > NOPs), thus making their corresponding objfiles.txt out of date. Then,
>>> > when "make install" traverses the backend subdirs, it updates
>>> > objfiles.txt, which causes probes.o to be rebuilt, resulting in an error
>>> > from dtrace(1).
>>>
>>> Gosh, that's pretty ugly.  I would have thought it would be a real
>>> no-no to update the .o file once it got generated.  If nothing else, a
>>> modification to the .c file concurrent with a make invocation might
>>> lead to the .o not getting rebuilt the next time make is run.
>>
>> I had the same thought, and wondered for a bit whether we should instead
>> have the compilation rules produce some intermediate file (prior to
>> dtrace fumbling), then emit the .o from dtrace -G.  OTOH this might be
>> more trouble than is worth for a feature that doesn't see a lot of use.
>
> Given the lack of further interest from the PostgreSQL community,
> that's my guess.  I've pushed this patch to master; let's see if we
> get any complaints.  If it makes life better for FreeBSD without
> making life worse for anyone else, I suppose we might as well do it
> until something better comes along.

Per http://www.postgresql.org/message-id/24541.1444928...@sss.pgh.pa.us
I have had to revert this patch, because it's breaking parallel builds
even for non-dtrace users.

Sorry for the inconvenience.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] DTrace build dependency rules

2015-10-13 Thread Robert Haas
On Tue, Aug 18, 2015 at 12:08 PM, Alvaro Herrera
 wrote:
> Robert Haas wrote:
>> On Sat, Aug 15, 2015 at 6:45 PM, Mark Johnston  wrote:
>
>> > The bug is in src/backend/Makefile. probes.o, the dtrace(1)-generated
>> > object file, depends on the objfiles.txt for each of the backend
>> > subdirs. These files depend in turn on the object files themselves; if
>> > objfiles.txt is out of date with respect to one of its object files, the
>> > mtime of objfiles.txt is updated with "touch" (see backend/common.mk).
>> > The problem is that dtrace -G, which runs at the end of the build,
>> > modifies a number of object files (it overwrites their probe sites with
>> > NOPs), thus making their corresponding objfiles.txt out of date. Then,
>> > when "make install" traverses the backend subdirs, it updates
>> > objfiles.txt, which causes probes.o to be rebuilt, resulting in an error
>> > from dtrace(1).
>>
>> Gosh, that's pretty ugly.  I would have thought it would be a real
>> no-no to update the .o file once it got generated.  If nothing else, a
>> modification to the .c file concurrent with a make invocation might
>> lead to the .o not getting rebuilt the next time make is run.
>
> I had the same thought, and wondered for a bit whether we should instead
> have the compilation rules produce some intermediate file (prior to
> dtrace fumbling), then emit the .o from dtrace -G.  OTOH this might be
> more trouble than is worth for a feature that doesn't see a lot of use.

Given the lack of further interest from the PostgreSQL community,
that's my guess.  I've pushed this patch to master; let's see if we
get any complaints.  If it makes life better for FreeBSD without
making life worse for anyone else, I suppose we might as well do it
until something better comes along.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] DTrace build dependency rules

2015-08-18 Thread Alvaro Herrera
Robert Haas wrote:
> On Sat, Aug 15, 2015 at 6:45 PM, Mark Johnston  wrote:

> > The bug is in src/backend/Makefile. probes.o, the dtrace(1)-generated
> > object file, depends on the objfiles.txt for each of the backend
> > subdirs. These files depend in turn on the object files themselves; if
> > objfiles.txt is out of date with respect to one of its object files, the
> > mtime of objfiles.txt is updated with "touch" (see backend/common.mk).
> > The problem is that dtrace -G, which runs at the end of the build,
> > modifies a number of object files (it overwrites their probe sites with
> > NOPs), thus making their corresponding objfiles.txt out of date. Then,
> > when "make install" traverses the backend subdirs, it updates
> > objfiles.txt, which causes probes.o to be rebuilt, resulting in an error
> > from dtrace(1).
> 
> Gosh, that's pretty ugly.  I would have thought it would be a real
> no-no to update the .o file once it got generated.  If nothing else, a
> modification to the .c file concurrent with a make invocation might
> lead to the .o not getting rebuilt the next time make is run.

I had the same thought, and wondered for a bit whether we should instead
have the compilation rules produce some intermediate file (prior to
dtrace fumbling), then emit the .o from dtrace -G.  OTOH this might be
more trouble than is worth for a feature that doesn't see a lot of use.

> > The attached patch fixes the problem by having probes.o depend on object
> > files directly, rather than on objfiles.txt. I've tested it with
> > PostgreSQL 9.0-9.4 on FreeBSD CURRENT.
> 
> I don't see a particular reason not to make this change, though.

I wanted to test it on Linux yesterday but didn't get any further than
installing a couple of packages.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] DTrace build dependency rules

2015-08-18 Thread Robert Haas
On Sat, Aug 15, 2015 at 6:45 PM, Mark Johnston  wrote:
> There seems to be a bug in the make rules when DTrace is enabled. It
> causes dtrace -G to be invoked twice when building PostgreSQL as a
> FreeBSD port: once during the build itself, and once during
> installation. For a long time this has been worked around on FreeBSD
> with a change to libdtrace itself, but this workaround is proving
> problematic and I'd like to fix the problem properly. I'm not sure
> whether the problem has been observed on other operating systems that
> support DTrace.
>
> The bug is in src/backend/Makefile. probes.o, the dtrace(1)-generated
> object file, depends on the objfiles.txt for each of the backend
> subdirs. These files depend in turn on the object files themselves; if
> objfiles.txt is out of date with respect to one of its object files, the
> mtime of objfiles.txt is updated with "touch" (see backend/common.mk).
> The problem is that dtrace -G, which runs at the end of the build,
> modifies a number of object files (it overwrites their probe sites with
> NOPs), thus making their corresponding objfiles.txt out of date. Then,
> when "make install" traverses the backend subdirs, it updates
> objfiles.txt, which causes probes.o to be rebuilt, resulting in an error
> from dtrace(1).

Gosh, that's pretty ugly.  I would have thought it would be a real
no-no to update the .o file once it got generated.  If nothing else, a
modification to the .c file concurrent with a make invocation might
lead to the .o not getting rebuilt the next time make is run.

> The attached patch fixes the problem by having probes.o depend on object
> files directly, rather than on objfiles.txt. I've tested it with
> PostgreSQL 9.0-9.4 on FreeBSD CURRENT.

I don't see a particular reason not to make this change, though.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] DTrace build dependency rules

2015-08-17 Thread Mark Johnston
Hi,

There seems to be a bug in the make rules when DTrace is enabled. It
causes dtrace -G to be invoked twice when building PostgreSQL as a
FreeBSD port: once during the build itself, and once during
installation. For a long time this has been worked around on FreeBSD
with a change to libdtrace itself, but this workaround is proving
problematic and I'd like to fix the problem properly. I'm not sure
whether the problem has been observed on other operating systems that
support DTrace.

The bug is in src/backend/Makefile. probes.o, the dtrace(1)-generated
object file, depends on the objfiles.txt for each of the backend
subdirs. These files depend in turn on the object files themselves; if
objfiles.txt is out of date with respect to one of its object files, the
mtime of objfiles.txt is updated with "touch" (see backend/common.mk).
The problem is that dtrace -G, which runs at the end of the build,
modifies a number of object files (it overwrites their probe sites with
NOPs), thus making their corresponding objfiles.txt out of date. Then,
when "make install" traverses the backend subdirs, it updates
objfiles.txt, which causes probes.o to be rebuilt, resulting in an error
from dtrace(1).

The attached patch fixes the problem by having probes.o depend on object
files directly, rather than on objfiles.txt. I've tested it with
PostgreSQL 9.0-9.4 on FreeBSD CURRENT.

Thanks!
-Mark
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 98b978f..b1e3969 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -180,8 +180,8 @@ $(top_builddir)/src/include/utils/probes.h: utils/probes.h
 	$(LN_S) "../../../$(subdir)/utils/probes.h" .
 
 
-utils/probes.o: utils/probes.d $(SUBDIROBJS)
-	$(DTRACE) $(DTRACEFLAGS) -C -G -s $(call expand_subsys,$^) -o $@
+utils/probes.o: utils/probes.d $(call expand_subsys,$(SUBDIROBJS))
+	$(DTRACE) $(DTRACEFLAGS) -C -G -s $< $(call expand_subsys,$(SUBDIROBJS)) -o $@
 
 
 ##

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers