Re: [HACKERS] Buildfarm issues on specific machines
Jim C. Nasby [EMAIL PROTECTED] writes: And 7.3 is also failing, with a different error: ccache gcc -traditional-cpp -g -O2 -fno-strict-aliasing -g -Wall -Wmissing-prototypes -Wmissing-declarations -I../../../../src/include -I/opt/local/include -c -o printtup.o printtup.c In file included from /usr/include/machine/param.h:30, from /usr/include/sys/socket.h:67, from ../../../../src/include/libpq/pqcomm.h:28, from ../../../../src/include/libpq/libpq-be.h:24, from ../../../../src/include/libpq/libpq.h:21, from printtup.c:20: /usr/include/ppc/param.h:98: macro btodb requires 2 arguments, but only 1 given /usr/include/ppc/param.h:100: macro dbtob requires 2 arguments, but only 1 given This has been seen before, eg http://archives.postgresql.org/pgsql-hackers/2003-10/msg00579.php http://archives.postgresql.org/pgsql-hackers/2003-11/msg00163.php At the time we felt this was Apple's fault and didn't want to back-patch any changes: http://archives.postgresql.org/pgsql-hackers/2003-11/msg00170.php I'm not sure I want to invest a lot of work in extracting and back-patching the necessary changes to make pre-7.4 PG build on Apple's recent toolchains. I think that changing -traditional-cpp to -no-cpp-precomp in template/darwin would get you past the above-quoted failure, but there are probably more problems waiting after that. A quick look through the 7.4 CVS history suggests there were quite a few fixes for Darwin, many of them done in a way with possible side-effects on other platforms, so just back-patching would probably be out of the question anyway. In short, OS X 10.2 wasn't a supported platform when 7.2/7.3 came out, and I don't want to retroactively try to make it so. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
Tom Lane wrote: This is a considerably bigger issue for the buildfarm than it would be for ordinary users of our distribution, since in the distro it's only the contrib modules that you actually need to run through your local flex. Couldn't we just run the distprep actions (flex, bison) through contrib as well? That wouldn't hurt anyone, I think. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
Peter Eisentraut [EMAIL PROTECTED] writes: Couldn't we just run the distprep actions (flex, bison) through contrib as well? That wouldn't hurt anyone, I think. No objection here (though of course it doesn't affect the buildfarm issue). regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
On Wed, Jul 20, 2005 at 10:32:00AM -0400, Tom Lane wrote: In short, OS X 10.2 wasn't a supported platform when 7.2/7.3 came out, and I don't want to retroactively try to make it so. All I needed to hear. I'll pull those from cuckoo's config. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Buildfarm issues on specific machines
On Sat, Jul 16, 2005 at 11:17:29PM -0400, Tom Lane wrote: cuckoo [7.3, 7.2]: --enable-nls without OS support This looks like pilot error; but the later branches don't fail on this machine, so did we change something in this area? Should I just stop using nls on 7.2 and 7.3 or does someone want to poke around with this further? Also, is -hackers the best place to ask about buildfarm issues, or should I post questions elsewhere? -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
Jim C. Nasby [EMAIL PROTECTED] writes: On Sat, Jul 16, 2005 at 11:17:29PM -0400, Tom Lane wrote: cuckoo [7.3, 7.2]: --enable-nls without OS support This looks like pilot error; but the later branches don't fail on this machine, so did we change something in this area? Should I just stop using nls on 7.2 and 7.3 or does someone want to poke around with this further? I don't intend to look at it; but perhaps someone who cares more about the NLS code will ... Also, is -hackers the best place to ask about buildfarm issues, or should I post questions elsewhere? hackers is definitely the place. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Buildfarm issues on specific machines
On Tue, Jul 19, 2005 at 02:29:08PM -0400, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: On Sat, Jul 16, 2005 at 11:17:29PM -0400, Tom Lane wrote: cuckoo [7.3, 7.2]: --enable-nls without OS support This looks like pilot error; but the later branches don't fail on this machine, so did we change something in this area? Should I just stop using nls on 7.2 and 7.3 or does someone want to poke around with this further? I don't intend to look at it; but perhaps someone who cares more about the NLS code will ... Then I guess the question is... is it more valuable to have a working buildfarm environment for 7.2 and 7.3, or is the obnoxious failure better to spur someone into looking at it? :) Should this maybe be made a TODO and I'll adjust my config until someone tackles the TODO? Also, what do people think about having the buildfarm track different compile/build options on each environment? ISTM there's value in being able to change-up config options to make sure that different combinations work. My thought is having the buildfarm configured so that it knows what options on a machine should work (based on external dependancies) and then the script can cycle through different configs. Of course this means the server would have to do a better job of tracking per-config-setting info... -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Buildfarm issues on specific machines
Jim C. Nasby wrote: Then I guess the question is... is it more valuable to have a working buildfarm environment for 7.2 and 7.3, or is the obnoxious failure better to spur someone into looking at it? :) Should this maybe be made a TODO and I'll adjust my config until someone tackles the TODO? I don't think 7.2 and 7.3 deserve heroic efforts to get every possible build in a green state. The main reason to run buildfarm at all on these branches is to make sure that any maintenance changes don't break things. Also, what do people think about having the buildfarm track different compile/build options on each environment? ISTM there's value in being able to change-up config options to make sure that different combinations work. My thought is having the buildfarm configured so that it knows what options on a machine should work (based on external dependancies) and then the script can cycle through different configs. Of course this means the server would have to do a better job of tracking per-config-setting info... I actually abandoned an earlier attempt to create a buildfarm because I tried to cater for every possible combination. I do not want to get into that again. Buildfarm is not going to find every problem, no matter how hard we try. So I want to follow the KISS principle, if for no other reason than that I would far rather be working on cool postgres features than on the buildfarm :-) There is already a good list of features to be worked on. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Buildfarm issues on specific machines
On Tue, Jul 19, 2005 at 03:17:49PM -0400, Andrew Dunstan wrote: Jim C. Nasby wrote: Then I guess the question is... is it more valuable to have a working buildfarm environment for 7.2 and 7.3, or is the obnoxious failure better to spur someone into looking at it? :) Should this maybe be made a TODO and I'll adjust my config until someone tackles the TODO? I don't think 7.2 and 7.3 deserve heroic efforts to get every possible build in a green state. The main reason to run buildfarm at all on these branches is to make sure that any maintenance changes don't break things. OK, I'll tweak cuckoo's config accordingly then. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Buildfarm issues on specific machines
(trimming cc list...) On Tue, Jul 19, 2005 at 02:25:38PM -0500, Jim C. Nasby wrote: OK, I'll tweak cuckoo's config accordingly then. And now it's failing on make, at least for 7.2... ccache gcc -O3 -pipe -traditional-cpp -g -O2 -g -Wall -Wmissing-prototypes -Wmissing-declarations -I. -I../../../src/include -I/opt/local/include -c -o bootparse.o bootparse.c y.tab.c:953: warning: no previous prototype for `Int_yyparse' bootparse.y: In function `Int_yyparse': bootparse.y:278: error: syntax error at '##' token bootparse.y:278: error: `T_' undeclared (first use in this function) bootparse.y:278: error: (Each undeclared identifier is reported only once bootparse.y:278: error: for each function it appears in.) bootparse.y:278: error: parse error before IndexElem make[3]: *** [bootparse.o] Error 1 make[2]: *** [bootstrap-recursive] Error 2 make[1]: *** [all] Error 2 make: *** [all] Error 2 -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
And 7.3 is also failing, with a different error: ccache gcc -traditional-cpp -g -O2 -fno-strict-aliasing -g -Wall -Wmissing-prototypes -Wmissing-declarations -I../../../../src/include -I/opt/local/include -c -o printtup.o printtup.c In file included from /usr/include/machine/param.h:30, from /usr/include/sys/socket.h:67, from ../../../../src/include/libpq/pqcomm.h:28, from ../../../../src/include/libpq/libpq-be.h:24, from ../../../../src/include/libpq/libpq.h:21, from printtup.c:20: /usr/include/ppc/param.h:98: macro btodb requires 2 arguments, but only 1 given /usr/include/ppc/param.h:100: macro dbtob requires 2 arguments, but only 1 given make[4]: *** [printtup.o] Error 1 make[3]: *** [common-recursive] Error 2 make[2]: *** [access-recursive] Error 2 make[1]: *** [all] Error 2 make: *** [all] Error 2 See http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=cuckoodt=2005-07-19%2019:38:17 and http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=cuckoodt=2005-07-19%2019:28:30 -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Buildfarm issues on specific machines
On Sun, 17 Jul 2005, Tom Lane wrote:
The short answer is that you should install flex 2.5.4, or else forget
about testing the 7.2 branch. I don't think anyone will be very
interested in making 7.2 work with flex 2.5.31.
Actually there are problems in the 7.3 branch as well in the cube,
tsearch, and seg modules. Here are some patches for the 7.2 version check
and 7.2 and 7.3 tsearch code. I'll work on getting cube and seg up to
speed as well if people agree we want these fixes.
Kris JurkaIndex: config/programs.m4
===
RCS file: /projects/cvsroot/pgsql/config/programs.m4,v
retrieving revision 1.7
diff -c -r1.7 programs.m4
*** config/programs.m4 28 Aug 2001 14:59:11 - 1.7
--- config/programs.m4 16 Jul 2005 18:11:24 -
***
*** 26,32
then
echo '%%' conftest.l
if $pgac_candidate -t conftest.l 2/dev/null | grep FLEX_SCANNER
/dev/null 21; then
! if $pgac_candidate --version | grep '2\.5\.3' /dev/null 21; then
pgac_broken_flex=$pgac_candidate
continue
fi
--- 26,32
then
echo '%%' conftest.l
if $pgac_candidate -t conftest.l 2/dev/null | grep FLEX_SCANNER
/dev/null 21; then
! if $pgac_candidate --version | grep '2\.5\.3$' /dev/null 21; then
pgac_broken_flex=$pgac_candidate
continue
fi
? contrib/tsearch/libtsearch.so.0.0
? contrib/tsearch/parser.c
? contrib/tsearch/results
? contrib/tsearch/tsearch.sql
Index: contrib/tsearch/parser.h
===
RCS file: /projects/cvsroot/pgsql/contrib/tsearch/Attic/parser.h,v
retrieving revision 1.3
diff -c -r1.3 parser.h
*** contrib/tsearch/parser.h28 Oct 2001 06:25:41 - 1.3
--- contrib/tsearch/parser.h16 Jul 2005 18:11:53 -
***
*** 5,11
int tokenlen;
int tsearch_yylex(void);
void start_parse_str(char *, int);
- void start_parse_fh(FILE *, int);
void end_parse(void);
#endif
--- 5,10
Index: contrib/tsearch/parser.l
===
RCS file: /projects/cvsroot/pgsql/contrib/tsearch/Attic/parser.l,v
retrieving revision 1.1
diff -c -r1.1 parser.l
*** contrib/tsearch/parser.l12 Oct 2001 23:19:09 - 1.1
--- contrib/tsearch/parser.l16 Jul 2005 18:11:53 -
***
*** 1,56
%{
#include string.h
#include deflex.h
#include parser.h
- /* postgres allocation function */
- #include postgres.h
- #define free pfree
- #define mallocpalloc
- #define realloc repalloc
-
- #ifdef strdup
- #undef strdup
- #endif
- #define strduppstrdup
-
-
char *token = NULL; /* pointer to token */
char *s = NULL; /* for returning full defis-word */
YY_BUFFER_STATE buf = NULL; /* buffer to parse; it need for parse from string
*/
- int lrlimit = -1; /* for limiting read from filehandle ( -1 - unlimited
read ) */
- int bytestoread = 0; /* for limiting read from filehandle */
-
- /* redefine macro for read limited length */
- #define YY_INPUT(buf,result,max_size) \
- if ( yy_current_buffer-yy_is_interactive ) { \
- int c = '*', n; \
- for ( n = 0; n max_size \
- (c = getc( tsearch_yyin )) != EOF c != '\n';
++n ) \
- buf[n] = (char) c; \
- if ( c == '\n' ) \
- buf[n++] = (char) c; \
- if ( c == EOF ferror( tsearch_yyin ) ) \
- YY_FATAL_ERROR( input in flex scanner failed ); \
- result = n; \
- } else { \
- if ( lrlimit == 0 ) \
- result=YY_NULL; \
- else { \
- if ( lrlimit0 ) { \
- bytestoread = ( lrlimit max_size ) ? max_size
: lrlimit; \
- lrlimit -= bytestoread; \
- } else \
- bytestoread = max_size; \
- if ( ((result = fread( buf, 1, bytestoread,
tsearch_yyin )) == 0) \
-ferror( tsearch_yyin ) ) \
- YY_FATAL_ERROR( input in flex scanner failed
); \
- } \
- }
-
- #define YY_NO_UNPUT
%}
/* parser's state for parsing defis-word */
--- 1,14
%{
+ #include postgres.h
#include string.h
#include deflex.h
#include parser.h
char *token = NULL; /* pointer to token */
char *s = NULL; /* for returning full defis-word */
YY_BUFFER_STATE buf = NULL; /* buffer to parse; it need for parse from string
*/
%}
/* parser's state for parsing defis-word */
***
*** 308,320
BEGIN INITIAL;
}
- /* start
Re: [HACKERS] Buildfarm issues on specific machines
First off, thanks for looking into this, Tom, and thanks to Andrew for all the stellar work on the buildfarm, I'm glad to be a part of it. Perhaps this will help in the diagnosis of why REL7_2_STABLE fails on arbor (aka caribou). Please let me know if there is anything I can try on this side, or if there is any other info you could use. Thanks, Pete arbor:~# whereis flex flex: /usr/bin/flex /usr/share/man/man1/flex.1.gz arbor:~# whereis gcc gcc: /usr/bin/gcc /usr/share/man/man1/gcc.1.gz [EMAIL PROTECTED]:~$ flex -V flex 2.5.31 arbor:~# crontab -l 30 2 * * * PATH=/bin:/usr/bin:/home/pete/bin:/home/pete/projects/pg_build_farm/build-farm-2.05 cd ~/projects/pg_build_farm/build-farm-2.05 for foo in HEAD REL7_2_STABLE REL7_3_STABLE REL7_4_STABLE REL8_0_STABLE REL8_1_STABLE; do ./run_build.pl --verbose $foo; done On Sat, Jul 16, 2005 at 11:17:29PM -0400, Tom Lane wrote: [Clip] caribou [7.2]: no flex installed This looks like pilot error as well, though again I don't understand why the later branches seem to work. Are we sure the same PATH is being used for every branch here? Why doesn't the buildfarm report for 7.2 show the PATH? regards, tom lane -- Peter D. St. Onge, http://pete.economics.utoronto.ca/ IT Admin, Department of Economics [EMAIL PROTECTED] SB01, 150 St. George, University of Toronto 416-946-3594 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Buildfarm issues on specific machines
Tom, thanks for this. I regularly send out private emails about what appear to be local issues. Tom Lane wrote: I spent a little time today cleaning up easily-fixed problems that are causing buildfarm failures in various back branches. Hopefully that will result in a few more green entries over the new few days. While I was looking, I noticed several machines that seem to be failing because of local issues: potoroo [HEAD, 7.4]: lock file /tmp/.s.PGSQL.65432.lock already exists I'm not sure if this is a problem with a stale lock file left around from an old run, or if it happens because the machine is configured to try to build/test several branches in parallel. In any case, it might be worthwhile to try to hack the buildfarm script so that the Unix socket files are allocated in a per-branch scratch directory, not in /tmp. Or try to change pg_regress to use different port numbers for different branches? Buildfarm is set up so that each branch gets its own non-standard port. It also prevents you (via a lock file) from running concurrently on a branch unless you have multiple repositories. So only make check should have any problems here, not any of the other tests, and only because the port for that set of tests is hardcoded. So I think the right (and certainly simplest) thing is to make the tmp port relative to the default port. Something like what is below - I chose 5 instead of 6 as the leading digit to avoid possible overflow where the default port number is a high 4 digit number. I guess if we were paranoid we could also check for 5 digit default port numbers. cheers andrew Index: src/test/regress/GNUmakefile === RCS file: /projects/cvsroot/pgsql/src/test/regress/GNUmakefile,v retrieving revision 1.49 diff -c -r1.49 GNUmakefile *** src/test/regress/GNUmakefile11 May 2005 21:52:03 - 1.49 --- src/test/regress/GNUmakefile17 Jul 2005 14:52:59 - *** *** 45,50 --- 45,51 -e 's,@libdir@,$(libdir),g' \ -e 's,@pkglibdir@,$(pkglibdir),g' \ -e 's,@datadir@,$(datadir),g' \ + -e 's,@default_port@,$(default_port),g' \ -e 's/@VERSION@/$(VERSION)/g' \ -e 's/@host_tuple@/$(host_tuple)/g' \ -e 's,@GMAKE@,$(MAKE),g' \ Index: src/test/regress/pg_regress.sh === RCS file: /projects/cvsroot/pgsql/src/test/regress/pg_regress.sh,v retrieving revision 1.58 diff -c -r1.58 pg_regress.sh *** src/test/regress/pg_regress.sh 25 Jun 2005 23:04:06 - 1.58 --- src/test/regress/pg_regress.sh 17 Jul 2005 14:52:59 - *** *** 342,348 unset PGHOST unset PGHOSTADDR fi ! PGPORT=65432 export PGPORT # Get rid of environment stuff that might cause psql to misbehave --- 342,348 unset PGHOST unset PGHOSTADDR fi ! [EMAIL PROTECTED]@ export PGPORT # Get rid of environment stuff that might cause psql to misbehave osprey [HEAD]: could not create shared memory segment: Cannot allocate memory DETAIL: Failed system call was shmget(key=2, size=1957888, 03600). Kernel shmem settings too small... dragonfly [HEAD]: libz link error As per earlier discussion, I maintain this is local misconfiguration. cobra [7.4, 7.3, 7.2]: --with-tcl but no Tk Possibly adding --without-tk to the configure options is the right answer. Otherwise, install Tk or remove --with-tcl. cuckoo [7.3, 7.2]: --enable-nls without OS support This looks like pilot error; but the later branches don't fail on this machine, so did we change something in this area? caribou [7.2]: no flex installed This looks like pilot error as well, though again I don't understand why the later branches seem to work. Are we sure the same PATH is being used for every branch here? Why doesn't the buildfarm report for 7.2 show the PATH? regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
Kris Jurka [EMAIL PROTECTED] writes: On Sun, 17 Jul 2005, Tom Lane wrote: The short answer is that you should install flex 2.5.4, or else forget about testing the 7.2 branch. I don't think anyone will be very interested in making 7.2 work with flex 2.5.31. Actually there are problems in the 7.3 branch as well in the cube, tsearch, and seg modules. Here are some patches for the 7.2 version check and 7.2 and 7.3 tsearch code. I'll work on getting cube and seg up to speed as well if people agree we want these fixes. The cube and seg fixes were pretty invasive, and I have a feeling that there were other changes needed in all the .l files to play nice with 2.5.31 (though a quick troll of the CVS logs failed to show any other patches specifically marked that way). My opinion is that we should just say flex 2.5.31 is unsupported before PG 7.4 --- and possibly fix the configure tests to reject it in 7.3 too. This is a considerably bigger issue for the buildfarm than it would be for ordinary users of our distribution, since in the distro it's only the contrib modules that you actually need to run through your local flex. On a buildfarm machine you had better be running the project-approved versions of bison and flex. At this writing, the approved version of flex is still 2.5.4, even for CVS tip. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Buildfarm issues on specific machines
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: potoroo [HEAD, 7.4]: lock file /tmp/.s.PGSQL.65432.lock already exists I'm not sure if this is a problem with a stale lock file left around from an old run, or if it happens because the machine is configured to try to build/test several branches in parallel. In any case, it might be worthwhile to try to hack the buildfarm script so that the Unix socket files are allocated in a per-branch scratch directory, not in /tmp. Or try to change pg_regress to use different port numbers for different branches? [ snip ] So I think the right (and certainly simplest) thing is to make the tmp port relative to the default port. I was thinking more along the lines of altering pg_regress to let the temp port be specified as a script parameter, though certainly there is nothing wrong with letting the Makefile supply 5 as the normal value. I believe that if we do something like TEMP_PORT = 5$(default_port) check: pg_regress ... --temp_port=$(TEMP_PORT) then the port could be overridden without any source code hacks by gmake TEMP_PORT=nnn check. In any case, there's still a question in my mind of exactly what is happening on potoroo. The times of the recent reports don't look like it's been configured to try to do multiple builds in parallel, so why is it failing like this? Has a temporary-installation postmaster somehow been left running somewhere? (The mere existence of the lock file would not cause this failure --- the PID in the lock file must correspond to an existing process matching the postmaster's UID, else it'll be deemed a stale lock file.) regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Buildfarm issues on specific machines
Tom Lane wrote: I believe that if we do something like TEMP_PORT = 5$(default_port) check: pg_regress ... --temp_port=$(TEMP_PORT) then the port could be overridden without any source code hacks by gmake TEMP_PORT=nnn check. Works for me. Let's do it. If I understand this right it would not need any changes on the buildfarm side. cheers andrew ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Buildfarm issues on specific machines
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: I believe that if we do something like TEMP_PORT = 5$(default_port) check: pg_regress ... --temp_port=$(TEMP_PORT) then the port could be overridden without any source code hacks by gmake TEMP_PORT=nnn check. Works for me. Let's do it. If I understand this right it would not need any changes on the buildfarm side. Done as far back as 7.4, so that this should actually work as long as you don't try to parallel-test 7.3 and 7.2. The buildfarm config stuff should recommend choosing 4-digit port numbers, because the patch I put in will fall back to 65432 if the configuration port is 5 digits. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Buildfarm issues on specific machines
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
I believe that if we do something like
TEMP_PORT = 5$(default_port)
check:
pg_regress ... --temp_port=$(TEMP_PORT)
then the port could be overridden without any source code hacks by
gmake TEMP_PORT=nnn check.
Works for me. Let's do it. If I understand this right it would not need
any changes on the buildfarm side.
Done as far back as 7.4, so that this should actually work as long as
you don't try to parallel-test 7.3 and 7.2.
Excellent. Thankyou.
The buildfarm config stuff should recommend choosing 4-digit port
numbers, because the patch I put in will fall back to 65432 if the
configuration port is 5 digits.
We alread had this in the distributed sample config:
branch_ports =
{
REL8_0_STABLE = 5682,
REL7_4_STABLE = 5681,
REL7_3_STABLE = 5680,
REL7_2_STABLE = 5679,
HEAD = 5678,
}
I have added your recommendation to the comments.
cheers
andrew
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
[HACKERS] Buildfarm issues on specific machines
I spent a little time today cleaning up easily-fixed problems that are causing buildfarm failures in various back branches. Hopefully that will result in a few more green entries over the new few days. While I was looking, I noticed several machines that seem to be failing because of local issues: potoroo [HEAD, 7.4]: lock file /tmp/.s.PGSQL.65432.lock already exists I'm not sure if this is a problem with a stale lock file left around from an old run, or if it happens because the machine is configured to try to build/test several branches in parallel. In any case, it might be worthwhile to try to hack the buildfarm script so that the Unix socket files are allocated in a per-branch scratch directory, not in /tmp. Or try to change pg_regress to use different port numbers for different branches? osprey [HEAD]: could not create shared memory segment: Cannot allocate memory DETAIL: Failed system call was shmget(key=2, size=1957888, 03600). Kernel shmem settings too small... dragonfly [HEAD]: libz link error As per earlier discussion, I maintain this is local misconfiguration. cobra [7.4, 7.3, 7.2]: --with-tcl but no Tk Possibly adding --without-tk to the configure options is the right answer. Otherwise, install Tk or remove --with-tcl. cuckoo [7.3, 7.2]: --enable-nls without OS support This looks like pilot error; but the later branches don't fail on this machine, so did we change something in this area? caribou [7.2]: no flex installed This looks like pilot error as well, though again I don't understand why the later branches seem to work. Are we sure the same PATH is being used for every branch here? Why doesn't the buildfarm report for 7.2 show the PATH? regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Buildfarm issues on specific machines
Pete St. Onge [EMAIL PROTECTED] writes: Perhaps this will help in the diagnosis of why REL7_2_STABLE fails on arbor (aka caribou). Please let me know if there is anything I can try on this side, or if there is any other info you could use. [EMAIL PROTECTED]:~$ flex -V flex 2.5.31 Ah. PG 7.2's configure script thinks this version of flex is too old to use, and so ignores it. It's actually too new --- 2.5.4 is the only flex version safe to use with older PG releases. I recall that we have seen some apparent bugs in 2.5.31; I don't recall at the moment whether we've worked around all the issues as of recent releases, but we surely had not as of 7.2. The short answer is that you should install flex 2.5.4, or else forget about testing the 7.2 branch. I don't think anyone will be very interested in making 7.2 work with flex 2.5.31. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] buildfarm issues
On Sun, Mar 06, 2005 at 21:12:03 -0800, Josh Berkus josh@agliodbs.com wrote: Also, I think you should be recording the compile-time switches used on each machine and indexing them indivdually. I'd hate to find out that, for example, we'd broken --with-odbc and didn't know it because nobody in the buildfarm was using it. Another significant example is --enable-integer-datetimes. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] buildfarm issues
Andrew, or 2.6.x ... in fact it's almost impossible to tell what might be installed on a Gentoo system, or how it was compiled. So I'm really not sure how we should treat such systems. Distribution, General OS, Kernel Version, Other Info e.g. SuSELinux 2.6.8-7 64-Bit MS Windows XP SP2 Also, I think you should be recording the compile-time switches used on each machine and indexing them indivdually. I'd hate to find out that, for example, we'd broken --with-odbc and didn't know it because nobody in the buildfarm was using it. Second is the fact that systems change over time. People upgrade their machines. I'm considering a facility to allow people to change the os-version,compiler-version aspects of their registered personality - these will become essentially timestamped pieces of information, so we'll still be able to tie a set of values to a history item. Yes ... actually, a quick (and automated) reminder e-mail once a month wouldn't hurt. And this machine registration info could be kept as a config file in the Buildfarm directory, for ease of update, and sent in with the buildfarm results. Fourth, there are some significant gaps in the buildfarm coverage. I am I've a few responses from the PWN, such as a YDL machine on PPC4. I'll be passing these people along this week. Finally, a volunteer or two to help me with both running and administering this would be great. Knowledhge of perl and Template Toolkit a decided advantage. I can help with the people end of things, but my web skills suck. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] buildfarm issues
Now that we've been running for a while there are a few buildfarm issues that I need to address. First, do we keep the right data about the members? Essentially, we keep: operating_system, os_version, compiler, compiler_version, architecture. For Linux, we genarlly ask for the Distribution/distro-version instead of the OS/os-version. However, that lead to interesting situations - Gentoo for example is so flexible that in version 2004.03 you might easily be using kernel version 2.4.x or 2.6.x ... in fact it's almost impossible to tell what might be installed on a Gentoo system, or how it was compiled. So I'm really not sure how we should treat such systems. Second is the fact that systems change over time. People upgrade their machines. I'm considering a facility to allow people to change the os-version,compiler-version aspects of their registered personality - these will become essentially timestamped pieces of information, so we'll still be able to tie a set of values to a history item. Third, what can be done to improve the usefulness of the server / web site? I already have the following items on my list of TODOs: . allow sorting of member list by personality feature (e.g. see all the OS/X boxes together) . filter dashboard page to only show certain members (this already works but you have to construct the list manually, e.g. http://www.pgbuildfarm.org/cgi-bin/show_status.pl?member=catmember=dogmember=lorismember=gibbon ) . an email alerting system for when a system changes state Fourth, there are some significant gaps in the buildfarm coverage. I am currently personally supplying the Windows and Cygwin members, but since that is on my laptop which is frequently required for other work it's not ideal, and there can be large gaps. Other gaps include HPUX, for example. If you run a machine that should be covered and can help, please do. After the initial setup there should be very little work involved. Finally, a volunteer or two to help me with both running and administering this would be great. Knowledhge of perl and Template Toolkit a decided advantage. Buildfarm was created to assist the principal hackers - so any and all (constructive) input is welcome. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] buildfarm issues
On Friday 04 March 2005 10:11, Andrew Dunstan wrote: Now that we've been running for a while there are a few buildfarm issues that I need to address. First, do we keep the right data about the members? Essentially, we keep: operating_system, os_version, compiler, compiler_version, architecture. For Linux, we genarlly ask for the Distribution/distro-version instead of the OS/os-version. However, that lead to interesting situations - Gentoo for example is so flexible that in version 2004.03 you might easily be using kernel version 2.4.x or 2.6.x ... in fact it's almost impossible to tell what might be installed on a Gentoo system, or how it was compiled. So I'm really not sure how we should treat such systems. Second is the fact that systems change over time. People upgrade their machines. I'm considering a facility to allow people to change the os-version,compiler-version aspects of their registered personality - these will become essentially timestamped pieces of information, so we'll still be able to tie a set of values to a history item. What about using uname(1), cc -v, etc to glean this information and post it with each event logged? I belive you have all this stuff already in the config.log that is used already ? Third, what can be done to improve the usefulness of the server / web site? I already have the following items on my list of TODOs: . allow sorting of member list by personality feature (e.g. see all the OS/X boxes together) . filter dashboard page to only show certain members (this already works but you have to construct the list manually, e.g. http://www.pgbuildfarm.org/cgi-bin/show_status.pl?member=catmember=dogmem ber=lorismember=gibbon ) . an email alerting system for when a system changes state Fourth, there are some significant gaps in the buildfarm coverage. I am currently personally supplying the Windows and Cygwin members, but since that is on my laptop which is frequently required for other work it's not ideal, and there can be large gaps. Other gaps include HPUX, for example. If you run a machine that should be covered and can help, please do. After the initial setup there should be very little work involved. Finally, a volunteer or two to help me with both running and administering this would be great. Knowledhge of perl and Template Toolkit a decided advantage. Buildfarm was created to assist the principal hackers - so any and all (constructive) input is welcome. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] buildfarm issues
Darcy Buskermolen wrote: On Friday 04 March 2005 10:11, Andrew Dunstan wrote: Now that we've been running for a while there are a few buildfarm issues that I need to address. First, do we keep the right data about the members? Essentially, we keep: operating_system, os_version, compiler, compiler_version, architecture. For Linux, we genarlly ask for the Distribution/distro-version instead of the OS/os-version. However, that lead to interesting situations - Gentoo for example is so flexible that in version 2004.03 you might easily be using kernel version 2.4.x or 2.6.x ... in fact it's almost impossible to tell what might be installed on a Gentoo system, or how it was compiled. So I'm really not sure how we should treat such systems. Second is the fact that systems change over time. People upgrade their machines. I'm considering a facility to allow people to change the os-version,compiler-version aspects of their registered personality - these will become essentially timestamped pieces of information, so we'll still be able to tie a set of values to a history item. What about using uname(1), cc -v, etc to glean this information and post it with each event logged? I belive you have all this stuff already in the config.log that is used already ? See previous para - on Linux we want the distro name and version, not Linux plus kernel version. uname doesn't seem to help much there. Also, I have no idea how portable cc -v is. Can we guarantee to have the compiler version properly identified on every platform? cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] buildfarm issues
Andrew, A couple of things, 1. we need to develop a matrix of systems/os/compiler to see what coverage we do have and compare it to the INSTALL guide. 2. the run_build.pl should be changed to keep the information on the system to date (and have the matrix in 1 change) 3. have the run_build.pl script check the build farm CVS to see if there is a new version of the build farm code and either download and/or alert the owner of the system I am able and willing to help out in all of the above. I am also willing to help fill the gap in systems if needed. I have been using perl for over 10 years now (since perl 4 days) and have been doing web/postgres coding for the last 6 years. Let me know what I can do to help Jim -- Original Message --- From: Andrew Dunstan [EMAIL PROTECTED] To: Darcy Buskermolen [EMAIL PROTECTED] Cc: PostgreSQL-development pgsql-hackers@postgresql.org Sent: Fri, 04 Mar 2005 14:28:09 -0500 Subject: Re: [HACKERS] buildfarm issues Darcy Buskermolen wrote: On Friday 04 March 2005 10:11, Andrew Dunstan wrote: Now that we've been running for a while there are a few buildfarm issues that I need to address. First, do we keep the right data about the members? Essentially, we keep: operating_system, os_version, compiler, compiler_version, architecture. For Linux, we genarlly ask for the Distribution/distro-version instead of the OS/os-version. However, that lead to interesting situations - Gentoo for example is so flexible that in version 2004.03 you might easily be using kernel version 2.4.x or 2.6.x ... in fact it's almost impossible to tell what might be installed on a Gentoo system, or how it was compiled. So I'm really not sure how we should treat such systems. Second is the fact that systems change over time. People upgrade their machines. I'm considering a facility to allow people to change the os-version,compiler-version aspects of their registered personality - these will become essentially timestamped pieces of information, so we'll still be able to tie a set of values to a history item. What about using uname(1), cc -v, etc to glean this information and post it with each event logged? I belive you have all this stuff already in the config.log that is used already ? See previous para - on Linux we want the distro name and version, not Linux plus kernel version. uname doesn't seem to help much there. Also, I have no idea how portable cc -v is. Can we guarantee to have the compiler version properly identified on every platform? cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq --- End of Original Message --- ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] buildfarm issues
Andrew Dunstan [EMAIL PROTECTED] writes: Also, I have no idea how portable cc -v is. Not at all. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
