[HACKERS] Japanies translation breaks solaris build
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_mothdt=2010-05-13%2021:06:01 msgfmt -o po/ja.mo po/ja.po WARNING: the string after closing is ignored at line number 11. Error, No space after directive at line number 2008. ERROR: Exiting... gmake[2]: *** [po/ja.mo] Error 2 The problem is that it contains mix of DOS/Unix end of lines. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Japanies translation breaks solaris build
Takahiro Itagaki píše v pá 14. 05. 2010 v 19:38 +0900: Zdenek Kotala zdenek.kot...@sun.com wrote: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_mothdt=2010-05-13%2021:06:01 The problem is that it contains mix of DOS/Unix end of lines. I removed two CRs in ja.po. Thanks. Gothic moth is green again http://www.pgbuildfarm.org/cgi-bin/show_history.pl?nm=gothic_mothbr=HEAD The rest solaris machine will recover during a night. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Hi Tom, I'm sorry that I did not look on it early. I played with it and there are some facts. gothic(sparc) and codlin(x86) uses Sun Studio 12 nad I setup them to use very high optimization. Gothic: --- -xalias_level=basic -xarch=native -xdepend -xmemalign=8s -xO5 -xprefetch=auto,explicit Codlin: --- -xalias_level=basic -xarch=native -xdepend -xO4 -xprefetch=auto,explicit -xO5 is highest optimization, -xO4 is little bit worse A play with flags and found that -xO4 -xalias_level=basic generates problem. -xO3 -xalias_level=basic works fine -xO5 works fine As documentation say: Cite from Sun studio compiler guide: http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view xalias_level=basic -- If you use the -xalias_level=basic option, the compiler assumes that memory references that involve different C basic types do not alias each other. The compiler also assumes that references to all other types can alias each other as well as any C basic type. The compiler assumes that references using char * can alias any other type. For example, at the -xalias_level=basic level, the compiler assumes that a pointer variable of type int * is not going to access a float object. Therefore it is safe for the compiler to perform optimizations that assume a pointer of type float * will not alias the same memory that is referenced with a pointer of type int *. -x04 - Preforms automatic inlining of functions contained in the same file in addition to performing -xO3 optimizations. This automatic inlining usually improves execution speed, but sometimes makes it worse. In general, this level results in increased code size. I redefined bitfields to char in HLWORD and it works. Your guess is correct. But question still where is the place when bitfields works bad. Any idea where I should look? IIRC, I had this problem also on head, when I tried to fix tsearch regression test for Czech locale. This problem appears and disappears. Zdenek Dne 11.03.10 00:37, Tom Lane napsal(a): Since the buildfarm is mostly green these days, I took some time to look into the few remaining consistent failures. One is that gothic_moth and codlin_moth fail on contrib/tsearch2 in the 8.2 branch, with a regression diff like this: *** 2453,2459 body bSea/b view wowubfoo/b bar/u iqq/i a href=http://www.google.com/foo.bar.html; target=_blankYESnbsp;/a ! ff-bg script document.write(15); /script --- 2453,2459 body bSea/b view wowubfoo/b bar/u iqq/i a href=http://www.google.com/foo.bar.html; target=_blankYESnbsp;/a ! ff-bgff-bg script document.write(15); /script These animals are not testing any branches older than 8.2. The same test appears in newer branches and passes, but the code involved got migrated to core and probably changed around a bit. I traced through this test on my own machine and determined that the way it's supposed to work is like this: the tsearch parser breaks the string into a series of tokens that include these: ff-bg compound word ff compound word element - punctuation bg compound word element The highlight function is then supposed to set skip = 1 on the compound word, causing it to be skipped when genhl() reconstructs the text. The failure looks to me like the compound word is not getting skipped. Both the setting and the testing of the flag seem to be absolutely straightforward portable code; although the skip struct field is a bitfield, which is a C feature we don't use very heavily. My conclusion is that this is probably a compiler bug. Both buildfarm animals appear to be using Sun Studio, although on different architectures which weakens the compiler-bug theory a bit. Even though we are not seeing the failure in later PG branches, it would probably be worth looking into more closely, because if it's biting 8.2 it could start biting newer code as well. But without access to a machine showing the problem it is difficult to do much. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Dne 11.03.10 16:24, Greg Stark napsal(a): On Wed, Mar 10, 2010 at 11:37 PM, Tom Lanet...@sss.pgh.pa.us wrote: My conclusion is that this is probably a compiler bug. Both buildfarm animals appear to be using Sun Studio, although on different architectures which weakens the compiler-bug theory a bit. Even though we are not seeing the failure in later PG branches, it would probably be worth looking into more closely, because if it's biting 8.2 it could start biting newer code as well. But without access to a machine showing the problem it is difficult to do much. Could be this: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6750087 It's fixed in patch 124861-11 which came out Feb 23, 2009. Is this patch missing on both gothic-moth and codlin-moth? It seems as a our case. See compiler versions: Ghost: -bash-3.2$ cc -V cc: Sun C 5.9 SunOS_sparc Patch 124867-09 2008/11/25 Codlin -bash-4.0$ cc -V cc: Sun C 5.9 SunOS_i386 Patch 124868-10 2009/04/30 I should apply patch on Ghost, but Codlin have to wait, because I don't have a control on compiler version. I try to find update SS12 somewhere on the disk/network. The patch which you refer does not fix cc itself but some others binaries/libs which cc uses. I try to update Ghost and we will see what happen. I suppose it's possible to have a configure test to check for whether this patch is present but I'm not sure how much it's worthwhile given that it'll only help people who happen to recompile their 8.2 server after the next Postgres patch. And I'm not sure we can check for patches without assuming the CC is the OS-shipped cc. Does cc itself have an option to list which patches it has applied to it? cc is not shipped with solaris you have to install it separately. And bug appear only when you use high optimization (see my email). You can see patch version when you run cc -V but you see only compiler version. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Dne 11.03.10 17:37, Tom Lane napsal(a): Zdenek Kotalazdenek.kot...@sun.com writes: -xO4 -xalias_level=basic generates problem. -xO3 -xalias_level=basic works fine -xO5 works fine As documentation say: Cite from Sun studio compiler guide: http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view xalias_level=basic -- If you use the -xalias_level=basic option, the compiler assumes that memory references that involve different C basic types do not alias each other. The compiler also assumes that references to all other types can alias each other as well as any C basic type. The compiler assumes that references using char * can alias any other type. For example, at the -xalias_level=basic level, the compiler assumes that a pointer variable of type int * is not going to access a float object. Therefore it is safe for the compiler to perform optimizations that assume a pointer of type float * will not alias the same memory that is referenced with a pointer of type int *. I think you need to turn that off. On gcc we use -fno-strict-aliasing which disables the type of compiler assumption that this is talking about. I'm not sure exactly how that might create the specific failure we are seeing here, but I can point you to lots and lots of places in the sources where such an assumption would break things. OK. I first try to update compiler to latest version to see if it helps and finally I will remove aliasing. Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Tom Lane píše v čt 11. 03. 2010 v 11:37 -0500: Zdenek Kotala zdenek.kot...@sun.com writes: -xO4 -xalias_level=basic generates problem. -xO3 -xalias_level=basic works fine -xO5 works fine As documentation say: Cite from Sun studio compiler guide: http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view xalias_level=basic -- If you use the -xalias_level=basic option, the compiler assumes that memory references that involve different C basic types do not alias each other. The compiler also assumes that references to all other types can alias each other as well as any C basic type. The compiler assumes that references using char * can alias any other type. For example, at the -xalias_level=basic level, the compiler assumes that a pointer variable of type int * is not going to access a float object. Therefore it is safe for the compiler to perform optimizations that assume a pointer of type float * will not alias the same memory that is referenced with a pointer of type int *. I think you need to turn that off. On gcc we use -fno-strict-aliasing which disables the type of compiler assumption that this is talking about. I'm not sure exactly how that might create the specific failure we are seeing here, but I can point you to lots and lots of places in the sources where such an assumption would break things. Reconfigured and both animal are green. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql with GSS can crash
Magnus Hagander píše v po 01. 03. 2010 v 16:55 +0100: 2010/3/1 Zdenek Kotala zdenek.kot...@sun.com: Magnus Hagander píše v čt 25. 02. 2010 v 15:17 +0100: On Thu, Feb 25, 2010 at 15:04, Zdenek Kotala zdenek.kot...@sun.com wrote: Hi all, I got following stack: fd7ffed14b70 strlen () + 40 fd7ffed71665 snprintf () + e5 fd7fff36d088 pg_GSS_startup () + 88 fd7fff36d43a pg_fe_sendauth () + 15a fd7fff36e557 PQconnectPoll () + 3b7 fd7fff36e152 connectDBComplete () + a2 fd7fff36dc32 PQsetdbLogin () + 1b2 0041e96d main () + 30d 0041302c () It seems that connection is not fully configured and krbsrvname or pghost is not filled. Following code in fe-auth.c pg_GSS_startup() causes a crash: 440 maxlen = NI_MAXHOST + strlen(conn-krbsrvname) + 2; 441 temp_gbuf.value = (char *) malloc(maxlen); 442 snprintf(temp_gbuf.value, maxlen, %...@%s, 443 conn-krbsrvname, conn-pghost); 444 temp_gbuf.length = strlen(temp_gbuf.value); And following code in fe-connect.c fillPGconn() fill NULL value. 571 tmp = conninfo_getval(connOptions, krbsrvname); 572 conn-krbsrvname = tmp ? strdup(tmp) : NULL; I think that pg_GSS_startup should sanity the input. How did you get NULL in there? :-) There's a default set for that one that's PG_KRB_SRVNAM, so it really should never come out as NULL, I think... Yeah, you are right. conn-krbsrvname is postgres and conn-pghost is null Ah, good. We should defentd against that then. As for pghost, that certainly seems to be a bug. We check that one in krb5 and SSPI, but for some reason we seem to be missing it in GSSAPI. Yes. The check should be in GSSAPI too. However what I see in pg_hba.conf is following line: local all all gss Gss is used on local unix socket which probably cause a problem that conn-pghost is not filled when psql tries to connect. So there are really two errors - because we should disallow that. See attached patch - can you confirm it removes the crash with just the client side applied, and then that it properly rejects GSS with the server side applied as well? I tested it, but I cannot reproduce crash because I cannot setup illegal combination now ;-). I think it is OK. Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql with GSS can crash
Magnus Hagander píše v čt 25. 02. 2010 v 15:17 +0100: On Thu, Feb 25, 2010 at 15:04, Zdenek Kotala zdenek.kot...@sun.com wrote: Hi all, I got following stack: fd7ffed14b70 strlen () + 40 fd7ffed71665 snprintf () + e5 fd7fff36d088 pg_GSS_startup () + 88 fd7fff36d43a pg_fe_sendauth () + 15a fd7fff36e557 PQconnectPoll () + 3b7 fd7fff36e152 connectDBComplete () + a2 fd7fff36dc32 PQsetdbLogin () + 1b2 0041e96d main () + 30d 0041302c () It seems that connection is not fully configured and krbsrvname or pghost is not filled. Following code in fe-auth.c pg_GSS_startup() causes a crash: 440 maxlen = NI_MAXHOST + strlen(conn-krbsrvname) + 2; 441 temp_gbuf.value = (char *) malloc(maxlen); 442 snprintf(temp_gbuf.value, maxlen, %...@%s, 443 conn-krbsrvname, conn-pghost); 444 temp_gbuf.length = strlen(temp_gbuf.value); And following code in fe-connect.c fillPGconn() fill NULL value. 571 tmp = conninfo_getval(connOptions, krbsrvname); 572 conn-krbsrvname = tmp ? strdup(tmp) : NULL; I think that pg_GSS_startup should sanity the input. How did you get NULL in there? :-) There's a default set for that one that's PG_KRB_SRVNAM, so it really should never come out as NULL, I think... Yeah, you are right. conn-krbsrvname is postgres and conn-pghost is null As for pghost, that certainly seems to be a bug. We check that one in krb5 and SSPI, but for some reason we seem to be missing it in GSSAPI. Yes. The check should be in GSSAPI too. However what I see in pg_hba.conf is following line: local all all gss Gss is used on local unix socket which probably cause a problem that conn-pghost is not filled when psql tries to connect. thanks Zdenek Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] psql with GSS can crash
Hi all, I got following stack: fd7ffed14b70 strlen () + 40 fd7ffed71665 snprintf () + e5 fd7fff36d088 pg_GSS_startup () + 88 fd7fff36d43a pg_fe_sendauth () + 15a fd7fff36e557 PQconnectPoll () + 3b7 fd7fff36e152 connectDBComplete () + a2 fd7fff36dc32 PQsetdbLogin () + 1b2 0041e96d main () + 30d 0041302c () It seems that connection is not fully configured and krbsrvname or pghost is not filled. Following code in fe-auth.c pg_GSS_startup() causes a crash: 440 maxlen = NI_MAXHOST + strlen(conn-krbsrvname) + 2; 441 temp_gbuf.value = (char *) malloc(maxlen); 442 snprintf(temp_gbuf.value, maxlen, %...@%s, 443 conn-krbsrvname, conn-pghost); 444 temp_gbuf.length = strlen(temp_gbuf.value); And following code in fe-connect.c fillPGconn() fill NULL value. 571 tmp = conninfo_getval(connOptions, krbsrvname); 572 conn-krbsrvname = tmp ? strdup(tmp) : NULL; I think that pg_GSS_startup should sanity the input. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] codlin_month is up and complain - PL/Python crash
Dne 17.02.10 18:39, Peter Eisentraut napsal(a): On ons, 2010-02-17 at 11:26 -0500, Tom Lane wrote: But the behavior gcc appears to exhibit is that it won't warn about variables that are only assigned once before the PG_TRY is entered, and that seems reasonable to me since such a variable ought to have the correct value either way. FWIW, this is a Sun Studio build that is complaining here. Yes It is SS12. I add volatile keyword and problem disappears. The code difference is following: PLy_spi_execute+0x742: 83 ec 0c subl $0xc,%esp PLy_spi_execute+0x745: ff b5 b8 f9 ff ff pushl 0xf9b8(%ebp) PLy_spi_execute+0x74b: e8 fc ff ff ff call MemoryContextSwitch PLy_spi_execute+0x742: 8b 85 cc f9 ff ff movl 0xf9cc(%ebp),%eax PLy_spi_execute+0x748: 83 ec 0c subl $0xc,%esp PLy_spi_execute+0x74b: 50 pushl %eax PLy_spi_execute+0x74c: e8 fc ff ff ff call MemoryContextSwitch Good to mention that SS inline PLy_spi_execute_query inside PLy_spi_execute(), because it is only one caller. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] codlin_month is up and complain - PL/Python crash
I revived codlin_month and it falls during PL/Python test: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=codlin_mothdt=2010-02-16%2015:09:05 TRAP: BadArgument(!(((context) != 0 (Node*)((context)))-type) == T_AllocSetContext, File: mcxt.c, Line: 641) feaf5005 _lwp_kill (1, 6, 80459c8, fea9bbde) + 15 fea9bbea raise(6, 0, 8045a18, fea725aa) + 22 fea725ca abort(8046670, 8361f80, 8045a48, 8719ccf, 89021f0, 89021e4) + f2 086d07c0 ExceptionalCondition (89021f0, 89021e4, 89021dc, 281) + 58 08719ccf MemoryContextSwitchTo (89264ac, 0, 0, 8045a7c) + 47 fec21990 PLy_spi_execute (0, 8b141cc, 80460f8, fe84abde) + 750 fe84ad6e PyCFunction_Call (8b0ff6c, 8b141cc, 0, fe8a8d92) + 19e fe8a91a0 call_function (80461bc, 1, 610f2d31, fe8a3206) + 41c fe8a6221 PyEval_EvalFrameEx (8b5798c, 0, 8b0cbdc, 0) + 3029 fe8a9310 fast_function (8b05144, 80462fc, 0, 0, 0, fe91c63c) + 108 fe8a8e72 call_function (80462fc, 0, 80462d8, fe8a3206) + ee fe8a6221 PyEval_EvalFrameEx (8b576a4, 0, 8b0cbdc, 8b0cbdc) + 3029 fe8a7cd0 PyEval_EvalCodeEx (8ab4770, 8b0cbdc, 8b0cbdc, 0, 0, 0) + 91c fe8a3102 PyEval_EvalCode (8ab4770, 8b0cbdc, 8b0cbdc, fec17831) + 32 fec1799c PLy_function_handler (8046980, 8b5d508, 8046880, fec1480f) + 17c fec14b92 plpython_call_handler (8046980, 8046bb0, 8046be8, 8323774) + 3aa 08324393 ExecEvalFunc (8a033b0, 8a0329c, 8a0390c, 8a039b8) + e33 0832b1bc ExecProject (8a03920, 8046c6c, 2, 8977abc) + 834 08348785 ExecResult (8a03210, 8a03184, 0, 1) + 9d 0831f66f ExecProcNode (8a03210, 1, 8a037ec, 8731314) + 227 0831a186 ExecutorRun (8a02d7c, 1, 0, 8719ad4) + 2de 084d7778 PortalRun (898effc, 7fff, 1, 8977b38, 8977b38) + 450 084ceae9 exec_simple_query (8976984, 0, 80473b8, 84d5185) + ba9 084d51a2 PostgresMain (2, 8973b4c, 897398c, 893d00c, 893d008, 130d7661) + 7fa 0844aded BackendRun (898c3d0) + 1cd 084440f3 ServerLoop (1, 89561d4, 3, fea7bb7e, 5c54, feb83cd8) + 973 08443004 PostmasterMain (3) + 119c 0837db12 main (3, 8047b14, 8047b24, 80fa21f) + 1ea 080fa27d _start (3, 8047be8, 8047fb0, 8047fb0, 0, 8047c35) + 7d It seems that problem is with compiler aggressive optimization. I change it to lower level and now it works fine. Interesting is that MemoryContext corruption only appears with PL/Python. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] buildfarm breakage
Andrew Dunstan píše v po 08. 02. 2010 v 20:07 -0500: Our Solaris *moth members seem to have stopped building. Have we lost them? Hi Andrew, The answer is not simple. Yes, we lost Solaris 8 and 9 machines which was reinstalled and now they are used for different purpose. It was planned before the April and I announced it long time ago. It unfortunately happed and timing looks strange. And I did not find replacement. I have replacement for nevada/x86 machine already, but I need to setup it which is one item in my very long TODO list :(. Solaris 10 Sparc/x86 and nevada sparc are covered at this moment. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Deadlock in vacuum (check fails)
I found following strange error on gothic moth: == pgsql.22658/src/test/regress/regression.diffs === *** /zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/pgsql.22658/src/test/regress/expected/vacuum.out Tue Jan 12 22:06:13 2010 --- /zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/pgsql.22658/src/test/regress/results/vacuum.out Tue Jan 12 22:35:25 2010 *** *** 94,99 --- 94,103 -- only non-system tables should be changed VACUUM FULL pg_am; VACUUM FULL pg_class; + ERROR: deadlock detected + DETAIL: Process 5913 waits for AccessExclusiveLock on relation 2662 of database 16384; blocked by process 5915. + Process 5915 waits for AccessShareLock on relation 1259 of database 16384; blocked by process 5913. + HINT: See server log for query details. VACUUM FULL pg_database; VACUUM FULL vaccluster; VACUUM FULL vacid; Detail here: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=gothic_mothdt=2010-01-12%2021:06:01 Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Dne 4.01.10 19:28, Alvaro Herrera napsal(a): Bruce Momjian escribió: I considered that but realize that pg_migrator has to read pg_controldata in both the old and new servers, meaning it would need access to both C structures, and considering they both have the same structure names, that would require some odd C tricks. Add to that you don't know which version of Postgres you are migrating from/to during compile and the idea of using C becomes even less attractive. However, keep in mind that this might not be the last time on which we will want to read something from a C struct, so perhaps it would be good to bite the bullet and write the odd tricks. Does it already have access (at compile time) to the old and new source trees? I have some proof of concept when each control data struct version version have one header file like pg_control_843.h and structure like ControlFileData has name ControlFileData_843. The main pg_control.h defines types without version like typedef ControlFileData_843 ControlFileData; I planed to do it for 8.5 but unfortunately no time :( commit fest is too close. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Greg Smith píše v út 15. 12. 2009 v 12:10 -0500: Please send that updated version, and let's keep working on this into the next CommitFest, where it will be in the front of the queue rather than how it ended up at the tail of this one just based on its submission date. You're not really getting a fair chunk of time here between your review and the end here because of problems lining up reviewer time, that shouldn't happen next time. Make sense. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Bernd Helmle píše v po 14. 12. 2009 v 20:42 +0100: --On 14. Dezember 2009 20:33:12 +0100 Bernd Helmle maili...@oopsware.de wrote: Since the author has pretty much admitted he didn't fix any of the issues that were raised by the last committer review, I'm a little confused about why you're asking for another one. It wasn't clear to me what Zdenek meant with If I think about it. Oh, and i was under the opinion the last discussions were about executor probes only (note the patch is split up into two parts now, SLRU and executor probes). The latter won't be fixed, but it seems the SLRU part at least is ready. I would like to add SimpleLruInit probe. I'm busy with PG packaging, but I hope that I will send updated version tomorrow. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Bernd Helmle píše v pá 11. 12. 2009 v 17:13 +0100: --On 11. Dezember 2009 11:28:54 -0300 Alvaro Herrera alvhe...@commandprompt.com wrote: without compiled probes: AVG(2531.68) with compiled probes: AVG(2511.97) Were the probes enabled? Hmm, they were just compiled in, i didn't anything to instrument them with dtrace. I've just started a pgbench/dtrace run with instrumented probes aset_alloc, aset_free and aset_realloc which just counts the calls to them during pgbench, the first run gives me tps = 1035.045523 (excluding connections establishing) Ideas? When probes are activated they have performance impact. It is normal. Important is that you can use it when you need it on production system. No recompilation, no extra binary, no downtime and it is safe. Performance impact depends on Dscript Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Robert Haas píše v čt 10. 12. 2009 v 23:55 -0500: On Wed, Dec 9, 2009 at 9:04 AM, Zdenek Kotala zdenek.kot...@sun.com wrote: But in normal situation database does also other thing and palloc is only one part of code path. It is why I run second test and use sun studio profiling tools (collect/analyzer) to determine how much CPU ticks cost the probes during pg_bench run. And results are much better. AllocSet alloc function takes about 4-5% and probes assembler code takes 0.1-0.2% on 64bit. I did not test 32bit but my expectation is that it should be about 0.3-0.4%. There's not really enough detail here to determine what you tested and what the results were, and I don't think this patch has any chance at all of getting committed without that. Please clarify. If there's some real-world test where this probe costs 0.3%-0.4%, I think that is sufficient grounds for rejecting this patch. I understand the desire of people to be able to use dtrace, but our performance is too hard-won for me to want to give any measurable of it up for tracing and instrumentation hooks that will only be used by a small number of users in a small number of situations. As I mentioned I run pg_bench -c10 -t1000 and collect data from backends. collect and analyzer is similar tool to gprof. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Robert Haas píše v pá 11. 12. 2009 v 12:55 -0500: On Fri, Dec 11, 2009 at 12:55 PM, Zdenek Kotala zdenek.kot...@sun.com wrote: Bernd Helmle píše v pá 11. 12. 2009 v 17:13 +0100: --On 11. Dezember 2009 11:28:54 -0300 Alvaro Herrera alvhe...@commandprompt.com wrote: without compiled probes: AVG(2531.68) with compiled probes: AVG(2511.97) Were the probes enabled? Hmm, they were just compiled in, i didn't anything to instrument them with dtrace. I've just started a pgbench/dtrace run with instrumented probes aset_alloc, aset_free and aset_realloc which just counts the calls to them during pgbench, the first run gives me tps = 1035.045523 (excluding connections establishing) Ideas? When probes are activated they have performance impact. It is normal. Important is that you can use it when you need it on production system. No recompilation, no extra binary, no downtime and it is safe. Performance impact depends on Dscript Yeah. The problem here is the impact when the probes aren't enabled. I thought we had an idea of using the AllocSet dispatch mechanism to make this zero-overhead in the case where the probes are not enabled. What happened to that notion? We know that performance impact is less then 1% probably less then 0.6%. The question is if it is acceptable or not. I personally think that it is acceptable. However if not, I will start work on backup solution with dtraced AllocSet and some switching mechanism. But it needs little discussion about design. And first we need decision about current performance impact. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 11. 12. 2009 v 13:56 -0500: Robert Haas robertmh...@gmail.com writes: As far as I am concerned that is way too much, particularly considering that your test case isn't designed to be particularly memory-allocation intensive, and if it is up to me I will reject this. Even a quarter-percent slowdown for a feature that will be used only by a small fraction of users only a small fraction of time time seems totally unacceptable to me. It seems to me that anyone who really needs this can instrument the alloc functions anyway --- isn't one of the features of DTrace supposed to be that you can monitor calls to a particular function without any prearranged code support? Or is that one of the things like zero overhead that turns out to be more marketing-speak than reality? There are several types of probes. For example for PID provider probes you can monitor all entry and return point from global function. And also you can put probe on each asm instruction in the function. These probes have zero overhead, because dtrace understand ABI and know where args are. Unfortunately user defined probes has small overhead which is price for universal solution which works with all compilers and linkers. Anyway I concur with Robert's opinion that the use-case is far too small to justify incurring a measurable overhead for everybody. OK. There might be some small argument for putting these in under an extra #ifdef, but they wouldn't get into any regular production build. unfortunately #ifdef solution kills main dtrace goal - without recompilation :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 11. 12. 2009 v 14:38 -0500: Robert Haas robertmh...@gmail.com writes: I thought we had an idea of using the AllocSet dispatch mechanism to make this zero-overhead in the case where the probes are not enabled. What happened to that notion? I must have missed that discussion, but +1 --- should be possible to get to zero-overhead-when-off that way. The trick is to figure out what/where enables the alternate implementation. The current design assumes that the callers of FooContextCreate choose the implementation, but we don't want that here. I thought about it. I think we can use GUC variable (e.g. dtraced_alloc) and hook switch pointers to dtraced AsetFunctions. The problem is how to distribute to all backend. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 11. 12. 2009 v 15:11 -0500: Zdenek Kotala zdenek.kot...@sun.com writes: I thought about it. I think we can use GUC variable (e.g. dtraced_alloc) and hook switch pointers to dtraced AsetFunctions. The problem is how to distribute to all backend. You set the GUC in postgresql.conf. No big deal. If we go this route it would be nice to think about making a facility that has some usefulness for non-DTrace platforms too. Do you mean general facility for switching memory allocator? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Dne 10.12.09 15:51, Bernd Helmle napsal(a): --On 8. Dezember 2009 11:10:44 +0100 Zdenek Kotala zdenek.kot...@sun.com wrote: If you think that it is better I could split patch into two separate patches and both can be reviewed separately. I split up this patch into two separate patches: one for SLRU and one for the executor probes. I've done some documentation on the SLRU part, maybe you can look over it (to make sure i didn't break anything). I left the executor probes out of the documentation for now, since it seems not to be clear how they would manifest. Out of curiosity: Why do we want to pass the SlruCtl pointer down to the probes? I don't understand what those probes are going to do with those pointers, can you explain? You need to determine which SLRU is used. Because SLRUs are initialized during startup pointer should be same in all backends. If I think more about it. Maybe it could be goot to add probe also into SimpleLruInit to catch name of SLRUs. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Frank Ch. Eigler píše v čt 10. 12. 2009 v 14:11 -0500: Zdenek Kotala zdenek.kot...@sun.com writes: [...] + header = (StandardChunkHeader *) + ((char *) ret - STANDARDCHUNKHEADERSIZE); + +// TRACE_POSTGRESQL_MCXT_ALLOC(context-name, context, size, header-size, true); + [...] If the dormant overhead of these probes is measured or suspected to be excessive, consider using the dtrace-generated per-probe foo_ENABLED() conditional, or a postgres configuration global thusly: TRACE_POSTGRESQL_MCXT_ALLOC and TRACE_POSTGRESQL_ASET_ALLOC are duplicated probes. Have them both make sense but from performance point of view to have one of them is acceptable. foo_enable() is good to use when number of argument and their evaluation cost too much. In this case it does no seem to be much useful. See ASM code: AllocSetAlloc+0x17: xorq %rax,%rax AllocSetAlloc+0x1a: nop AllocSetAlloc+0x1b: nop AllocSetAlloc+0x1c: testl %eax,%eax AllocSetAlloc+0x1e: je +0xb AllocSetAlloc+0x2b AllocSetAlloc+0x20: movq %r13,%rdi AllocSetAlloc+0x23: movq %r14,%rsi AllocSetAlloc+0x26: nop AllocSetAlloc+0x27: nop AllocSetAlloc+0x28: nop AllocSetAlloc+0x29: nop AllocSetAlloc+0x2a: nop if (__builtin_expect(TRACE_POSTGRESQL_MCXT_ALLOC_ENABLED(), 0)) TRACE_POSTGRESQL_MCXT_ALLOC(...); so that the whole instrumentation parameter setup/call can be placed out of the hot line with gcc -freorder-blocks. compiler specific construct is not good way. Do not forget that also other compiler exists. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Greg Smith píše v út 08. 12. 2009 v 22:44 -0500: Zdenek Kotala wrote: thanks for your useful comments. I attached new doc patch version. I removed example changes and add link to create database cluster (I hope) everywhere. Please, look on it and let me know if there is still something what should be changed. That looks much better. There's only one bit that sticks out oddly now: + Note: The commandinitdb/command might be invoked by + commandpg_ctl initdb/command and commandinitdb/command cannot be in + default path on a productnamePostgreSQL/productname installations. What is that supposed to mean exactly? Ahh, It is somethink what I want to do, but it is not ready yet in this patch, but I already documented it. Idea is to install initdb and postgres into libexecdir and packager can select if libexecdir will be equal bindir or not. The paragraph should be removed at this moment. Shell I send modified patch or does committer remove it before commit? thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Bernd Helmle píše v út 08. 12. 2009 v 22:06 +0100:
--On 8. Dezember 2009 15:51:52 -0500 Greg Smith g...@2ndquadrant.com
wrote:
Try this instead, which will give you a test where checkpoints have a
minimal impact, but lots of memory will be thrown around:
pgbench -i -s 10 db
pgbench -S -c 10 -T 600 db
Thanks for the input, will try
I modified probes to reduce overhead. Prototype patch is attached. Main
point is to remove mcxt_alloc probe and keep only aset_alloc. I did also
some testing with interesting results. At first I prepare special C
store function (attached) which do only allocation and deallocation and
I measured how long it takes:
On 32bit the memory allocation is slow down 8.4% and on 64bit it is
only 4.6%. Good to mention that I call palloc and pfree but in standard
behavior pfree is not much used and memory is freed when context is
destroyed. It means that we should think about 4.2% and 2.3% instead.
But in normal situation database does also other thing and palloc is
only one part of code path. It is why I run second test and use sun
studio profiling tools (collect/analyzer) to determine how much CPU
ticks cost the probes during pg_bench run. And results are much better.
AllocSet alloc function takes about 4-5% and probes assembler code takes
0.1-0.2% on 64bit. I did not test 32bit but my expectation is that it
should be about 0.3-0.4%.
Zdenek
diff -r 68b8827f4738 src/backend/utils/mmgr/aset.c
--- a/src/backend/utils/mmgr/aset.c Fri Nov 13 11:17:04 2009 +
+++ b/src/backend/utils/mmgr/aset.c Wed Dec 09 14:41:34 2009 +0100
@@ -64,6 +64,7 @@
#include postgres.h
+#include pg_trace.h
#include utils/memutils.h
/* Define this to detail debug alloc information */
@@ -463,6 +464,8 @@
AssertArg(AllocSetIsValid(set));
+ TRACE_POSTGRESQL_ASET_RESET(context);
+
/* Nothing to do if no pallocs since startup or last reset */
if (set-isReset)
return;
@@ -495,6 +498,8 @@
#endif
block-freeptr = datastart;
block-next = NULL;
+ TRACE_POSTGRESQL_ASET_BLOCK_RESET(context, block,
+block-endptr - ((char *) block));
}
else
{
@@ -503,6 +508,8 @@
/* Wipe freed memory for debugging purposes */
memset(block, 0x7F, block-freeptr - ((char *) block));
#endif
+ TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block,
+block-endptr - ((char *) block));
free(block);
}
block = next;
@@ -529,6 +536,7 @@
AllocBlock block = set-blocks;
AssertArg(AllocSetIsValid(set));
+ TRACE_POSTGRESQL_ASET_DELETE(context);
#ifdef MEMORY_CONTEXT_CHECKING
/* Check for corruption and leaks before freeing */
@@ -548,6 +556,8 @@
/* Wipe freed memory for debugging purposes */
memset(block, 0x7F, block-freeptr - ((char *) block));
#endif
+ TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block,
+ block-endptr - ((char *) block));
free(block);
block = next;
}
@@ -570,6 +580,7 @@
AssertArg(AllocSetIsValid(set));
+ TRACE_POSTGRESQL_ASET_ALLOC(context, size);
/*
* If requested size exceeds maximum for chunks, allocate an entire block
* for this request.
@@ -623,6 +634,7 @@
set-isReset = false;
AllocAllocInfo(set, chunk);
+ TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize);
return AllocChunkGetPointer(chunk);
}
@@ -771,6 +783,7 @@
errdetail(Failed on request of size %lu.,
(unsigned long) size)));
}
+ TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize);
block-aset = set;
block-freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -830,6 +843,7 @@
AllocChunk chunk = AllocPointerGetChunk(pointer);
AllocFreeInfo(set, chunk);
+ TRACE_POSTGRESQL_ASET_FREE(context, pointer);
#ifdef MEMORY_CONTEXT_CHECKING
/* Test for someone scribbling on unused space in chunk */
@@ -866,6 +880,9 @@
set-blocks = block-next;
else
prevblock-next = block-next;
+
+ TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block,
+ block-endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
/* Wipe freed memory for debugging purposes */
memset(block, 0x7F, block-freeptr - ((char *) block));
@@ -905,6 +922,8 @@
AllocChunk chunk = AllocPointerGetChunk(pointer);
Size oldsize = chunk-size;
+ TRACE_POSTGRESQL_ASET_REALLOC(context, pointer, size, oldsize);
+
#ifdef MEMORY_CONTEXT_CHECKING
/* Test for someone scribbling on unused space in chunk */
if (chunk-requested_size oldsize)
@@ -948,6 +967,7 @@
*/
AllocBlock block = set-blocks;
AllocBlock prevblock = NULL;
+ AllocBlock newblock;
Size chksize;
Size blksize;
@@ -967,7 +987,11 @@
/* Do the realloc */
chksize = MAXALIGN(size);
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
- block = (AllocBlock) realloc(block, blksize);
+ newblock = (AllocBlock) realloc(block, blksize);
+
+ TRACE_POSTGRESQL_ASET_BLOCK_REALLOC(context, block, newblock, blksize);
+ block = newblock;
+
if (block == NULL)
{
MemoryContextStats(TopMemoryContext);
diff -r 68b8827f4738
Re: [HACKERS] [patch] pg_ctl init extension
Robert Haas píše v st 09. 12. 2009 v 08:56 -0500: On Dec 9, 2009, at 8:32 AM, Zdenek Kotala zdenek.kot...@sun.com wrote: snip Ahh, It is somethink what I want to do, but it is not ready yet in this patch, but I already documented it. Idea is to install initdb and postgres into libexecdir and packager can select if libexecdir will be equal bindir or not. The paragraph should be removed at this moment. Shell I send modified patch or does committer remove it before commit? I think Peter claimed this one but as far as I am concerned, I would always rather have an updated patch. OK, here it is. Thanks Zdenek diff -r ab32ed8164e7 doc/src/sgml/config.sgml --- a/doc/src/sgml/config.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/config.sgml Wed Dec 09 15:17:54 2009 +0100 @@ -54,9 +54,10 @@ para One way to set these parameters is to edit the file filenamepostgresql.conf/indextermprimarypostgresql.conf//, -which is normally kept in the data directory. (applicationinitdb/ -installs a default copy there.) An example of what this file might look -like is: +which is normally kept in the data directory. +(link linkend=creating-clusterdatabase cluster initialization/link +process installs a default copy there.) +An example of what this file might look like is: programlisting # This is a comment log_connections = yes @@ -365,8 +366,8 @@ Determines the maximum number of concurrent connections to the database server. The default is typically 100 connections, but might be less if your kernel settings will not support it (as -determined during applicationinitdb/). This parameter can -only be set at server start. +determined during link linkend=creating-clusterdatabase cluster +initialization/link). This parameter can only be set at server start. /para para @@ -747,7 +748,8 @@ Sets the amount of memory the database server uses for shared memory buffers. The default is typically 32 megabytes (literal32MB/), but might be less if your kernel settings will -not support it (as determined during applicationinitdb/). +not support it (as determined during link linkend=creating-clusterdatabase +cluster initialization)/link. This setting must be at least 128 kilobytes. (Non-default values of symbolBLCKSZ/symbol change the minimum.) However, settings significantly higher than the minimum are usually needed @@ -4267,10 +4269,10 @@ keywords literalUS/, literalNonEuro/, and literalNonEuropean/ are synonyms for literalMDY/. See xref linkend=datatype-datetime for more information. The -built-in default is literalISO, MDY/, but -applicationinitdb/application will initialize the -configuration file with a setting that corresponds to the -behavior of the chosen varnamelc_time/varname locale. +built-in default is literalISO, MDY/, but +link linkend=creating-clusterdatabase cluster initialization/link +will initialize the configuration file with a setting that corresponds +to the behavior of the chosen varnamelc_time/varname locale. /para /listitem /varlistentry @@ -4476,9 +4478,9 @@ specifying the configuration. See xref linkend=textsearch for further information. The built-in default is literalpg_catalog.simple/, but -applicationinitdb/application will initialize the -configuration file with a setting that corresponds to the -chosen varnamelc_ctype/varname locale, if a configuration +link linkend=creating-clusterdatabase cluster initialization/link +will initialize the configuration file with a setting that corresponds +to the chosen varnamelc_ctype/varname locale, if a configuration matching that locale can be identified. /para /listitem @@ -5240,8 +5242,9 @@ listitem para Allows modification of the structure of system tables. -This is used by commandinitdb/command. -This parameter can only be set at server start. +This is used by link linkend=creating-clusterdatabase cluster +initialization/link process. This parameter can only be set at server +start. /para /listitem /varlistentry diff -r ab32ed8164e7 doc/src/sgml/manage-ag.sgml --- a/doc/src/sgml/manage-ag.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/manage-ag.sgml Wed Dec 09 15:17:54 2009 +0100 @@ -107,10 +107,9 @@ Since you need to be connected to the database server in order to execute the commandCREATE DATABASE/command command, the question remains how the emphasisfirst/ database at any given - site can be created. The first database is always created by the - commandinitdb/ command when the data storage area
Re: [HACKERS] [patch] executor and slru dtrace probes
Dne 8.12.09 00:27, Bernd Helmle napsal(a): --On 13. November 2009 23:29:41 +0100 Zdenek Kotala zdenek.kot...@sun.com wrote: t contains two DTrace probe groups. One is related to monitoring SLRU and second is about executor nodes. I merged it with the head. Original end of mail thread is here: http://archives.postgresql.org/pgsql-hackers/2009-04/msg00148.php I've started to review this. It seems to me the attached patch wasn't adjusted or discussed again to address Tom's complaints? At least the executor probes contained here hold still the same issues mentioned by Tom in the discussion linked here. I did not make any change. I only revival patch and merge it with head. I think that SLRU probes are OK and acceptable. Tom's issues with executor probes are still there and I expect discussion about them. IIRC Theo uses these probes in production. If you think that it is better I could split patch into two separate patches and both can be reviewed separately. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Greg, thanks for your useful comments. I attached new doc patch version. I removed example changes and add link to create database cluster (I hope) everywhere. Please, look on it and let me know if there is still something what should be changed. Thanks Zdenek Greg Smith píše v ne 06. 12. 2009 v 22:29 -0500: I just noticed that there was an updated patch here that never made its way onto the CommitFest app listing. I just fixed that and took a quick look at it. I was in favor of this code change, but I have to say even I don't really like how it ended up getting documented--and I'm sure there are others would be downright hostile toward it. The biggest problem is that all of the places that used to say commandinitdb when talking about creating a cluster now just say database cluster initialization--with no link to a section covering that topic. That's not a good forward step. The part I'm more favorable toward that I expect other people to really cringe at is that the examples showing how to manually run initdb have all been switched to use pg_ctl initdb too. If we're going to have a smooth transition toward supporting both styles of working in the next release, I think what needs to happen to the documentation here is adding a very clear section saying that initdb and pg_ctl initdb are the same thing, and noting why both forms exist. Maybe a short note in both pg_ctl and initdb pointing at each other; not sure yet what's best. Then update all the current places that say initdb that have been rewritten in this doc patch to database cluster initialization to reference things appropriate still. Going as far as making all the examples exclusively use the pg_ctl form right now is probably more than the community at large wants to handle just yet I suspect. At best, maybe we could make some or all of those either use both forms, or link them to the new discussion of alternatives section. I'm glad we made some progress (and are basically at code complete now) on this patch this round. Given that this patch doesn't have a large amount of support, I think that the remaining work here is fine-tuning the documentation to cover the new option available without introducing and abrupt change people won't like. I'm going to mark this returned with feedback for now since I think that's going to take a bit more work than we really have time for right now, particularly given the limited number of people who care about this change. Zdenek, once this CommitFest clears out, I can help out with getting the documentation parts here smoothed over so this is in proper shape to commit during the next one; I don't think there's anything left you need to do. diff -r ab32ed8164e7 doc/src/sgml/config.sgml --- a/doc/src/sgml/config.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/config.sgml Mon Dec 07 17:12:32 2009 +0100 @@ -54,9 +54,10 @@ para One way to set these parameters is to edit the file filenamepostgresql.conf/indextermprimarypostgresql.conf//, -which is normally kept in the data directory. (applicationinitdb/ -installs a default copy there.) An example of what this file might look -like is: +which is normally kept in the data directory. +(link linkend=creating-clusterdatabase cluster initialization/link +process installs a default copy there.) +An example of what this file might look like is: programlisting # This is a comment log_connections = yes @@ -365,8 +366,8 @@ Determines the maximum number of concurrent connections to the database server. The default is typically 100 connections, but might be less if your kernel settings will not support it (as -determined during applicationinitdb/). This parameter can -only be set at server start. +determined during link linkend=creating-clusterdatabase cluster +initialization/link). This parameter can only be set at server start. /para para @@ -747,7 +748,8 @@ Sets the amount of memory the database server uses for shared memory buffers. The default is typically 32 megabytes (literal32MB/), but might be less if your kernel settings will -not support it (as determined during applicationinitdb/). +not support it (as determined during link linkend=creating-clusterdatabase +cluster initialization)/link. This setting must be at least 128 kilobytes. (Non-default values of symbolBLCKSZ/symbol change the minimum.) However, settings significantly higher than the minimum are usually needed @@ -4267,10 +4269,10 @@ keywords literalUS/, literalNonEuro/, and literalNonEuropean/ are synonyms for literalMDY/. See xref linkend=datatype-datetime for more information. The -built-in default is literalISO, MDY/, but -applicationinitdb/application will
Re: [HACKERS] [patch] pg_ctl init extension
I attached updated patch and doc patch.
Zdenek
Peter Eisentraut píše v so 21. 11. 2009 v 13:19 +0200:
On lör, 2009-11-14 at 14:50 +0100, Zdenek Kotala wrote:
Peter Eisentraut píše v so 14. 11. 2009 v 10:41 +0200:
On tor, 2009-09-17 at 21:43 +0200, Zdenek Kotala wrote:
Attached patch extends pg_ctl command with init option.
pg_ctl -D /var/lib/postgres [-s] init
This should replace usage of initdb command which has problematic name
as we already discussed several times. Initdb binary will be still
there, but it can be renamed and move into execlib dir in the future.
Patch does not contains documentation changes. They will depends on
decision which database initialization method will be preferred.
OK, let's see. The patch is pretty straightforward, but does anyone
else actually want this? Comments?
Maybe we could ask on general where is more admins. I will send voting
email.
I think this is over now. There was some support, some don't care, but
could make sense, and no one violently objecting, so please finish the
patch up with documentation, and it can go in as far as I'm concerned.
Someone was proposing that pg_ctl initdb be an alias to pg_ctl init.
Perhaps you could add that.
diff -r 2d87758e836b src/bin/pg_ctl/pg_ctl.c
--- a/src/bin/pg_ctl/pg_ctl.c Sun Nov 22 22:06:30 2009 +
+++ b/src/bin/pg_ctl/pg_ctl.c Fri Dec 04 22:13:28 2009 +0100
@@ -57,6 +57,7 @@
typedef enum
{
NO_COMMAND = 0,
+ INIT_COMMAND,
START_COMMAND,
STOP_COMMAND,
RESTART_COMMAND,
@@ -100,6 +101,7 @@
static void do_help(void);
static void set_mode(char *modeopt);
static void set_sig(char *signame);
+static void do_init(void);
static void do_start(void);
static void do_stop(void);
static void do_restart(void);
@@ -615,6 +617,38 @@
}
static void
+do_init(void)
+{
+ char pg_initdb[MAXPGPATH];
+ char cmd[MAXPGPATH];
+ int ret;
+
+ if ((ret = find_other_exec(argv0, initdb, initdb (PostgreSQL) PG_VERSION \n,
+ pg_initdb)) 0)
+ {
+ write_stderr(_(%s: could not find initdb\n),
+ progname);
+ exit(1);
+ }
+
+ if (post_opts == NULL)
+ post_opts = ;
+
+ if (!silent_mode)
+ snprintf(cmd, MAXPGPATH, SYSTEMQUOTE \%s\ %s%s SYSTEMQUOTE,
+ pg_initdb, pgdata_opt, post_opts);
+ else
+ snprintf(cmd, MAXPGPATH, SYSTEMQUOTE \%s\ %s%s \%s\ SYSTEMQUOTE,
+ pg_initdb, pgdata_opt, post_opts, DEVNULL);
+
+ if ( system(cmd) != 0 )
+ {
+ write_stderr(_(%s: database initialization failed.\n), progname);
+ exit(1);
+ }
+}
+
+static void
do_start(void)
{
pgpid_t pid;
@@ -1536,6 +1570,7 @@
printf(_(%s is a utility to start, stop, restart, reload configuration files,\n
report the status of a PostgreSQL server, or signal a PostgreSQL process.\n\n), progname);
printf(_(Usage:\n));
+ printf(_( %s init[db] [-D DATADIR] [-s] [-o \OPTIONS\]\n), progname);
printf(_( %s start [-w] [-t SECS] [-D DATADIR] [-s] [-l FILENAME] [-o \OPTIONS\]\n), progname);
printf(_( %s stop[-W] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n), progname);
printf(_( %s restart [-w] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n
@@ -1568,7 +1603,7 @@
#endif
printf(_( -l, --log FILENAME write (or append) server log to FILENAME\n));
printf(_( -o OPTIONS command line options to pass to postgres\n
- (PostgreSQL server executable)\n));
+ (PostgreSQL server executable) or initdb\n));
printf(_( -p PATH-TO-POSTGRESnormally not necessary\n));
printf(_(\nOptions for stop or restart:\n));
printf(_( -m SHUTDOWN-MODE can be \smart\, \fast\, or \immediate\\n));
@@ -1825,7 +1860,11 @@
exit(1);
}
- if (strcmp(argv[optind], start) == 0)
+ if (strcmp(argv[optind], init) == 0)
+ctl_command = INIT_COMMAND;
+ else if (strcmp(argv[optind], initdb) == 0)
+ctl_command = INIT_COMMAND;
+ else if (strcmp(argv[optind], start) == 0)
ctl_command = START_COMMAND;
else if (strcmp(argv[optind], stop) == 0)
ctl_command = STOP_COMMAND;
@@ -1922,6 +1961,9 @@
switch (ctl_command)
{
+ case INIT_COMMAND:
+ do_init();
+ break;
case STATUS_COMMAND:
do_status();
break;
diff -r 5af127be4a67 doc/src/sgml/config.sgml
--- a/doc/src/sgml/config.sgml Mon Nov 23 09:06:21 2009 +0100
+++ b/doc/src/sgml/config.sgml Fri Dec 04 21:04:34 2009 +0100
@@ -114,8 +52,8 @@
para
One way to set these parameters is to edit the file
filenamepostgresql.conf/indextermprimarypostgresql.conf//,
-which is normally kept in the data directory. (applicationinitdb/
-installs a default copy there.) An example of what this file might look
+which is normally kept in the data directory. (database cluster initialization
+process installs a default copy there.) An example of what this file might look
like is:
programlisting
# This is a comment
@@ -425,7 +363,7 @@
Determines the maximum number
Re: [HACKERS] [PATCH] Add solaris path for docbook COLLATEINDEX
Bruce Momjian píše v po 30. 11. 2009 v 12:32 -0500: Zdenek Kotala wrote: collateindex.pl is stored in /usr/share/sgml/docbook/. Attached fix modify docbook.m4 to find correct path. It would be nice also backported the fix back at least to 8.2. I am not happy looking in a directory _above_ a specified directory by default: [$DOCBOOKSTYLE/bin $DOCBOOKSTYLE/.. $PATH]) That seems possibly unsafe. I suggest you just add it to the PATH for Solaris builds. I'm not sure if it is unsafer that searching in $PATH. Anyway I don't think that your proposed solution is good way. I'm able to do it but how many other people can? Who know where collateindex.pl stays on Solaris? configure is here to make live easier for people. If we know how to do it automatically we should do it. If you think that $DOCBOOKSTYLE/.. is not good than I propose to use /usr/share/sgml/docbook/ directly. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Add solaris path for docbook COLLATEINDEX
Peter Eisentraut píše v po 30. 11. 2009 v 21:27 +0200: On mån, 2009-11-30 at 19:53 +0100, Zdenek Kotala wrote: Bruce Momjian píše v po 30. 11. 2009 v 12:32 -0500: I am not happy looking in a directory _above_ a specified directory by default: [$DOCBOOKSTYLE/bin $DOCBOOKSTYLE/.. $PATH]) That seems possibly unsafe. I suggest you just add it to the PATH for Solaris builds. I'm not sure if it is unsafer that searching in $PATH. Anyway I don't think that your proposed solution is good way. I'm able to do it but how many other people can? Who know where collateindex.pl stays on Solaris? configure is here to make live easier for people. If we know how to do it automatically we should do it. Note that $DOCBOOKSTYLE/bin corresponds to the location of the file in an unpacked source archive and $PATH is of course where programs normally go. The $DOCBOOKSTYLE/.. location makes no sense at all, because that basically says that the installer intentionally moved the file, but to a completely nonstandard location. I'm not sgml//docbook guru. Do you think that Solaris location of collateindex.pl is wrong? Does exist any recommendation for this? I could log a bug, but I need some link with recommendation. If you think that $DOCBOOKSTYLE/.. is not good than I propose to use /usr/share/sgml/docbook/ directly. That would be the less ugly solution. Patch attached. thanks Zdenek diff -r 2d87758e836b config/docbook.m4 --- a/config/docbook.m4 Sun Nov 22 22:06:30 2009 + +++ b/config/docbook.m4 Mon Nov 30 20:55:15 2009 +0100 @@ -93,7 +93,7 @@ [AC_REQUIRE([PGAC_PATH_DOCBOOK_STYLESHEETS])dnl if test -n $DOCBOOKSTYLE; then AC_PATH_PROGS(COLLATEINDEX, collateindex.pl, [], -[$DOCBOOKSTYLE/bin $PATH]) +[$DOCBOOKSTYLE/bin /usr/share/sgml/docbook $PATH]) else AC_PATH_PROGS(COLLATEINDEX, collateindex.pl) fi])# PGAC_PATH_COLLATEINDEX -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] Add solaris path for docbook COLLATEINDEX
collateindex.pl is stored in /usr/share/sgml/docbook/. Attached fix modify docbook.m4 to find correct path. It would be nice also backported the fix back at least to 8.2. Thanks Zdenek diff -r 2d87758e836b config/docbook.m4 --- a/config/docbook.m4 Sun Nov 22 22:06:30 2009 + +++ b/config/docbook.m4 Wed Nov 25 21:27:09 2009 +0100 @@ -93,7 +93,7 @@ [AC_REQUIRE([PGAC_PATH_DOCBOOK_STYLESHEETS])dnl if test -n $DOCBOOKSTYLE; then AC_PATH_PROGS(COLLATEINDEX, collateindex.pl, [], -[$DOCBOOKSTYLE/bin $PATH]) +[$DOCBOOKSTYLE/bin $DOCBOOKSTYLE/.. $PATH]) else AC_PATH_PROGS(COLLATEINDEX, collateindex.pl) fi])# PGAC_PATH_COLLATEINDEX -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Peter Eisentraut píše v so 14. 11. 2009 v 10:41 +0200: On tor, 2009-09-17 at 21:43 +0200, Zdenek Kotala wrote: Attached patch extends pg_ctl command with init option. pg_ctl -D /var/lib/postgres [-s] init This should replace usage of initdb command which has problematic name as we already discussed several times. Initdb binary will be still there, but it can be renamed and move into execlib dir in the future. Patch does not contains documentation changes. They will depends on decision which database initialization method will be preferred. OK, let's see. The patch is pretty straightforward, but does anyone else actually want this? Comments? Maybe we could ask on general where is more admins. I will send voting email. thanks for looking on this Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] DTrace compiler warnings
Hmm, const is also problem on solaris. dtrace generates probe.h file and eats const. It generates following noise on solaris build: postgres.c, line 554: warning: argument #1 is incompatible with prototype: prototype: pointer to char : ../../../src/include/utils/probes.h, line 582 argument : pointer to const char IIRC, it was discussed. I cc Robert. He should know answer why const is ignored. Zdenek Bernd Helmle píše v so 14. 11. 2009 v 14:54 +0100: I'm just seeing these kind of compiler warnings in current HEAD: pgstat.c: In function ‘pgstat_report_activity’: pgstat.c:2276: warning: passing argument 1 of ‘__dtrace_probe$postgresql$statement__status$v1$63686172202a’ discards qualifiers from pointer target type There are more of them (e.g. postgres.c), all passing a const char pointer. Platform is Snow Leopard, 10.6.2, gcc 4.2.1 -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [Patch] Fix enum type mismatch
Attached patch fixed following warning: ../../../src/include/nodes/parsenodes.h, line 487: warning: enumerator value overflows INT_MAX (2147483647) The reason is clear, enum is int not unsigned. It is short fix, but I'm thinking about enum conversion to #define. We use e.g. in the same file. 60 typedef uint32 AclMode; /* a bitmask of privilege bits */ 61 62 #define ACL_INSERT (10) /* for relations */ 63 #define ACL_SELECT (11) 64 #define ACL_UPDATE (12) 65 #define ACL_DELETE (13) 66 #define ACL_TRUNCATE(14) 67 #define ACL_REFERENCES (15) 68 #define ACL_TRIGGER (16) Zdenek diff -r 68b8827f4738 src/include/nodes/parsenodes.h --- a/src/include/nodes/parsenodes.h Fri Nov 13 11:17:04 2009 + +++ b/src/include/nodes/parsenodes.h Fri Nov 13 16:21:38 2009 +0100 @@ -484,7 +484,7 @@ CREATE_TABLE_LIKE_INDEXES = 1 2, CREATE_TABLE_LIKE_STORAGE = 1 3, CREATE_TABLE_LIKE_COMMENTS = 1 4, - CREATE_TABLE_LIKE_ALL = 0x + CREATE_TABLE_LIKE_ALL = 0x7FFF /* enum is int */ } CreateStmtLikeOption; /* -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] dtrace probes for memory manager
Attached patch contains new dtrace probes for memory management. Main
purpose is to analyze memory footprint - for example how many memory
needs transaction, peak memory per context, when memory block is reused
or when it is allocate by malloc and so on.
There are three groups of probes:
1) general memory context operation:
mcxt-alloc
mcxt-create
mcxt-delete
mcxt-free
mcxt-realloc
mcxt-reset
2) AllocSet operations (called from mcxt)
aset-alloc
aset-delete
aset-free
aset-realloc
aset-reset
3) AllocSet Block operations.
aset-block-free
aset-block-new
aset-block-realloc
aset-block-reset
Zdenek
diff -r 68b8827f4738 src/backend/utils/mmgr/aset.c
--- a/src/backend/utils/mmgr/aset.c Fri Nov 13 11:17:04 2009 +
+++ b/src/backend/utils/mmgr/aset.c Fri Nov 13 21:24:56 2009 +0100
@@ -64,6 +64,7 @@
#include postgres.h
+#include pg_trace.h
#include utils/memutils.h
/* Define this to detail debug alloc information */
@@ -463,6 +464,8 @@
AssertArg(AllocSetIsValid(set));
+ TRACE_POSTGRESQL_ASET_RESET(context);
+
/* Nothing to do if no pallocs since startup or last reset */
if (set-isReset)
return;
@@ -495,6 +498,8 @@
#endif
block-freeptr = datastart;
block-next = NULL;
+ TRACE_POSTGRESQL_ASET_BLOCK_RESET(context, block,
+block-endptr - ((char *) block));
}
else
{
@@ -503,6 +508,8 @@
/* Wipe freed memory for debugging purposes */
memset(block, 0x7F, block-freeptr - ((char *) block));
#endif
+ TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block,
+block-endptr - ((char *) block));
free(block);
}
block = next;
@@ -529,6 +536,7 @@
AllocBlock block = set-blocks;
AssertArg(AllocSetIsValid(set));
+ TRACE_POSTGRESQL_ASET_DELETE(context);
#ifdef MEMORY_CONTEXT_CHECKING
/* Check for corruption and leaks before freeing */
@@ -548,6 +556,8 @@
/* Wipe freed memory for debugging purposes */
memset(block, 0x7F, block-freeptr - ((char *) block));
#endif
+ TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block,
+ block-endptr - ((char *) block));
free(block);
block = next;
}
@@ -570,6 +580,7 @@
AssertArg(AllocSetIsValid(set));
+ TRACE_POSTGRESQL_ASET_ALLOC(context, size);
/*
* If requested size exceeds maximum for chunks, allocate an entire block
* for this request.
@@ -623,6 +634,7 @@
set-isReset = false;
AllocAllocInfo(set, chunk);
+ TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize);
return AllocChunkGetPointer(chunk);
}
@@ -771,6 +783,7 @@
errdetail(Failed on request of size %lu.,
(unsigned long) size)));
}
+ TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize);
block-aset = set;
block-freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -830,6 +843,7 @@
AllocChunk chunk = AllocPointerGetChunk(pointer);
AllocFreeInfo(set, chunk);
+ TRACE_POSTGRESQL_ASET_FREE(context, pointer);
#ifdef MEMORY_CONTEXT_CHECKING
/* Test for someone scribbling on unused space in chunk */
@@ -866,6 +880,9 @@
set-blocks = block-next;
else
prevblock-next = block-next;
+
+ TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block,
+ block-endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
/* Wipe freed memory for debugging purposes */
memset(block, 0x7F, block-freeptr - ((char *) block));
@@ -905,6 +922,8 @@
AllocChunk chunk = AllocPointerGetChunk(pointer);
Size oldsize = chunk-size;
+ TRACE_POSTGRESQL_ASET_REALLOC(context, pointer, size, oldsize);
+
#ifdef MEMORY_CONTEXT_CHECKING
/* Test for someone scribbling on unused space in chunk */
if (chunk-requested_size oldsize)
@@ -948,6 +967,7 @@
*/
AllocBlock block = set-blocks;
AllocBlock prevblock = NULL;
+ AllocBlock newblock;
Size chksize;
Size blksize;
@@ -967,7 +987,11 @@
/* Do the realloc */
chksize = MAXALIGN(size);
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
- block = (AllocBlock) realloc(block, blksize);
+ newblock = (AllocBlock) realloc(block, blksize);
+
+ TRACE_POSTGRESQL_ASET_BLOCK_REALLOC(context, block, newblock, blksize);
+ block = newblock;
+
if (block == NULL)
{
MemoryContextStats(TopMemoryContext);
diff -r 68b8827f4738 src/backend/utils/mmgr/mcxt.c
--- a/src/backend/utils/mmgr/mcxt.c Fri Nov 13 11:17:04 2009 +
+++ b/src/backend/utils/mmgr/mcxt.c Fri Nov 13 21:24:56 2009 +0100
@@ -22,7 +22,7 @@
#include postgres.h
#include utils/memutils.h
-
+#include pg_trace.h
/*
* GLOBAL MEMORY *
@@ -123,6 +123,8 @@
{
AssertArg(MemoryContextIsValid(context));
+ TRACE_POSTGRESQL_MCXT_RESET(context-name, context);
+
/* save a function call in common case where there are no children */
if (context-firstchild != NULL)
MemoryContextResetChildren(context);
@@ -168,6 +170,8 @@
MemoryContextDeleteChildren(context);
+ TRACE_POSTGRESQL_MCXT_DELETE(context-name, context, context-parent);
+
/*
* We delink the
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 13. 11. 2009 v 16:06 -0500: Zdenek Kotala zdenek.kot...@sun.com writes: Attached patch contains new dtrace probes for memory management. This is a bad idea and I want to reject it outright. No ordinary user is really going to care about those details, and palloc is a sufficiently hot hot-spot that even the allegedly negligible overhead of an inactive dtrace probe is going to cost us. I don't think that impact is too big here. User space DTrace probes are implemented like function call. When probe is inactive call is replaced by nop. Only what stay in code is arguments preparations. there is asm code 32bit intel from palloc: MemoryContextAlloc+0xae:pushl $0x0 MemoryContextAlloc+0xb0:pushl -0x8(%ebx) MemoryContextAlloc+0xb3:pushl 0xc(%ebp) MemoryContextAlloc+0xb6:movl 0x8(%ebp),%eax MemoryContextAlloc+0xb9:pushl %eax MemoryContextAlloc+0xba:pushl 0x14(%eax) MemoryContextAlloc+0xbd:nop MemoryContextAlloc+0xbe:nop MemoryContextAlloc+0xbf:nop MemoryContextAlloc+0xc0:nop MemoryContextAlloc+0xc1:nop MemoryContextAlloc+0xc2:addl $0x20,%esp You can see is that overhead depends on number of argument. I used five arguments now but two can be enough. Only dtrace script will be complicated in some cases after that. By my opinion if you compare whole palloc code path, Dtrace probes overhead is minimal. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Alvaro Herrera píše v pá 13. 11. 2009 v 18:34 -0300: Zdenek Kotala wrote: Attached patch contains new dtrace probes for memory management. Main purpose is to analyze memory footprint - for example how many memory needs transaction, peak memory per context, when memory block is reused or when it is allocate by malloc and so on. Having had to instrument these to figure out some problems, I'd give this patch a +1. However, the performance argument is compelling. As a compromise, maybe we could have a #define that needs to be turned on at compile time to enable these probes; so a regular dtrace-enabled build would not have them, but if you really needed to analyze memory allocations, you could recompile to turn them on. But point of dtrace probes is that they are here without recompilation :(. Do we have any test which we could use for performance penalty testing? I don't think that overhead is significant. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Alvaro Herrera píše v pá 13. 11. 2009 v 18:34 -0300: Zdenek Kotala wrote: Attached patch contains new dtrace probes for memory management. Main purpose is to analyze memory footprint - for example how many memory needs transaction, peak memory per context, when memory block is reused or when it is allocate by malloc and so on. Having had to instrument these to figure out some problems, I'd give this patch a +1. However, the performance argument is compelling. As a compromise, maybe we could have a #define that needs to be turned on at compile time to enable these probes; so a regular dtrace-enabled build would not have them, but if you really needed to analyze memory allocations, you could recompile to turn them on. The another idea is to have two AllocSet functions set. One without and one with dtrace probes. And switch it by GUC flag for example. From mcxt I create and delete context is important, rest can be taken from alloc set probes. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [patch] executor and slru dtrace probes
I attached patch which was already sent on february/april, but it was
lost in time. It is originally from Robert Lor and Theo Schlossnagle.
It contains two DTrace probe groups. One is related to monitoring SLRU
and second is about executor nodes.
I merged it with the head.
Original end of mail thread is here:
http://archives.postgresql.org/pgsql-hackers/2009-04/msg00148.php
Zdenek
diff -r 68b8827f4738 src/backend/access/transam/slru.c
--- a/src/backend/access/transam/slru.c Fri Nov 13 11:17:04 2009 +
+++ b/src/backend/access/transam/slru.c Fri Nov 13 23:29:14 2009 +0100
@@ -57,6 +57,7 @@
#include storage/fd.h
#include storage/shmem.h
#include miscadmin.h
+#include pg_trace.h
/*
@@ -372,6 +373,7 @@
{
SlruShared shared = ctl-shared;
+ TRACE_POSTGRESQL_SLRU_READPAGE_START((uintptr_t)ctl, pageno, write_ok, xid);
/* Outer loop handles restart if we must wait for someone else's I/O */
for (;;)
{
@@ -399,6 +401,7 @@
}
/* Otherwise, it's ready to use */
SlruRecentlyUsed(shared, slotno);
+ TRACE_POSTGRESQL_SLRU_READPAGE_DONE(slotno);
return slotno;
}
@@ -446,6 +449,7 @@
SlruReportIOError(ctl, pageno, xid);
SlruRecentlyUsed(shared, slotno);
+ TRACE_POSTGRESQL_SLRU_READPAGE_DONE(slotno);
return slotno;
}
}
@@ -470,6 +474,8 @@
SlruShared shared = ctl-shared;
int slotno;
+ TRACE_POSTGRESQL_SLRU_READPAGE_READONLY((uintptr_t)ctl, pageno, xid);
+
/* Try to find the page while holding only shared lock */
LWLockAcquire(shared-ControlLock, LW_SHARED);
@@ -511,6 +517,8 @@
int pageno = shared-page_number[slotno];
bool ok;
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_START((uintptr_t)ctl, pageno, slotno);
+
/* If a write is in progress, wait for it to finish */
while (shared-page_status[slotno] == SLRU_PAGE_WRITE_IN_PROGRESS
shared-page_number[slotno] == pageno)
@@ -525,7 +533,10 @@
if (!shared-page_dirty[slotno] ||
shared-page_status[slotno] != SLRU_PAGE_VALID ||
shared-page_number[slotno] != pageno)
+ {
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_DONE();
return;
+ }
/*
* Mark the slot write-busy, and clear the dirtybit. After this point, a
@@ -569,6 +580,8 @@
/* Now it's okay to ereport if we failed */
if (!ok)
SlruReportIOError(ctl, pageno, InvalidTransactionId);
+
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_DONE();
}
/*
@@ -593,6 +606,8 @@
SlruFileName(ctl, path, segno);
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_START((uintptr_t)ctl, path, pageno, slotno);
+
/*
* In a crash-and-restart situation, it's possible for us to receive
* commands to set the commit status of transactions whose bits are in
@@ -607,6 +622,7 @@
{
slru_errcause = SLRU_OPEN_FAILED;
slru_errno = errno;
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -614,6 +630,7 @@
(errmsg(file \%s\ doesn't exist, reading as zeroes,
path)));
MemSet(shared-page_buffer[slotno], 0, BLCKSZ);
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(true, -1, -1);
return true;
}
@@ -622,6 +639,7 @@
slru_errcause = SLRU_SEEK_FAILED;
slru_errno = errno;
close(fd);
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -631,6 +649,7 @@
slru_errcause = SLRU_READ_FAILED;
slru_errno = errno;
close(fd);
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -638,9 +657,12 @@
{
slru_errcause = SLRU_CLOSE_FAILED;
slru_errno = errno;
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
+ TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(true, -1, -1);
+
return true;
}
@@ -668,6 +690,8 @@
char path[MAXPGPATH];
int fd = -1;
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_START((uintptr_t)ctl, pageno, slotno);
+
/*
* Honor the write-WAL-before-data rule, if appropriate, so that we do not
* write out data before associated WAL records. This is the same action
@@ -753,6 +777,7 @@
{
slru_errcause = SLRU_OPEN_FAILED;
slru_errno = errno;
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -781,6 +806,7 @@
slru_errno = errno;
if (!fdata)
close(fd);
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -794,6 +820,7 @@
slru_errno = errno;
if (!fdata)
close(fd);
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -808,6 +835,7 @@
slru_errcause = SLRU_FSYNC_FAILED;
slru_errno = errno;
close(fd);
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
return false;
}
@@ -815,10 +843,12 @@
{
slru_errcause = SLRU_CLOSE_FAILED;
slru_errno = errno;
+ TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno);
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
u235sentinel píše v út 20. 10. 2009 v 12:22 -0600: Now I'm running and will add a few more things in like readline. The goal is to build plr and plperl and load them into postgres. Hmm, I'm afraid that 64bit plperl is a problem. Solaris 10 is not shipped with 64bit perl :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
u235sentinel píše v ne 18. 10. 2009 v 17:50 -0600: Are you sure about this? When I try to build and don't have openssl in the lib/include path it claims it needs it. As I'm building 64 bit I can now build postgres in 64 bit with openssl 98k just fine. However when I run it I'm getting the same error message. If you want to link against to builtin OpenSSL you need following setup: ./configure ... --with-openssl --with-includes=/usr/sfw/include --with-libs=/usr/lib/amd64:/usr/sfw/lib/amd64 and important is: LD_OPTIONS=-R/usr/sfw/lib/amd64 -L/usr/sfw/lib/amd64 Or if you don't need own compilation, you can use built-in PostgreSQL 8.3. It is located in /usr/postgres/8.3/bin or /usr/postgres/8.3/bin/64. See man postgres_83 for details. Also you need to apply last patch 138827-05: http://sunsolve.sun.com/search/document.do?assetkey=1-21-138827-05-1 Or if you still needs own compilation try to compile openssl 98k with Sun Studio. Or if you cannot compile it with Sun Studio, you can try -mimpure-text gcc switch to compile OpenSSL. It is workaround for some kind of linking issues. Let me know it it helps Zdenek I'm curious if this is a lost hope. My boss is recommending we flatten the Sun box and install redhat linux (which I'm fine with). I'd rather not as threading in Solaris is better. Thoughts? thanks Zdenek Kotala wrote: You can look on http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=ghost_mothdt=2009-10-07%2021:06:00 How it is built. You also does not needed own version of Openssl. All security fixes are backported. It is located in /usr/sfw/lib or /usr/sfw/lib/64 Sometimes are problem with gcc and solaris linker. IIRC, I had problem with PLPerl compilation. Zdenek Dne 8.10.09 03:48, u235sentinel napsal(a): So I compiled postgres with Solaris 10 and have problems running it. # ./pg_ctl ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file /usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value 0xfd7fff1cf210 does not fit Killed # ldd pg_ctl libpq.so.5 =/usr/local/postgres64/lib/libpq.so.5 libm.so.2 = /usr/lib/64/libm.so.2 libxml2.so.2 = /usr/lib/64/libxml2.so.2 libz.so.1 = /usr/lib/64/libz.so.1 libreadline.so.6 = /usr/local/lib/libreadline.so.6 libcurses.so.1 =/usr/lib/64/libcurses.so.1 librt.so.1 =/usr/lib/64/librt.so.1 libsocket.so.1 =/usr/lib/64/libsocket.so.1 libc.so.1 = /usr/lib/64/libc.so.1 libpthread.so.1 = /usr/lib/64/libpthread.so.1 libnsl.so.1 = /lib/64/libnsl.so.1 libgcc_s.so.1 = /usr/sfw/lib/amd64/libgcc_s.so.1 libaio.so.1 = /lib/64/libaio.so.1 libmd.so.1 =/lib/64/libmd.so.1 libmp.so.2 =/lib/64/libmp.so.2 libscf.so.1 = /lib/64/libscf.so.1 libdoor.so.1 = /lib/64/libdoor.so.1 libuutil.so.1 = /lib/64/libuutil.so.1 libgen.so.1 = /lib/64/libgen.so.1 # file /usr/local/postgres64/lib/libpq.so.5 /usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib AMD64 Version 1 [SSE CMOV], dynamically linked, not stripped What am I missing??? Here's my environment. Solaris 10 x86_64 with postgres 8.3.8 and openssl 98k using gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) , sunstudio12.1 and GNU Make 3.80 I've even monkied with LD_LIBRARY_PATH but getting the same issues. Seems when I don't compile in openssl everything is fine. Thanks! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
Andrew Chernow píše v ne 18. 10. 2009 v 21:09 -0400: I'm curious if this is a lost hope. My boss is recommending we flatten the Sun box and install redhat linux (which I'm fine with). I'd rather not as threading in Solaris is better. Maybe solaris threads were better 10-15 years ago, but I'm not convinced that is still the case. Any data supporting that argument, solaris 10 threads vs. linux 2.6.11+ kernel (p)threads? I can point on this article: http://tweakers.net/reviews/649/all/database-test-sun-ultrasparc-t1-vs-punt-amd-opteron.html Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
Andrew Chernow píše v po 19. 10. 2009 v 14:14 -0400: # ./pg_ctl ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file /usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value 0xfd7fff1cf210 does not fit Killed symbol (unknown). Can you turn on debugging symbols? Knowing the symbol may point to a library that was not compiled properly. Also you can try to run LD_DEBUG=basic ./pg_ctl and also elfdump my local lib | grep R_AMD64_32 it should show when symbols came from. By theway what S10 version do you use? ld -V and uname -a also helps. So I run 'ldd pg_ctl' to see if everything is linking ok. And I'm wondering if there is a problem with libpq.so.5 as mentioned in the original error # file /usr/local/postgres64/lib/libpq.so.5 /usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib AMD64 Version 1 [SSE CMOV], dynamically linked, not stripped Ok. So looking good. Maybe there is a library or header libpq needs that I'm missing in 64 bit? # ldd /usr/local/postgres64/lib/libpq.so.5 Are you sure that all pg_ctl referenced libraries and all libpq.so referenced libraries were built as 64-bit using PIC? Are you linking with any static library that may contain 32-bit objects? That error is most commonly PIC or arch-mismatch. Agree, I went through linker bugs and missing PIC is often root cause of this problem. See http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6261066 Problem was that ./configure badly setup PIC switch on amd64 platform. Please, could you compile pure postgreSQL without other own libraries like readline and openssl? It should help to find which library is culprit. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
You can look on http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=ghost_mothdt=2009-10-07%2021:06:00 How it is built. You also does not needed own version of Openssl. All security fixes are backported. It is located in /usr/sfw/lib or /usr/sfw/lib/64 Sometimes are problem with gcc and solaris linker. IIRC, I had problem with PLPerl compilation. Zdenek Dne 8.10.09 03:48, u235sentinel napsal(a): So I compiled postgres with Solaris 10 and have problems running it. # ./pg_ctl ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file /usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value 0xfd7fff1cf210 does not fit Killed # ldd pg_ctl libpq.so.5 =/usr/local/postgres64/lib/libpq.so.5 libm.so.2 = /usr/lib/64/libm.so.2 libxml2.so.2 = /usr/lib/64/libxml2.so.2 libz.so.1 = /usr/lib/64/libz.so.1 libreadline.so.6 = /usr/local/lib/libreadline.so.6 libcurses.so.1 =/usr/lib/64/libcurses.so.1 librt.so.1 =/usr/lib/64/librt.so.1 libsocket.so.1 =/usr/lib/64/libsocket.so.1 libc.so.1 = /usr/lib/64/libc.so.1 libpthread.so.1 = /usr/lib/64/libpthread.so.1 libnsl.so.1 = /lib/64/libnsl.so.1 libgcc_s.so.1 = /usr/sfw/lib/amd64/libgcc_s.so.1 libaio.so.1 = /lib/64/libaio.so.1 libmd.so.1 =/lib/64/libmd.so.1 libmp.so.2 =/lib/64/libmp.so.2 libscf.so.1 = /lib/64/libscf.so.1 libdoor.so.1 = /lib/64/libdoor.so.1 libuutil.so.1 = /lib/64/libuutil.so.1 libgen.so.1 = /lib/64/libgen.so.1 # file /usr/local/postgres64/lib/libpq.so.5 /usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib AMD64 Version 1 [SSE CMOV], dynamically linked, not stripped What am I missing??? Here's my environment. Solaris 10 x86_64 with postgres 8.3.8 and openssl 98k using gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) , sunstudio12.1 and GNU Make 3.80 I've even monkied with LD_LIBRARY_PATH but getting the same issues. Seems when I don't compile in openssl everything is fine. Thanks! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstore crasesh on 64bit Sparc
Tom Lane píše v čt 01. 10. 2009 v 12:28 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: I'm looking why cometh_month fails and it is problem with last hstore putback: I think this is the same 64-bit problem we fixed last night --- wait for the next rebuild before worrying. Correct, last build seems OK. There only remains issue with plpython UTF8 test which need some investigation. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] hstore crasesh on 64bit Sparc
I'm looking why cometh_month fails and it is problem with last hstore putback: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_mothdt=2009-09-30%2021:06:00 Stack trace is following: 6f013828 hstore_hash (7fffa050, 1395298, 1, 41, 7370616365414143, 0) + c0 0001005f396c FunctionCall1 (100b9ca10, 1395238, 7fffa4cf, 100ba0b48, 43, 100ba0b44) + 4c 0001002e8264 TupleHashTableHash (7fffa6f0, 8, 100b9f9e8, 1, 7fffa6ef, 0) + 15c 0001005fa734 hash_search (100ba20f8, 7fffa6f0, 1, 7fffa6ef, 0, 100ba4191) + 2c 0001002e7e74 LookupTupleHashEntry (100ba00e8, 100b9ae90, 7fffa7ef, 0, 1, 100b96b40) + 114 000100308420 lookup_hash_entry (100b9a978, 100b9b588, 0, 0, 0, 0) + 118 status: process terminated by SIGSEGV (Segmentation Fault), addr=500ba410c It is not assert problem but Segmentation Fault. 32bit compilation works fine (ghost moth). Any idea? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Peter Eisentraut píše v čt 17. 09. 2009 v 23:00 +0300: On tor, 2009-09-17 at 21:43 +0200, Zdenek Kotala wrote: Attached patch extends pg_ctl command with init option. pg_ctl -D /var/lib/postgres [-s] init This should replace usage of initdb command which has problematic name as we already discussed several times. Initdb binary will be still there, but it can be renamed and move into execlib dir in the future. If the name is a problem, why not change the name? What you are proposing above is effectively a very elaborate name change, so why not go for a simpler one? The idea is to have one command for server control. By my opinion init logically belongs to group of command like start/stop. It is also possible to add parameter for init+start in one command and so on. If you look on ZFS you have only two commands to manage everything, it is easy and you can start to use ZFS very quickly. I think this patch increase usability/adoption od postgreSQL for newbies. And second big advantage is that it would be possible easily extend pg_ctl to cope with different postgresql versions (e.g. pg_ctl -v 8.2 -D . init). There is no reason why pg_ctl couldn't start different postgreSQL version depends on PG_VERSION in data directory. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [patch] pg_ctl init extension
Attached patch extends pg_ctl command with init option.
pg_ctl -D /var/lib/postgres [-s] init
This should replace usage of initdb command which has problematic name
as we already discussed several times. Initdb binary will be still
there, but it can be renamed and move into execlib dir in the future.
Patch does not contains documentation changes. They will depends on
decision which database initialization method will be preferred.
Zdenek
*** pgsql_init.8d83e5030d44/src/bin/pg_ctl/pg_ctl.c 2009-09-17 21:42:20.865268360 +0200
--- /export/home/zk200664/work/mercurial/pgsql_init/src/bin/pg_ctl/pg_ctl.c 2009-09-17 21:15:04.630265322 +0200
***
*** 57,62
--- 57,63
typedef enum
{
NO_COMMAND = 0,
+ INIT_COMMAND,
START_COMMAND,
STOP_COMMAND,
RESTART_COMMAND,
***
*** 100,105
--- 101,107
static void do_help(void);
static void set_mode(char *modeopt);
static void set_sig(char *signame);
+ static void do_init(void);
static void do_start(void);
static void do_stop(void);
static void do_restart(void);
***
*** 615,620
--- 617,655
}
static void
+ do_init(void)
+ {
+ char pg_initdb[MAXPGPATH];
+ char cmd[MAXPGPATH];
+ int ret;
+
+ if ((ret = find_other_exec(argv0, initdb, initdb (PostgreSQL) PG_VERSION \n,
+ pg_initdb)) 0)
+
+ {
+ write_stderr(_(%s: could not find initdb\n),
+ progname);
+ exit(1);
+ }
+
+ if (post_opts == NULL)
+ post_opts = ;
+
+ if (!silent_mode)
+ snprintf(cmd, MAXPGPATH, SYSTEMQUOTE \%s\ %s%s SYSTEMQUOTE,
+ pg_initdb, pgdata_opt, post_opts);
+ else
+ snprintf(cmd, MAXPGPATH, SYSTEMQUOTE \%s\ %s%s \%s\ SYSTEMQUOTE,
+ pg_initdb, pgdata_opt, post_opts, DEVNULL);
+
+ if ( system(cmd) != 0 )
+ {
+ write_stderr(_(%s: database initialization failed.\n), progname);
+ exit(1);
+ }
+ }
+
+ static void
do_start(void)
{
pgpid_t pid;
***
*** 694,700
progname, exitcode);
exit(1);
}
-
if (old_pid != 0)
{
pg_usleep(100);
--- 729,734
***
*** 1535,1540
--- 1569,1575
printf(_(%s is a utility to start, stop, restart, reload configuration files,\n
report the status of a PostgreSQL server, or signal a PostgreSQL process.\n\n), progname);
printf(_(Usage:\n));
+ printf(_( %s init[-D DATADIR] [-s] [-o \OPTIONS\]\n), progname);
printf(_( %s start [-w] [-t SECS] [-D DATADIR] [-s] [-l FILENAME] [-o \OPTIONS\]\n), progname);
printf(_( %s stop[-W] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n), progname);
printf(_( %s restart [-w] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n
***
*** 1567,1573
#endif
printf(_( -l, --log FILENAME write (or append) server log to FILENAME\n));
printf(_( -o OPTIONS command line options to pass to postgres\n
! (PostgreSQL server executable)\n));
printf(_( -p PATH-TO-POSTGRESnormally not necessary\n));
printf(_(\nOptions for stop or restart:\n));
printf(_( -m SHUTDOWN-MODE can be \smart\, \fast\, or \immediate\\n));
--- 1602,1608
#endif
printf(_( -l, --log FILENAME write (or append) server log to FILENAME\n));
printf(_( -o OPTIONS command line options to pass to postgres\n
! (PostgreSQL server executable) or initdb\n));
printf(_( -p PATH-TO-POSTGRESnormally not necessary\n));
printf(_(\nOptions for stop or restart:\n));
printf(_( -m SHUTDOWN-MODE can be \smart\, \fast\, or \immediate\\n));
***
*** 1824,1830
exit(1);
}
! if (strcmp(argv[optind], start) == 0)
ctl_command = START_COMMAND;
else if (strcmp(argv[optind], stop) == 0)
ctl_command = STOP_COMMAND;
--- 1859,1867
exit(1);
}
! if (strcmp(argv[optind], init) == 0)
! ctl_command = INIT_COMMAND;
! else if (strcmp(argv[optind], start) == 0)
ctl_command = START_COMMAND;
else if (strcmp(argv[optind], stop) == 0)
ctl_command = STOP_COMMAND;
***
*** 1921,1926
--- 1958,1966
switch (ctl_command)
{
+ case INIT_COMMAND:
+ do_init();
+ break;
case STATUS_COMMAND:
do_status();
break;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Gothic moth fails on Tsearch2 contrib module check (PG8.2)
You can see strange error on gothic moth (Solaris Nevada, SunStudio 12, Sparc): http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=gothic_mothdt=2009-09-08%2019:06:01 = pgsql.19404/contrib/tsearch2/regression.diffs === *** ./expected/tsearch2.out Tue Sep 8 21:08:08 2009 --- ./results/tsearch2.out Tue Sep 8 21:38:03 2009 *** *** 2453,2459 body bSea/b view wow ubfoo/b bar/u iqq/i a href=http://www.google.com/foo.bar.html; target=_blankYES nbsp;/a ! ff-bg script document.write(15); /script --- 2453,2459 body bSea/b view wow ubfoo/b bar/u iqq/i a href=http://www.google.com/foo.bar.html; target=_blankYES nbsp;/a ! ff-bgff-bg script document.write(15); /script == ff-bg is doubled in the output and space is missing. It smells like compiler bug. I will look closer on it next week, but can somebody point me place in the code where I shall start? Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-Solaris dtrace support is disabled in 8.4!!!?
Dne 5.09.09 00:09, Josh Berkus napsal(a): On 9/4/09 2:49 PM, Tom Lane wrote: Wasn't anybody paying attention? Apparently nobody is using it on other platforms. I know I'm not. I think that buildfarm members can enable it for supported platform. I already did it for my machine. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] set_client_encoding is broken
If you look on gothic_moth and comet_moth http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=gothic_mothdt=2009-08-30%2020:06:00 http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_mothdt=2009-08-29%2021:06:00 you can see following error: ../../src/test/regress/pg_regress --inputdir=. --psqldir=/zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/inst/bin --dbname=contrib_regression --multibyte=UTF8 --no-locale unaccent (using postmaster on Unix socket, default port) == dropping database contrib_regression == psql: FATAL: invalid value for parameter client_encoding: UTF8 command failed: /zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/inst/bin/psql -X -c DROP DATABASE IF EXISTS \contrib_regression\ postgres gmake[1]: *** [installcheck] Error 2 gmake[1]: Leaving directory `/zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/pgsql.4092/contrib/unaccent' [4a9ae815.696e:1] LOG: connection received: host=[local] [4a9ae815.696e:2] LOG: connection authorized: user=postgres database=postgres [4a9ae815.696e:3] LOG: conversion between UTF8 and LATIN2 is not supported [4a9ae815.696e:4] FATAL: invalid value for parameter client_encoding: UTF8 The assign_client_encoding-SetClientEncoding fails to find conversion function. http://doxygen.postgresql.org/backend_2commands_2variable_8c.html#7f2d0624e7b7fb46644c5ce284e6479c http://doxygen.postgresql.org/mbutils_8c.html#8eeff4ecab443ba7073c426fcd4bc4d6 I guess that flat auth file patch http://archives.postgresql.org/pgsql-committers/2009-08/msg00301.php is culprint. It seems that backend does not have loaded pg_encoding yet when SetClientEncoding is processed. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] set_client_encoding is broken
Tom Lane píše v po 31. 08. 2009 v 11:00 -0400: 3. Push the startup-packet GUC processing (approx. lines 3340..3395 of postgres.c, as of CVS HEAD) into InitPostgres, so it can be run during the startup transaction. This is not too unclean, though it would mean exporting process_postgres_switches() from postgres.c; I guess the main thing I don't like about it is that InitPostgres has enough weird responsibilities already. I'm leaning to the third choice, but I wonder if anyone has any comments or better ideas. It seems to me that 3 is OK. Another possibility is that InitPostgres can only fill up rel cache and GUC processing can stay on the same place. But in general, this problem can affect any other GUC variable which has assign hook and needs to lookup. I don't know how it works before, but I'm afraid that user can get error message in server encoding before it is correctly set. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Tom Lane píše v so 22. 08. 2009 v 09:56 -0400:
Zdenek Kotala zdenek.kot...@sun.com writes:
There are most important records from yesterdays issues.
Messages:
-
Aug 20 11:14:54 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap
space to grow stack for pid 507 (postgres)
Hmm, that seems to confirm the idea that something had run the machine
out of memory/swap space, which would explain the repeated ENOMEM fork
failures. But we're still no closer to understanding how come the
delay in the avlauncher didn't do what it was supposed to.
I found hungry process which eats up all memory and fortunately it is
not postgres :-).
I run also following dtrace script:
dtrace -n 'syscall::kill:entry / execname==postgres/ { printf(%i %
s, %i-%i : %i, timestamp, execname, pid, arg0, arg1); }'
and it show following (little bit modified) output:
snip
CPU Timestamp[ns] diff[ms]caller callee sig
0 275074552090899,96 28604 - 28608 16
3 2750745100280460100,23 28608 - 28604 16
1 2750746000144690899,86 28604 - 28608 16
3 2750746100380940100,24 28608 - 28604 16
2 2750747000135380899,75 28604 - 28608 16
3 2750747100171650100,04 28608 - 28604 16
0 2750748000101050899,93 28604 - 28608 16
3 2750748100331900100,23 28608 - 28604 16
1 2750749000148550899,82 28604 - 28608 16
3 2750749100386640100,24 28608 - 28604 16
2 275075095040899,71 28604 - 28608 16
3 2750750100127780100,03 28608 - 28604 16
We can see there that AVlauncher really wait 100ms, but it is not enough
when system is under stress.
I tested Alvaro's patch and it works, because it does not lead to stack
consumption, but it shows another bug in StartAutovacuumWorker() code.
When fork fails bn structure is freed but
ReleasePostmasterChildSlot() should be called as well. See error:
2009-08-24 11:50:20.360 CEST 3468 FATAL: no free slots in PMChildFlags array
and comment in source code:
/* Out of slots ... should never happen, else postmaster.c messed up */
I think that Alvaro's patch is good and it fix a crash problem. I also
think that AVlauncher could wait little bit more then 100ms. When system
cannot fork, I don't see any reason why hurry to repeat a fork
operation. Maybe 1s is good compromise.
Zdenek
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Zdenek Kotala píše v po 24. 08. 2009 v 13:47 +0200: I tested Alvaro's patch and it works, because it does not lead to stack consumption, but it shows another bug in StartAutovacuumWorker() code. When fork fails bn structure is freed but ReleasePostmasterChildSlot() should be called as well. See error: 2009-08-24 11:50:20.360 CEST 3468 FATAL: no free slots in PMChildFlags array and comment in source code: /* Out of slots ... should never happen, else postmaster.c messed up */ Just a confirmation that Alvaro's patch+ReleasePostmasterChildSlot() fix solves the problem and PostgreSQL survives well during a memory shortages. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Alvaro Herrera píše v pá 21. 08. 2009 v 17:48 -0400: Tom Lane wrote: I'd still like to have some fork-rate-limiting behavior in there somewhere. However, it might make sense for the avlauncher to do that rather than the postmaster. Does that idea seem more implementable? snip Does it just need a longer delay? I see two problem with delay. First is how big it should be? It depends on machine speed and load. And second is that it can be interrupted by signal. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Tom Lane píše v pá 21. 08. 2009 v 18:06 -0400: Maybe, but I think we need to understand exactly what happened first. I try to mine more data from the system to reconstruct what happen. Unfortunately, default postgresql log configuration does not have timestamp. The postgresql had no load, system was under load which could cause some slow system response. ... and ... bingo I'm able to reproduce it. I will look on it deeply with dtrace on Monday (I'm leaving now for a weekend). There are most important records from yesterdays issues. Messages: - Aug 20 11:14:54 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 507 (postgres) svc log: [ Aug 20 10:53:27 Stopping because service disabled. ] [ Aug 20 10:53:27 Executing stop method (/lib/svc/method/postgres_84 stop). ] waiting for server to shut down. done server stopped [ Aug 20 10:53:31 Method stop exited with status 0. ] [ Aug 20 10:54:17 Executing start method (/lib/svc/method/postgres_84 start). ] server starting [ Aug 20 10:54:19 Method start exited with status 0. ] [ Aug 20 11:14:58 Stopping because process dumped core. ] [ Aug 20 11:16:05 Executing stop method (/lib/svc/method/postgres_84 stop). ] pg_ctl: could not send stop signal (PID: 507): No such process [ Aug 20 11:16:12 Method stop exited with status 0. ] [ Aug 20 11:17:17 Executing start method (/lib/svc/method/postgres_84 start). ] ld.so.1: sh: fatal: mmap anon failed: Resource temporarily unavailable [ Aug 20 11:17:28 Method start failed due to signal KILL. ] [ Aug 20 11:18:39 Executing start method (/lib/svc/method/postgres_84 start). ] ld.so.1: sh: fatal: mmap anon failed: Resource temporarily unavailable [ Aug 20 11:18:50 Method start failed due to signal KILL. ] [ Aug 20 11:19:58 Executing start method (/lib/svc/method/postgres_84 start). ] ld.so.1: sh: fatal: mmap anon failed: Resource temporarily unavailable postgresql.log: --- LOG: received smart shutdown request LOG: autovacuum launcher shutting down LOG: shutting down LOG: database system is shut down LOG: database system was shut down at 2009-08-20 10:53:30 CEST LOG: database system is ready to accept connections LOG: autovacuum launcher started LOG: could not fork autovacuum worker process: Not enough space ... TopMemoryContext: 31192 total in 4 blocks; 6056 free (4 chunks); 25136 used TopTransactionContext: 8192 total in 1 blocks; 7792 free (1 chunks); 400 used TransactionAbortContext: 32768 total in 1 blocks; 32752 free (0 chunks); 16 used Portal hash: 8192 total in 1 blocks; 3912 free (0 chunks); 4280 used PortalMemory: 0 total in 0 blocks; 0 free (0 chunks); 0 used Relcache by OID: 8192 total in 1 blocks; 4936 free (0 chunks); 3256 used CacheMemoryContext: 274480 total in 18 blocks; 17072 free (2 chunks); 257408 used snip rest of memory context dump ERROR: out of memory DETAIL: Failed on request of size 8. LOG: could not fork autovacuum worker process: Not enough space ... LOG: could not fork autovacuum worker process: Not enough space WARNING: worker took too long to start; cancelled LOG: could not fork autovacuum worker process: Not enough space ... LOG: could not fork autovacuum worker process: Not enough space WARNING: worker took too long to start; cancelled 2009-08-21 13:30:48.701 CEST 416 LOG: database system was interrupted; last known up at 2009-08-20 10:54:21 CEST 2009-08-21 13:30:48.963 CEST 416 LOG: database system was not properly shut down; automatic recovery in progress -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
I found following core file of PG 8.4.0 on my system (Solaris Nevada b119): fe8ae42d _dowrite (85bf6e8, 3a, 8035e3c, 80350e8) + 8d fe8ae743 _ndoprnt (85bf6e8, 8035ec8, 8035e3c, 0) + 2ba fe8b322d vsnprintf (85bfaf0, 3ff, 85bf6e8, 8035ec8, 0, 0) + 65 082194ea appendStringInfoVA (8035e9c, 85bf6e8, 8035ec8) + 4a 083ca5d3 errmsg (849c340, 0) + 103 0829272d StartAutoVacWorker (fe97f000, 32, 85b82b0, 8035ef4, 82a1496, c) + 3d 082a1901 StartAutovacuumWorker (c, 8035f08, fe8ed28f, 10, 0, 8035fbc) + 71 082a1496 sigusr1_handler (10, 0, 8035fbc) + 186 fe8ed28f __sighndlr (10, 0, 8035fbc, 82a1310) + f fe8e031f call_user_handler (10) + 2af fe8e054f sigacthandler (10, 0, 8035fbc) + df --- called from signal handler with signal 16 (SIGUSR1) --- fe8f37f6 __systemcall (3, fec32b88, 0, fe8e0b46) + 6 fe8e0c71 thr_sigsetmask (3, 85abd50, 0, fe8e0d18) + 139 fe8e0d3f sigprocmask (3, 85abd50, 0) + 31 082a14a4 sigusr1_handler (10, 0, 8036340) + 194 fe8ed28f __sighndlr (10, 0, 8036340, 82a1310) + f fe8e031f call_user_handler (10) + 2af fe8e054f sigacthandler (10, 0, 8036340) + df ... 80x same sighandler stack --- called from signal handler with signal 16 (SIGUSR1) --- fe8f37f6 __systemcall (3, fec32b88, 0, fe8e0b46) + 6 fe8e0c71 thr_sigsetmask (3, 85abd50, 0, fe8e0d18) + 139 fe8e0d3f sigprocmask (3, 85abd50, 0) + 31 082a14a4 sigusr1_handler (10, 0, 80478fc) + 194 fe8ed28f __sighndlr (10, 0, 80478fc, 82a1310) + f fe8e031f call_user_handler (10) + 2af fe8e054f sigacthandler (10, 0, 80478fc) + df --- called from signal handler with signal 16 (SIGUSR1) --- fe8f1867 __pollsys (8047b50, 2, 8047c04, 0) + 7 fe89ce61 pselect (6, 8047c44, 0, 0, 8047c04, 0) + 199 fe89d236 select (6, 8047c44, 0, 0, 8047c38, 0) + 78 0829dc20 ServerLoop (feffb804, bd26003b, 41b21fcb, 85c1de0, 1, 0) + c0 0829d5d0 PostmasterMain (3, 85b72c8) + dd0 08227abf main (3, 85b72c8, 8047df0, 8047d9c) + 22f 080b893d _start (3, 8047e80, 8047ea5, 8047ea8, 0, 8047ec2) + 7d The problem what I see here is that StartAutovacuumWorker() fails and send SIGUSR1 to the postmaster, but it send it too quickly and signal handler is still active. When signal mask is unblocked in sigusr1_handler() than signal handler is run again... The reason why StartAutovacuumWorker() is interesting. Log says: LOG: could not fork autovacuum worker process: Not enough space It is strange and I don't understand it. May be too many nested signal handlers call could cause it. Strange also is that 100ms is not enough to protect this situation, but I think that sleep could interrupted by signal. My suggestion is to set for example gotUSR1=true in sigusr1_handler() and in the server loop check if we got a USR1 signal. It avoids any problems with signal handler which is not currently POSIX compliant anyway. any other ideas? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Alvaro Herrera píše v pá 21. 08. 2009 v 15:40 -0400: Zdenek Kotala wrote: The problem what I see here is that StartAutovacuumWorker() fails and send SIGUSR1 to the postmaster, but it send it too quickly and signal handler is still active. When signal mask is unblocked in sigusr1_handler() than signal handler is run again... The reason why StartAutovacuumWorker() is interesting. Log says: LOG: could not fork autovacuum worker process: Not enough space Does this mean that the machine is out of swap space? It is ENOMEM error. But it is strange. Machine has 4GB RAM and it was freshly installed PG84 without any data and with default configuration. It was not under load. I did not find any clue what happend with memory on this system. The question is if out of memory was a source or result of the pinpong. It is strange and I don't understand it. May be too many nested signal handlers call could cause it. Strange also is that 100ms is not enough to protect this situation, but I think that sleep could interrupted by signal. My suggestion is to set for example gotUSR1=true in sigusr1_handler() and in the server loop check if we got a USR1 signal. It avoids any problems with signal handler which is not currently POSIX compliant anyway. What 100ms? The pg_usleep call you see in ServerLoop is only there during shutdown; normally it would be the select() call that would be blocking the process. I mean AutoVacLauncherMain() http://doxygen.postgresql.org/autovacuum_8c.html#19ef1013e6110a4536ed92a454aba8c9 line 656 If sigusr1_handler needs rewriting, don't all the other sighandler as well? Note that the process is supposed to be running with signals blocked all the time except during those sleep/select calls, which is what (according to comments) let the sighandlers do nontrivial tasks. Comments says that it is OK. POSIX says that is not OK and my instinct say to trust the POSIX standard. Especially I do not see any reason why we need do this in signal handler. avl_sigterm_handler and so on are good example how it should be implemented in postmaster as well. The core shows that it is not good idea to have complicated signal handler. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] uuid contrib don't compile in OpenSolaris
Dne 24.07.09 19:23, Emanuel Calvo Franco napsal(a): Hi all, I have some issues to compile uuid contrib of 8.4 version. Touching something i see that the gmake don't find uuid.h. (pfexec gmake -d) Touching more, i add uuid.h into the uuid directory and i had a error message: missing separator. Did you use gmake the second time? What was the exact make and compiler error message? Every time I execute the gmake, the error is the same (uuid.h not found). WHen i copy uuid.h in the current dir of uuid contrib, the error changes to missing separator. As I say, looking in the web i found that maybe is a bad character (a hidden tab or something like this). This occurs only in opensolaris 200906 (i didn't test on previous versions). Hi, problem is that PostgreSQL uses ossp UUID which is not integrated in OpenSolaris. Solaris has own uuid implementation which seems to me similar but not same. Try man uuid_generate or less /usr/uuid/uuid.h It should be easy to port it but it needs a time :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] compilation with libeditpreferred is broken
I attached conservative version of patch which only reorder #define to avoid cross including half from readline and half from editline. Zdenek Tom Lane píše v čt 06. 08. 2009 v 18:13 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: It seems to me that editline never distributed history.h file and HAVE_EDITLINE_HISTORY_H is nonsense. But I'm not sure. I wouldn't count on that, in part because there are so many versions of editline. On an OS X machine I see $ ls -l /usr/include/*line* /usr/include/editline: total 16 -rw-r--r-- 1 root wheel 6882 Feb 19 2008 readline.h /usr/include/readline: total 16 lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 history.h@ - ../editline/readline.h lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 readline.h@ - ../editline/readline.h regards, tom lane diff -r dd477d7938da src/bin/psql/input.h --- a/src/bin/psql/input.h Wed Jun 03 00:38:34 2009 + +++ b/src/bin/psql/input.h Wed Aug 12 21:49:49 2009 +0200 @@ -18,17 +18,19 @@ #define USE_READLINE 1 #if defined(HAVE_READLINE_READLINE_H) #include readline/readline.h +#if defined(HAVE_READLINE_HISTORY_H) +#include readline/history.h +#endif #elif defined(HAVE_EDITLINE_READLINE_H) #include editline/readline.h +#if defined(HAVE_EDITLINE_HISTORY_H) +#include editline/history.h +#endif #elif defined(HAVE_READLINE_H) #include readline.h +#if defined(HAVE_HISTORY_H) +#include history.h #endif -#if defined(HAVE_READLINE_HISTORY_H) -#include readline/history.h -#elif defined(HAVE_EDITLINE_HISTORY_H) -#include editline/history.h -#elif defined(HAVE_HISTORY_H) -#include history.h #endif #endif -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] compilation with libeditpreferred is broken
Dne 7.08.09 00:13, Tom Lane napsal(a): Zdenek Kotala zdenek.kot...@sun.com writes: It seems to me that editline never distributed history.h file and HAVE_EDITLINE_HISTORY_H is nonsense. But I'm not sure. I wouldn't count on that, in part because there are so many versions of editline. On an OS X machine I see $ ls -l /usr/include/*line* /usr/include/editline: total 16 -rw-r--r-- 1 root wheel 6882 Feb 19 2008 readline.h /usr/include/readline: total 16 lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 history.h@ - ../editline/readline.h lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 readline.h@ - ../editline/readline.h It is only hack for application which wants to have readline and they don't detects libedit. Finally it is like: #include editline/readline.h #include editline/readline.h I little bit searching on Internet and it seems that when edit/history.h exists it is link to editline/readline.h. (See e.g. http://sysinf0.klabs.be/usr/include/editline/history.h?dist=;arch= ) This link is created by packager, because when you compiling libedit, make install does not create it. By my opinion HAVE_EDITLINE_HISTORY_H is overhead, but we can keep it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [Pg-migrator-general] Composite types break pg_migrated tables
Dne 6.08.09 04:29, Tom Lane napsal(a): Andrew Dunstan and...@dunslane.net writes: preventing a clash might be fairly difficult. Yeah, I was just thinking about that. The easiest way to avoid collisions would be to make pg_dump (in --binary-upgrade mode) responsible for being sure that *every* new pg_type and pg_class row OID matches what it was in the old DB. We could stop doing that once we have all the user tables in place --- I don't believe it's necessary to preserve the OIDs of user indexes. But we need to preserve toast table OIDs, and toast table index OIDs too if those are created at the same time they are now (else we risk one of them colliding with a toast table OID we want to create later). Oh, and pg_enum rows too. It seems doable, but we're certainly not going to back-patch any such thing into 8.4 ... Another way is to use direct catalog update which I presented on PgCon. I think it should be easy to finish it (2-3weeks) for 8.4 - needs small extension of bootstrap. And of course testing, testing ... Also to remove oid in catalog and replace it with standard column (type can be oid) should make things easier. But it is for 8.5. I will send a code on Monday for people who wants to look on it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] compilation with libeditpreferred is broken
When I try compile postgresql with --libeditpreferred option, compilation fails when readline is also installed on the system. You can see error report on: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dot_mothdt=2009-08-06%2012:46:04 The main problem is in src/bin/psql/input.h where is following ifdef magic: #define USE_READLINE 1 #if defined(HAVE_READLINE_READLINE_H) #include readline/readline.h #elif defined(HAVE_EDITLINE_READLINE_H) #include editline/readline.h #elif defined(HAVE_READLINE_H) #include readline.h #endif #if defined(HAVE_READLINE_HISTORY_H) #include readline/history.h #elif defined(HAVE_EDITLINE_HISTORY_H) #include editline/history.h #elif defined(HAVE_HISTORY_H) #include history.h #endif The problem is that libedit does not distribute editline/history.h and configure detects that there is readline/history.h and sets HAVE_READLINE_HISTORY_H macro. Finally input.h includes editline/readline.h and readline/history.h which causes symbol conflicts. It seems to me that editline never distributed history.h file and HAVE_EDITLINE_HISTORY_H is nonsense. But I'm not sure. I attached suggested fix, but it needs also some work in ./configure - depends if libedit/history.h existed. Anyone knows a history? I need to backported this fix for branches 8.2 - 8.4, because OpenSolaris PostgreSQL binaries build is broken now. Zdenek diff -r dd477d7938da src/bin/psql/input.h --- a/src/bin/psql/input.h Wed Jun 03 00:38:34 2009 + +++ b/src/bin/psql/input.h Thu Aug 06 22:24:58 2009 +0200 @@ -18,17 +18,16 @@ #define USE_READLINE 1 #if defined(HAVE_READLINE_READLINE_H) #include readline/readline.h +#if defined(HAVE_READLINE_HISTORY_H) +#include readline/history.h +#endif #elif defined(HAVE_EDITLINE_READLINE_H) #include editline/readline.h #elif defined(HAVE_READLINE_H) #include readline.h +#if defined(HAVE_HISTORY_H) +#include history.h #endif -#if defined(HAVE_READLINE_HISTORY_H) -#include readline/history.h -#elif defined(HAVE_EDITLINE_HISTORY_H) -#include editline/history.h -#elif defined(HAVE_HISTORY_H) -#include history.h #endif #endif -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] head contrib is broken (crypto)
It seems that last contrib crypto changes broke buildfarm. You can see problem e.g. here: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emperor_mothdt=2009-08-04%2020:06:01 I don't have time to look on it now. But it should be reproducible everywhere. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
Grzegorz Jaskiewicz píše v čt 16. 07. 2009 v 14:59 +0100: Why C89, and not C99 ? Virtually all compilers for last 4 years have/ had C99 support. Well, I think we want to run on systems that are older than 4 years, too. Sure, but that's probably less than 1% of all systems. The 4 years was a guess, I think its much more than that. For example Solaris 8 is 9 years old and still is used in production. I guess HP-UX is in same situation. And so on. I not able to say how many PostgreSQL runs on them but how Tom mentioned there is no significant reason to break old platform. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] pg_migrator not setting values of sequences?
Dne 14.07.09 04:37, Bruce Momjian napsal(a): Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tilmann Singer wrote: However, all of the sequences were at the initial values and not bumped up to the last used value as I would have expected. The first nextval call on any sequence in the migrated 8.4 database always returned 1. Wow, that is also surprising. I am going to have to run some tests to find the cause, but it certainly is not intended. Looks like pg_migrator neglects to include relkind 'S' in the set of tables that it needs to physically migrate. Thanks, I have fixed pg_migrator with the attached patch. Once we find the cause of the lovacuum problem, I will make a new pg_migrator release. I think you don't need transfer sequence table, because pg_dump should create DDL command (CREATE SEQUENCE) which should create new nice sequence relation. Zdenek
Re: [HACKERS] Upgrading our minimum required flex version for 8.5
Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400: Actually, most of the buildfarm members show which flex version they are running in the configure output. A quick look shows that of the 45 members that have reported on HEAD in the past 2 days, 22 are running 2.5.4, which is a lot higher than I was expecting. Most of these are the Solaris boxen, which I imagine can be updated fairly painlessly since there are some of them that are already running something newer. My three S10 are updated. All of them use version 2.5.35 now. S8, S9 is not yet under my control and they are not migrated yet to new location. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] First CommitFest: July 15th
Peter Eisentraut píše v pá 03. 07. 2009 v 09:19 +0300: On Friday 03 July 2009 05:16:41 Robert Haas wrote: On Thu, Jul 2, 2009 at 3:42 PM, Zdenek Kotalazdenek.kot...@sun.com wrote: Josh Berkus píše v st 01. 07. 2009 v 17:21 -0700: Folks, There's been a lot of discussion/argument around how to handle the last commitfest, but there seems to be a total consensus that we want to have the first CF on July 15th. Can we add flags like bump catalog version, bump page layout version, modify AM for each patch? It should help to track pg_upgrade changes. That's not a bad idea, and it wouldn't be hard to add various flags and things to the CommitFest app I wrote, but how would we maintain the information and keep it correct? It seems like there might be a danger that patch authors wouldn't know whether or not they were doing those things. Also, how would we handle changes by committers, who don't always go through the CommitFest process? I think this information could be computed automatically, if someone cared enough. It shouldn't be necessary to bother every single participant in the process with this. I think that developer is responsible for his patch. He should know what he doing. When he will send a patch and see checkbox like modified AM? then he should know what he modified? It is also warning for commiter that catalog version has to be bumped during a commit. I don't see any method how to check automatically. Something could be possible - like structure checker. But when meaning of data is going to be different. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] First CommitFest: July 15th
Robert Haas píše v čt 02. 07. 2009 v 22:16 -0400: On Thu, Jul 2, 2009 at 3:42 PM, Zdenek Kotalazdenek.kot...@sun.com wrote: Also, how would we handle changes by committers, who don't always go through the CommitFest process? I think that all head patches should go to through a new tool for recording also in case when developer is commiter itself. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading our minimum required flex version for 8.5
Tom Lane píše v čt 02. 07. 2009 v 11:32 -0400: Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: Right, this will only affect people doing development or otherwise building from a CVS pull. Of course, that includes the whole buildfarm. We might need to ask some people to upgrade there. Yes. What I was thinking of doing was committing a configure change to reject flex 2.5.31, and waiting to see how much of the buildfarm goes red. Solaris 10 has 2.5.4 version. OpenSolaris already has 2.5.33. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading our minimum required flex version for 8.5
Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400: I wrote: Yes. What I was thinking of doing was committing a configure change to reject flex 2.5.31, and waiting to see how much of the buildfarm goes red. Actually, most of the buildfarm members show which flex version they are running in the configure output. A quick look shows that of the 45 members that have reported on HEAD in the past 2 days, 22 are running 2.5.4, which is a lot higher than I was expecting. Most of these are the Solaris boxen, which I imagine can be updated fairly painlessly since there are some of them that are already running something newer. I hope, need to check/compile, but I don't expect any problems. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] First CommitFest: July 15th
Josh Berkus píše v st 01. 07. 2009 v 17:21 -0700: Folks, There's been a lot of discussion/argument around how to handle the last commitfest, but there seems to be a total consensus that we want to have the first CF on July 15th. Can we add flags like bump catalog version, bump page layout version, modify AM for each patch? It should help to track pg_upgrade changes. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] list_head naming conflict gcc 4.2/perl/solaris
Zdenek Kotala píše v po 01. 06. 2009 v 22:45 +0200: Tom Lane píše v po 01. 06. 2009 v 16:09 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: What is sys/list.h, and why is it being imported by the Perl headers? It seems that problem is with Perl. It includes sys/mode.h. The new change for gcc 4.2 is that mode.h includes vnode.h and it finally sys/list.h which is generic list for kernel structures. Can we do something like #define list_head solaris_list_head around the problematic #include? (Which one is that, anyway?) I will try it. There is following hack: CFLAGS=-D_SYS_VNODE_H which disable vnode.h and breaks include chain. I need to install fresh nightly build of Solaris to test it on live system. Perl includes sys/mode.h which is probably not necessary (it look likes that it is something for AIX), but it seems that sys/*.h headers modifications break more than only PostgreSQL. And kernel guys should fix it. Which means no change for PostgreSQL. Only who will use opensolaris devel builds 116 could have a problem. I will skip these build(s) on my buildfarm animals. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] list_head naming conflict gcc 4.2/perl/solaris
During integration gcc4.2 into Solaris. My colleague hit a following
problem with PostgreSQL compilation:
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6845982
cd
/builds/sfw-fixes/usr/src/cmd/postgres/postgresql-8.2/postgresql-8.2.13/src/pl/plperl
+ /ws/onnv-tools/SUNWspro/SS12/bin/cc -xO3 -m32 -xchip=pentium -xspace -Xa
-xildoff -xc99=all -xc99=none -xCC -KPIC -I. -I../../../src/include
-I/builds/sfw-fixes/proto/root_i386/usr/include
-I/builds/sfw-fixes/proto/root_i386/usr/sfw/include -I/usr/sfw/include
-I/usr/include/kerberosv5 -I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE -c -o
plperl.o plperl.c
/usr/include/sys/list.h, line 52: identifier redeclared: list_head
current : function(pointer to struct list {unsigned int list_size,
unsigned int list_offset, struct list_node {..} list_head}) returning pointer
to void
previous: function(pointer to struct List {enum NodeTag
{T_TIDBitmap(902), (blah blah)
At this moment there is no clear conclusion how to fix it in solaris system
headers to satisfy all side. :(
My idea is to rename list_head to pg_list_head (and other functions
analogously) to avoid name conflict.
Comments, better ideas?
Thanks Zdenek
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] list_head naming conflict gcc 4.2/perl/solaris
Robert Haas píše v po 01. 06. 2009 v 16:03 -0400:
On Mon, Jun 1, 2009 at 3:57 PM, Zdenek Kotala zdenek.kot...@sun.com wrote:
During integration gcc4.2 into Solaris. My colleague hit a following
problem with PostgreSQL compilation:
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6845982
cd
/builds/sfw-fixes/usr/src/cmd/postgres/postgresql-8.2/postgresql-8.2.13/src/pl/plperl
+ /ws/onnv-tools/SUNWspro/SS12/bin/cc -xO3 -m32 -xchip=pentium -xspace -Xa
-xildoff -xc99=all -xc99=none -xCC -KPIC -I. -I../../../src/include
-I/builds/sfw-fixes/proto/root_i386/usr/include
-I/builds/sfw-fixes/proto/root_i386/usr/sfw/include -I/usr/sfw/include
-I/usr/include/kerberosv5 -I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE
-c -o plperl.o plperl.c
/usr/include/sys/list.h, line 52: identifier redeclared: list_head
current : function(pointer to struct list {unsigned int list_size,
unsigned int list_offset, struct list_node {..} list_head}) returning
pointer to void
previous: function(pointer to struct List {enum NodeTag
{T_TIDBitmap(902), (blah blah)
At this moment there is no clear conclusion how to fix it in solaris system
headers to satisfy all side. :(
My idea is to rename list_head to pg_list_head (and other functions
analogously) to avoid name conflict.
Comments, better ideas?
Uh... isn't it rather bad form for the operating system headers to be
defining commonly used type names like list?
It is in solaris since 2003 for kernel structure linked list. When I
look on it closer, it seems that problem is more in perl which includes
some sys/ headers which is probably not good practice. :(
I'd be very surprised
if PostgreSQL is the only application that is going to break on this.
At this moment I know only about PostgreSQL. How I understand it now, it
appears only when source is compiling with Perl.
I will look more on it tomorrow.
thanks Zdenek
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] list_head naming conflict gcc 4.2/perl/solaris
Tom Lane píše v po 01. 06. 2009 v 16:09 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: What is sys/list.h, and why is it being imported by the Perl headers? It seems that problem is with Perl. It includes sys/mode.h. The new change for gcc 4.2 is that mode.h includes vnode.h and it finally sys/list.h which is generic list for kernel structures. Can we do something like #define list_head solaris_list_head around the problematic #include? (Which one is that, anyway?) I will try it. There is following hack: CFLAGS=-D_SYS_VNODE_H which disable vnode.h and breaks include chain. I need to install fresh nightly build of Solaris to test it on live system. Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq is not thread safe
Bruce Momjian píše v čt 28. 05. 2009 v 17:20 -0400: Done, patch attached and applied. I went with a warning because it seemed most appropriate, but it looks very large: http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html Should it be a notice? I prefer warning. It is important message and beginners usually don't know it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Tom Lane píše v čt 28. 05. 2009 v 11:42 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: I attached another cleanup patch which fixes following warnings reported by Sun Studio: I'm not too impressed with any of these. The proposed added initializers just increase future maintenance effort without solving any real problem (since the variables are required by C standard to initialize to zero). Agree. The proposed signature change on psql_completion is going to replace a warning on your system with outright failures on other people's. I check readline and definition is still same at least from 5.0 version. I'm not still understand why it should failure on other systems. I looked on revision 1.30 of the file and there is only readline-4.2 support mentioned. Is readline 4.2 the problem? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Tom Lane píše v čt 28. 05. 2009 v 11:57 -0400: ). AFAICS, Sun's compiler is just too stupid and shouldn't be emitting this warning. Perhaps the right response is to file a bug report against the compiler. I checked it and it is already know bug. It is new lint style check in Sun Studio 12. Unfortunately, problem is that current workflow does not allow to detect if code is dead or not in the verification phase. Next sun studio release could fix it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator and an 8.3-compatible tsvector data type
Bruce Momjian píše v čt 28. 05. 2009 v 17:42 -0400: Josh Berkus wrote: On 5/28/09 2:30 PM, Bruce Momjian wrote: Because no one has responded, I am going to prevent pg_migrator from working with a cluster that uses tsvector. I realize this limits pg_migrator's usefulness, but I have to move on. I don't know how to fix the issue, certainly. Why was there a format change to tsvector anyway? Was there an important reason for this? The ordering of the lexems was changed: The biggest problem is dictionary change. I'm not sure if it happened but IIRC Teodor mentioned it in Ottawa. If it happened It hits down tsvector compatibility at all. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator and an 8.3-compatible tsvector data type
Tom Lane píše v pá 29. 05. 2009 v 11:28 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: The biggest problem is dictionary change. I'm not sure if it happened but IIRC Teodor mentioned it in Ottawa. If it happened It hits down tsvector compatibility at all. No more than changing dictionary behavior in an existing installation. What was stated when the issue came up during 8.3 development is that you don't normally need to worry about small changes in dictionary behavior because overall text search behavior will still be close enough. (I seem to recall that I'd complained that any change in dictionary behavior would invalidate indexes based on the dictionary, and this was the answer.) It seems to be OK then. However, pg_migrator should inform about it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
I attached another cleanup patch which fixes following warnings reported
by Sun Studio:
zic.c, line 1534: warning: const object should have initializer: tzh0
dynloader.c, line 7: warning: empty translation unit
pgstat.c, line 666: warning: const object should have initializer: all_zeroes
pgstat.c, line 799: warning: const object should have initializer: all_zeroes
pgstat.c, line 2552: warning: const object should have initializer: all_zeroes
preproc.c, line 39569: warning: pointer expression or its operand do not
point to the same object yyerror_range, result is undefined and non-portable
tab-complete.c, line 587: warning: assignment type mismatch:
pointer to function(pointer to const char, int, int) returning pointer
to pointer to char = pointer to void
Following list is still unfixed plus see my comments:
gram.c, line 28487: warning: pointer expression or its operand do not point
to the same object yyerror_range, result is undefined and non-portable
- This is really strange warning. The code is really strange
because it points to -1 index of array, but I'm not bison guru.
Maybe it is correct, but it would be good if somebody check it.
../../../src/include/pg_config.h, line 782: warning: macro redefined:
_FILE_OFFSET_BITS
- This probably needs some extra love in configure.
regc_lex.c, line 401: warning: loop not entered at top
regc_lex.c, line 484: warning: loop not entered at top
regc_lex.c, line 578: warning: loop not entered at top
regc_lex.c, line 610: warning: loop not entered at top
regc_lex.c, line 870: warning: loop not entered at top
regc_lex.c, line 1073: warning: loop not entered at top
postgres.c, line 3845: warning: loop not entered at top
- Assert on not reached place probably confuse compiler. I'm not
sure if it make sense nowadays? Most compiler should optimize
this anyway and code is removed. I suppose to remove these
asserts.
Zdenek
diff -Nrc pgsql.orig.8fc4f032818a/src/backend/port/dynloader/solaris.c pgsql.orig/src/backend/port/dynloader/solaris.c
*** pgsql.orig.8fc4f032818a/src/backend/port/dynloader/solaris.c 2009-05-28 11:09:24.874020865 +0200
--- pgsql.orig/src/backend/port/dynloader/solaris.c 2009-05-28 11:09:24.893688008 +0200
***
*** 5,7
--- 5,11
*
* see solaris.h
*/
+
+ /* compiler complains about empty unit, let compiler quite */
+ extern int no_such_variable;
+
diff -Nrc pgsql.orig.8fc4f032818a/src/backend/postmaster/pgstat.c pgsql.orig/src/backend/postmaster/pgstat.c
*** pgsql.orig.8fc4f032818a/src/backend/postmaster/pgstat.c 2009-05-28 11:09:24.882883806 +0200
--- pgsql.orig/src/backend/postmaster/pgstat.c 2009-05-28 11:09:24.894106321 +0200
***
*** 662,669
void
pgstat_report_stat(bool force)
{
! /* we assume this inits to all zeroes: */
! static const PgStat_TableCounts all_zeroes;
static TimestampTz last_report = 0;
TimestampTz now;
--- 662,668
void
pgstat_report_stat(bool force)
{
! static const PgStat_TableCounts all_zeroes = {0,0,0,0,0,0,0,0,0,0,0};
static TimestampTz last_report = 0;
TimestampTz now;
***
*** 795,802
static void
pgstat_send_funcstats(void)
{
! /* we assume this inits to all zeroes: */
! static const PgStat_FunctionCounts all_zeroes;
PgStat_MsgFuncstat msg;
PgStat_BackendFunctionEntry *entry;
--- 794,800
static void
pgstat_send_funcstats(void)
{
! static const PgStat_FunctionCounts all_zeroes = {0,{0,0},{0,0}};
PgStat_MsgFuncstat msg;
PgStat_BackendFunctionEntry *entry;
***
*** 2548,2555
void
pgstat_send_bgwriter(void)
{
! /* We assume this initializes to zeroes */
! static const PgStat_MsgBgWriter all_zeroes;
/*
* This function can be called even if nothing at all has happened. In
--- 2546,2552
void
pgstat_send_bgwriter(void)
{
! static const PgStat_MsgBgWriter all_zeroes = { {0,0},0,0,0,0,0,0,0};
/*
* This function can be called even if nothing at all has happened. In
diff -Nrc pgsql.orig.8fc4f032818a/src/bin/psql/tab-complete.c pgsql.orig/src/bin/psql/tab-complete.c
*** pgsql.orig.8fc4f032818a/src/bin/psql/tab-complete.c 2009-05-28 11:09:24.890322342 +0200
--- pgsql.orig/src/bin/psql/tab-complete.c 2009-05-28 11:09:24.894581639 +0200
***
*** 557,563
/* Forward declaration of functions */
! static char **psql_completion(char *text, int start, int end);
static char *create_command_generator(const char *text, int state);
static char *drop_command_generator(const char *text, int state);
static char *complete_from_query(const char *text, int state);
--- 557,563
/* Forward declaration of functions */
! static char **psql_completion(const char *text, int start, int end);
static char *create_command_generator(const char *text, int state);
static char *drop_command_generator(const char *text, int state);
static
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Michael Meskes píše v čt 28. 05. 2009 v 13:33 +0200: On Thu, May 28, 2009 at 11:11:20AM +0200, Zdenek Kotala wrote: I attached another cleanup patch which fixes following warnings reported by Sun Studio: ... preproc.c, line 39569: warning: pointer expression or its operand do not point to the same object yyerror_range, result is undefined and non-portable ... Following list is still unfixed plus see my comments: gram.c, line 28487: warning: pointer expression or its operand do not point to the same object yyerror_range, result is undefined and non-portable ... These two should be the same, both coming from bison. Both files are auto-generated, thus it might be bison that has to be fixed to remove this warning. yeah it is generated, but question is if generated code is valid or it is bug in bison. If it bison bug we need to care about it. There is the code: yyerror_range[1] = yylloc; /* Using YYLLOC is tempting, but would change the location of the look-ahead. YYLOC is available though. */ YYLLOC_DEFAULT (yyloc, (yyerror_range - 1), 2); *++yylsp = yyloc; Problem is with YYLLOC_DEFAULT. When I look on macro definition #define YYLLOC_DEFAULT(Current, Rhs, N) \ Current.first_line = Rhs[1].first_line; \ Current.first_column = Rhs[1].first_column;\ Current.last_line= Rhs[N].last_line; \ Current.last_column = Rhs[N].last_column; It seems to me that it is OK, because 1 is used as a index which finally point on yyerror_range[0]. Given that I didn't find any mentioning of preproc in your patch I suppose it just hit the wrong list though. I'm sorry copy paste error. Yeah, I did not fix preproc too. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Michael Meskes píše v čt 28. 05. 2009 v 14:47 +0200:
On Thu, May 28, 2009 at 01:51:07PM +0200, Zdenek Kotala wrote:
Problem is with YYLLOC_DEFAULT. When I look on macro definition
#define YYLLOC_DEFAULT(Current, Rhs, N) \
Current.first_line = Rhs[1].first_line; \
Current.first_column = Rhs[1].first_column;\
Current.last_line= Rhs[N].last_line; \
Current.last_column = Rhs[N].last_column;
It seems to me that it is OK, because 1 is used as a index which finally
point on yyerror_range[0].
Wait, this is the bison definition. Well to be more precise the bison
definition in your bison version. Mine is different:
I took it from documentation. I have same as you in generated code.
# define YYLLOC_DEFAULT(Current, Rhs, N)\
do \
if (YYID (N))\
{ \
(Current).first_line = YYRHSLOC (Rhs, 1).first_line;\
(Current).first_column = YYRHSLOC (Rhs, 1).first_column; \
(Current).last_line= YYRHSLOC (Rhs, N).last_line; \
(Current).last_column = YYRHSLOC (Rhs, N).last_column; \
} \
else \
{ \
(Current).first_line = (Current).last_line = \
YYRHSLOC (Rhs, 0).last_line;\
(Current).first_column = (Current).last_column = \
YYRHSLOC (Rhs, 0).last_column; \
} \
while (YYID (0))
Having said that, it doesn't really matter as we redefine the macro:
#define YYLLOC_DEFAULT(Current, Rhs, N) \
do { \
if (N) \
(Current) = (Rhs)[1]; \
else \
(Current) = (Rhs)[0]; \
} while (0)
I have to admit that those version look strikingly unsimilar to me. There is
no
reference to Rhs[N] in our macro at all. But then I have no idea whether this
is needed.
Current is only int. See gramparse.h. I think we could rewrite it this
way:
#define YYLLOC_DEFAULT(Current, Rhs, N) \
do { \
if (N) \
(Current) = (Rhs)[1]; \
else \
(Current) = (Rhs)[N]; \
} while (0)
It is same result and compiler is quite.
Zdenek
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] problem with plural-forms
Peter Eisentraut píše v út 26. 05. 2009 v 13:39 +0300: Of course the concrete example that you show doesn't actually take advantage of this, so if it is important to you, please send a patch to fix it. Fix attached. I found only two problems, both in psql. I did not fix .po files. Is necessary to fix them manually or do you regenerate files? thanks Zdenek diff -r 7a4dd8cda3b6 src/bin/psql/describe.c --- a/src/bin/psql/describe.c Wed May 27 12:28:55 2009 + +++ b/src/bin/psql/describe.c Wed May 27 21:58:15 2009 +0200 @@ -2036,7 +2036,7 @@ if (conns == 0) appendPQExpBuffer(buf, _(No connections)); else -appendPQExpBuffer(buf, ngettext(1 connection, %d connections, conns), conns); +appendPQExpBuffer(buf, ngettext(%d connection, %d connections, conns), conns); } attr[i] = pg_strdup(buf.data); diff -r 7a4dd8cda3b6 src/bin/psql/print.c --- a/src/bin/psql/print.c Wed May 27 12:28:55 2009 + +++ b/src/bin/psql/print.c Wed May 27 21:58:15 2009 +0200 @@ -2348,7 +2348,7 @@ char default_footer[100]; total_records = opt-topt.prior_records + cont.nrows; - snprintf(default_footer, 100, ngettext((1 row), (%lu rows), total_records), total_records); + snprintf(default_footer, 100, ngettext((%lu row), (%lu rows), total_records), total_records); printTableAddFooter(cont, default_footer); } -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] problem with plural-forms
Here is output of: for FILE in `find . -name *.po`;do LC_ALL=C msgfmt -v -o /dev/null $FILE 2 msgfmt.txt; done Zdenek Peter Eisentraut píše v st 27. 05. 2009 v 23:08 +0300: On Monday 25 May 2009 19:11:24 Zdenek Kotala wrote: The problem here is (1 row) instead of (%lu row). When I run msgfmt without -v everything works fine but I think we should fixed it (there are more occurrences of this issue). I don't think we can find all these occurrences without the Solaris version of msgfmt. So please send a complete error log over all files, or better yet a patch. Processing file ./src/bin/initdb/po/ja.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/ja.po... 129 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/sl.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/sl.po... 99 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/es.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/es.po... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/ro.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/ro.po... 107 translated message(s), 0 fuzzy translation(s), 1 untranslated message(s). Processing file ./src/bin/initdb/po/sk.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/sk.po... 82 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/pt_BR.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/pt_BR.po... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/de.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/de.po... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/cs.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/cs.po... 83 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/ta.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/ta.po... 123 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/sv.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/sv.po... 126 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/it.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/it.po... 92 translated message(s), 34 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/ru.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/ru.po... 128 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/pl.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/pl.po... 96 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/zh_TW.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/zh_TW.po... 96 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/tr.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/tr.po... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/zh_CN.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/zh_CN.po... 96 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/fr.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/fr.po... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/initdb/po/ko.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/initdb/po/ko.po... 116 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file ./src/bin/pg_controldata/po/sk.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file ./src/bin/pg_controldata/po/sk.po... 41 translated message(s), 0 fuzzy translation(s
Re: [HACKERS] problem with plural-forms
Peter Eisentraut píše v út 26. 05. 2009 v 13:39 +0300: On Monday 25 May 2009 19:11:24 Zdenek Kotala wrote: snip The problem here is (1 row) instead of (%lu row). When I run msgfmt without -v everything works fine but I think we should fixed it (there are more occurrences of this issue). GNU gettext accepts this, and in fact the GNU gettext documentation explicitly points out that this allowed: In the English singular case, the number - always 1 - can be replaced with one: printf (ngettext (One file removed, %d files removed, n), n); This works because the `printf' function discards excess arguments that are not consumed by the format string. Yeah, I check also printf specification and it is allowed. One might consider this better style (English style, not C style) in some contexts. Of course the concrete example that you show doesn't actually take advantage of this, so if it is important to you, please send a patch to fix it. It is not a big issue, because it works without -v but I prefer to fix it. I will send a patch. I also sended question to i18n solaris group if it is supported on solaris. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Tom Lane píše v po 25. 05. 2009 v 13:07 -0400: Zdenek Kotala zdenek.kot...@sun.com writes: Tom Lane píše v ne 24. 05. 2009 v 18:46 -0400: In any case, the barriers to implementing 8.3-style hash indexes in 8.4 are pretty huge: you'd need to duplicate not only the hash AM code, but also all the hash functions, and therefore all of the hash pg_amop and pg_amproc entries. I'm not sure if I need duplicate functions. Generally yes but It seems to me that hash index does not changed functions behavior and they could be shared at this moment. No, the behavior of the hash functions themselves changed during 8.4. Twice, even: hmm, I'm missed it. :( So as far as I can see, you need completely separate copies of both hash_any() and the SQL-level functions that call it. I'm not really seeing that the proposed refactoring makes this any easier. You might as well just copy-and-paste all that old code into a separate set of files, and not worry about what is in access/hash.h. Yeah, in this case everything have to be duplicated which is not big deal in comparison to do same amount of work for GIN. Then I can start with GIN. The advantage of refactoring is then only nicer code. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Compiler warning cleanup
Peter Eisentraut píše v ne 24. 05. 2009 v 00:40 +0300: I think this is not the best way to do it, because it will confuse pgindent and editors and such. The DATA() macros in include/catalog have this solved; see include/catalog/genbki.sh. Attached variant with faked extern without ; which is what genbki.sh uses. Zdenek diff -Nrc pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsquery_op.c pgsql.orig/src/backend/utils/adt/tsquery_op.c *** pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsquery_op.c 2009-05-25 16:45:34.594747865 +0200 --- pgsql.orig/src/backend/utils/adt/tsquery_op.c 2009-05-25 16:45:34.599072981 +0200 *** *** 188,194 PG_FREE_IF_COPY(b,1); \ \ PG_RETURN_BOOL( CONDITION );\ ! } CMPFUNC(tsquery_lt, res 0); CMPFUNC(tsquery_le, res = 0); --- 188,196 PG_FREE_IF_COPY(b,1); \ \ PG_RETURN_BOOL( CONDITION );\ ! }\ ! /* keep compiler quiet - no extra ; */ \ ! extern int no_such_variable CMPFUNC(tsquery_lt, res 0); CMPFUNC(tsquery_le, res = 0); diff -Nrc pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsvector_op.c pgsql.orig/src/backend/utils/adt/tsvector_op.c *** pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsvector_op.c 2009-05-25 16:45:34.596082053 +0200 --- pgsql.orig/src/backend/utils/adt/tsvector_op.c 2009-05-25 16:45:34.599263998 +0200 *** *** 172,178 PG_FREE_IF_COPY(a,0);\ PG_FREE_IF_COPY(b,1);\ PG_RETURN_##ret( res action 0 ); \ ! } TSVECTORCMPFUNC(lt, , BOOL); TSVECTORCMPFUNC(le, =, BOOL); --- 172,180 PG_FREE_IF_COPY(a,0);\ PG_FREE_IF_COPY(b,1);\ PG_RETURN_##ret( res action 0 ); \ ! } \ ! /* keep compiler quiet - no extra ; */ \ ! extern int no_such_variable TSVECTORCMPFUNC(lt, , BOOL); TSVECTORCMPFUNC(le, =, BOOL); -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Tom Lane píše v ne 24. 05. 2009 v 18:46 -0400: Kenneth Marshall k...@rice.edu writes: On Sat, May 23, 2009 at 02:52:49PM -0400, Zdenek Kotala wrote: Attached patch cleanups hash index headers to allow compile hasham for 8.3 version. It helps to improve pg_migrator with capability to migrate database with hash index without reindexing. How does that work with the updated hash functions without a reindex? I looked at this patch and I don't see how it helps pg_migrator at all. It's just pushing some code and function declarations around. The main important thing is move hash_any/hash_uint32 function from hash am. It should reduce amount of duplicated code necessary for old hash index implementation. Rest is only rearrangement and cleanup. The rearrangement might be marginally nicer from a code beautification point of view --- right now we're a bit inconsistent about whether datatype-specific hash functions live in hashfunc.c or in the datatype's utils/adt/ file. I personally prefer to keep it in type definition. AM should be independent on types which are installed and data type code should be self contained. But I'm not sure that removing hashfunc.c altogether is an appropriatera solution to that, not least because of the loss of CVS history for the functions. I'd be inclined to leave the core hash_any() code where it is, if not all of these functions altogether. Until we will have better version control system, hashfunc.c can stay here, but what I need is hashfunc.h. Minimalistic version of this patch is to create hashfuct.h and modified related #include in C and header files. What does seem useful is to refactor the headers so that datatype hash functions don't need to include all of the AM's implementation details. But this patch seems to do both more and less than that --- I don't think it's gotten rid of all external #includes of access/hash.h, and in any case moving the function code is not necessary to that goal. Agree, I will prepare minimalistic version with hashfunc.h only. In any case, the barriers to implementing 8.3-style hash indexes in 8.4 are pretty huge: you'd need to duplicate not only the hash AM code, but also all the hash functions, and therefore all of the hash pg_amop and pg_amproc entries. I'm not sure if I need duplicate functions. Generally yes but It seems to me that hash index does not changed functions behavior and they could be shared at this moment. Given the close-to-zero usefulness of hash indexes in production installations, I don't think it's worth the trouble. It would be much more helpful to look into supporting 8.3-compatible GIN indexes. Agree, I wanted to quickly verify function naming collision problem and HASH index seems to me better candidate for this basic test. GIN has priority. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] problem with plural-forms
I tried to run msgfmt -v ... on solaris and I got following error: Processing file psql-cs.po... GNU PO file found. Generating the MO file in the GNU MO format. Processing file psql-cs.po... Lines 1311, 1312 (psql-cs.po): incompatible printf-format. 0 format specifier(s) in msgid, but 1 format specifier(s) in msgstr. ... ... Problem is in: #: print.c:2351 #, c-format msgid (1 row) msgid_plural (%lu rows) msgstr[0] (%lu řádka) msgstr[1] (%lu řádky) msgstr[2] (%lu řádek) The problem here is (1 row) instead of (%lu row). When I run msgfmt without -v everything works fine but I think we should fixed it (there are more occurrences of this issue). Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
I forgot to fix contrib. Updated patch attached. Zdenek Zdenek Kotala píše v pá 22. 05. 2009 v 16:23 -0400: Attached patch cleanups hash index headers to allow compile hasham for 8.3 version. It helps to improve pg_migrator with capability to migrate database with hash index without reindexing. I discussed this patch year ago with Alvaro when we tried to cleanup include bloating problem. It should reduce also number of including. The main point is that hash functions for datatypes are now in related data files in utils/adt directory. hash_any() and hash_uint32 it now in utils/hashfunc.c. It would be nice to have this in 8.4 because it allows to test index migration functionality. Thanks Zdenek hash_02.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql is broken in 8.4
Heikki Linnakangas píše v čt 21. 05. 2009 v 16:53 +0300: Zdenek Kotala wrote: Another problem is with resultset. When I run for example following command I got this output: postgres=# select oid from pg_am; oid -- 403 405 783 2742 That's odd. Work for me. Can you try to debug that, please? We looked on it with Peter and it is problem with plural translation of rows for Czech language. I'm going to fix it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] integer overflow in reloption.h
When I compile postgresql now I get following message:
../../../src/include/access/reloptions.h, line 45: warning: integer
overflow detected: op
The problem is on the following lines
typedef enum relopt_kind
{
...
RELOPT_KIND_MAX = (1 31)
}
enum is int datatype and 1 31 == -2147483648. It is reason why
compiler (sun studio) complains.
Is possible to change it to 1 30 to stop compiler generates noise?
Thanks Zdenek
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Attached patch cleanups hash index headers to allow compile hasham for
8.3 version. It helps to improve pg_migrator with capability to migrate
database with hash index without reindexing.
I discussed this patch year ago with Alvaro when we tried to cleanup
include bloating problem. It should reduce also number of including.
The main point is that hash functions for datatypes are now in related
data files in utils/adt directory. hash_any() and hash_uint32 it now in
utils/hashfunc.c.
It would be nice to have this in 8.4 because it allows to test index
migration functionality.
Thanks Zdenek
diff -Nrc pgsql_indexcompat.5d4d60e3a557/src/backend/access/hash/hashfunc.c pgsql_indexcompat/src/backend/access/hash/hashfunc.c
*** pgsql_indexcompat.5d4d60e3a557/src/backend/access/hash/hashfunc.c 2009-05-22 15:56:34.409314434 -0400
--- pgsql_indexcompat/src/backend/access/hash/hashfunc.c 1969-12-31 19:00:00.0 -0500
***
*** 1,528
- /*-
- *
- * hashfunc.c
- * Support functions for hash access method.
- *
- * Portions Copyright (c) 1996-2009, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- *
- * IDENTIFICATION
- * $PostgreSQL: pgsql/src/backend/access/hash/hashfunc.c,v 1.57 2009/01/01 17:23:35 momjian Exp $
- *
- * NOTES
- * These functions are stored in pg_amproc. For each operator class
- * defined for hash indexes, they compute the hash value of the argument.
- *
- * Additional hash functions appear in /utils/adt/ files for various
- * specialized datatypes.
- *
- * It is expected that every bit of a hash function's 32-bit result is
- * as random as every other; failure to ensure this is likely to lead
- * to poor performance of hash joins, for example. In most cases a hash
- * function should use hash_any() or its variant hash_uint32().
- *-
- */
-
- #include postgres.h
-
- #include access/hash.h
-
-
- /* Note: this is used for both char and boolean datatypes */
- Datum
- hashchar(PG_FUNCTION_ARGS)
- {
- return hash_uint32((int32) PG_GETARG_CHAR(0));
- }
-
- Datum
- hashint2(PG_FUNCTION_ARGS)
- {
- return hash_uint32((int32) PG_GETARG_INT16(0));
- }
-
- Datum
- hashint4(PG_FUNCTION_ARGS)
- {
- return hash_uint32(PG_GETARG_INT32(0));
- }
-
- Datum
- hashint8(PG_FUNCTION_ARGS)
- {
- /*
- * The idea here is to produce a hash value compatible with the values
- * produced by hashint4 and hashint2 for logically equal inputs; this is
- * necessary to support cross-type hash joins across these input types.
- * Since all three types are signed, we can xor the high half of the int8
- * value if the sign is positive, or the complement of the high half when
- * the sign is negative.
- */
- #ifndef INT64_IS_BUSTED
- int64 val = PG_GETARG_INT64(0);
- uint32 lohalf = (uint32) val;
- uint32 hihalf = (uint32) (val 32);
-
- lohalf ^= (val = 0) ? hihalf : ~hihalf;
-
- return hash_uint32(lohalf);
- #else
- /* here if we can't count on x 32 to work sanely */
- return hash_uint32((int32) PG_GETARG_INT64(0));
- #endif
- }
-
- Datum
- hashoid(PG_FUNCTION_ARGS)
- {
- return hash_uint32((uint32) PG_GETARG_OID(0));
- }
-
- Datum
- hashenum(PG_FUNCTION_ARGS)
- {
- return hash_uint32((uint32) PG_GETARG_OID(0));
- }
-
- Datum
- hashfloat4(PG_FUNCTION_ARGS)
- {
- float4 key = PG_GETARG_FLOAT4(0);
- float8 key8;
-
- /*
- * On IEEE-float machines, minus zero and zero have different bit patterns
- * but should compare as equal. We must ensure that they have the same
- * hash value, which is most reliably done this way:
- */
- if (key == (float4) 0)
- PG_RETURN_UINT32(0);
-
- /*
- * To support cross-type hashing of float8 and float4, we want to return
- * the same hash value hashfloat8 would produce for an equal float8 value.
- * So, widen the value to float8 and hash that. (We must do this rather
- * than have hashfloat8 try to narrow its value to float4; that could fail
- * on overflow.)
- */
- key8 = key;
-
- return hash_any((unsigned char *) key8, sizeof(key8));
- }
-
- Datum
- hashfloat8(PG_FUNCTION_ARGS)
- {
- float8 key = PG_GETARG_FLOAT8(0);
-
- /*
- * On IEEE-float machines, minus zero and zero have different bit patterns
- * but should compare as equal. We must ensure that they have the same
- * hash value, which is most reliably done this way:
- */
- if (key == (float8) 0)
- PG_RETURN_UINT32(0);
-
- return hash_any((unsigned char *) key, sizeof(key));
- }
-
- Datum
- hashoidvector(PG_FUNCTION_ARGS)
- {
- oidvector *key = (oidvector *) PG_GETARG_POINTER(0);
-
- return hash_any((unsigned char *) key-values, key-dim1 * sizeof(Oid));
- }
-
- Datum
- hashint2vector(PG_FUNCTION_ARGS)
- {
- int2vector *key = (int2vector *) PG_GETARG_POINTER(0);
-
[HACKERS] [PATCH] Compiler warning cleanup
Attached patch fixes following sun studio compiler warning: tsquery_op.c, line 193: warning: syntax error: empty declaration tsquery_op.c, line 194: warning: syntax error: empty declaration tsquery_op.c, line 195: warning: syntax error: empty declaration tsquery_op.c, line 196: warning: syntax error: empty declaration tsquery_op.c, line 197: warning: syntax error: empty declaration tsquery_op.c, line 198: warning: syntax error: empty declaration tsvector_op.c, line 177: warning: syntax error: empty declaration tsvector_op.c, line 178: warning: syntax error: empty declaration tsvector_op.c, line 179: warning: syntax error: empty declaration tsvector_op.c, line 180: warning: syntax error: empty declaration tsvector_op.c, line 181: warning: syntax error: empty declaration tsvector_op.c, line 182: warning: syntax error: empty declaration tsvector_op.c, line 183: warning: syntax error: empty declaration Thanks Zdenek diff -Nrc pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsquery_op.c pgsql.orig/src/backend/utils/adt/tsquery_op.c *** pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsquery_op.c 2009-05-22 17:00:22.081796110 -0400 --- pgsql.orig/src/backend/utils/adt/tsquery_op.c 2009-05-22 17:00:22.086155307 -0400 *** *** 190,201 PG_RETURN_BOOL( CONDITION );\ } ! CMPFUNC(tsquery_lt, res 0); ! CMPFUNC(tsquery_le, res = 0); ! CMPFUNC(tsquery_eq, res == 0); ! CMPFUNC(tsquery_ge, res = 0); ! CMPFUNC(tsquery_gt, res 0); ! CMPFUNC(tsquery_ne, res != 0); TSQuerySign makeTSQuerySign(TSQuery a) --- 190,201 PG_RETURN_BOOL( CONDITION );\ } ! CMPFUNC(tsquery_lt, res 0) ! CMPFUNC(tsquery_le, res = 0) ! CMPFUNC(tsquery_eq, res == 0) ! CMPFUNC(tsquery_ge, res = 0) ! CMPFUNC(tsquery_gt, res 0) ! CMPFUNC(tsquery_ne, res != 0) TSQuerySign makeTSQuerySign(TSQuery a) diff -Nrc pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsvector_op.c pgsql.orig/src/backend/utils/adt/tsvector_op.c *** pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsvector_op.c 2009-05-22 17:00:22.083116270 -0400 --- pgsql.orig/src/backend/utils/adt/tsvector_op.c 2009-05-22 17:00:22.086348815 -0400 *** *** 174,186 PG_RETURN_##ret( res action 0 ); \ } ! TSVECTORCMPFUNC(lt, , BOOL); ! TSVECTORCMPFUNC(le, =, BOOL); ! TSVECTORCMPFUNC(eq, ==, BOOL); ! TSVECTORCMPFUNC(ge, =, BOOL); ! TSVECTORCMPFUNC(gt, , BOOL); ! TSVECTORCMPFUNC(ne, !=, BOOL); ! TSVECTORCMPFUNC(cmp, +, INT32); Datum tsvector_strip(PG_FUNCTION_ARGS) --- 174,186 PG_RETURN_##ret( res action 0 ); \ } ! TSVECTORCMPFUNC(lt, , BOOL) ! TSVECTORCMPFUNC(le, =, BOOL) ! TSVECTORCMPFUNC(eq, ==, BOOL) ! TSVECTORCMPFUNC(ge, =, BOOL) ! TSVECTORCMPFUNC(gt, , BOOL) ! TSVECTORCMPFUNC(ne, !=, BOOL) ! TSVECTORCMPFUNC(cmp, +, INT32) Datum tsvector_strip(PG_FUNCTION_ARGS) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] psql is broken in 8.4
Hi all, last version of psql is broken: psql (8.4beta1, server 8.3.7) WARNING: psql version 8.4, server version 8.3. Some psql features might not work. Type help for help. postgres=# \d test ERROR: syntax error at or near , LINE 1: ...index, relhasrules, reltriggers 0, relhasoids, ,'', relta... ^ It should work. I think describeTableDetails() does not have something like: if (pset.sversion = 80100) ... Another problem is with resultset. When I run for example following command I got this output: postgres=# select oid from pg_am; oid -- 403 405 783 2742 See the garbage at the end. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql is broken in 8.4
Heikki Linnakangas píše v čt 21. 05. 2009 v 16:53 +0300: Zdenek Kotala wrote: last version of psql is broken: psql (8.4beta1, server 8.3.7) WARNING: psql version 8.4, server version 8.3. Some psql features might not work. Type help for help. postgres=# \d test ERROR: syntax error at or near , LINE 1: ...index, relhasrules, reltriggers 0, relhasoids, ,'', relta... ^ It should work. I think describeTableDetails() does not have something like: if (pset.sversion = 80100) ... I fixed this a while ago: revision 1.211 date: 2009/05/04 17:31:35; author: heikki; state: Exp; lines: +2 -2 Fix the query used for \d against 8.2 and 8.3 servers. Should work with a fresh checkout. Yeah, thanks. It works with beta2. Another problem is with resultset. When I run for example following command I got this output: postgres=# select oid from pg_am; oid -- 403 405 783 2742 That's odd. Work for me. Can you try to debug that, please? I will look on it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
