Re: [HACKERS] Further news on Clang - spurious warnings
On 3 Aug 2011, at 19:20, David E. Wheeler wrote: Yeah, not sure whether or not to tweak the Makefile to use Clang. I guess not? export CC=clang ./configure ... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sync rep fsync=off
On 18 Mar 2011, at 21:12, Robert Haas wrote: While investigating Simon's complaint about my patch of a few days ago, I discovered that synchronous replication appears to slow to a crawl if fsync is turned off on the standby. I'm not sure why this is happening or what the right behavior is in this case, but I think some kind of adjustment is needed because the current behavior is quite surprising. We have few servers here running 8.3. And few weeks ago I had to populate one database with quite a number of entries. I have script that does that, but it takes a while. I decided to turn fsck to off. Oddly enough, the server started to crawl quite badly, load was very high. That was 8.3 on rhel 5.4. My point is, it is sometimes bad combination of disks and controllers that does that. Not necessarily software. fsync off doesn't always mean that things are going to fly, it can cause it to expose hardware bottlenecks much quicker. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: cross column correlation ...
On 25 Feb 2011, at 13:18, Robert Haas wrote: People coming from Oracle are not favorably impressed either by the amount of monitoring data PostgreSQL can gather or by the number of knobs that are available to fix problems when they occur. We don't need to have as many knobs as Oracle and we probably don't want to, and for that matter we probably couldn't if we did want to for lack of manpower, but that doesn't mean we should have none. Still, having more data a user can probe would be nice. I wonder why everyone avoids Microsoft's approach to the subject. Apparently, they go in the 'auto-tune as much as possible' direction. And tests we did a while ago, involving asking team from Microsoft and a team from oracle to optimise set of queries for the same set of data (bookies data, loads of it) showed that the auto-tuning Microsoft has in their sql server performed much better than a team of over-sweating oracle dba's. In my current work place/camp we have many deployments of the same system, over different types of machines, each with different customer data that vary so much that queries need to be rather generic. Postgresql shows its strength with planner doing a good job for different variants of data, however we do a very little tweaking to the configuration parameters. Just because it is just too hard to overlook all of them. I guess that the systems could behave much better, but no one is going to tweak settings for 50 different installations over 50 different type of data and 50 different sets of hardware. If there was even a tiny amount of automation provided in the postgresql, I would welcome it with open arms. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: cross column correlation ...
On 26 Feb 2011, at 14:45, Robert Haas wrote: On Sat, Feb 26, 2011 at 4:33 AM, Grzegorz Jaskiewicz I don't think *anyone* is avoiding that approach. There is almost universal consensus here that auto-tuning is better than manual tuning, even to the extent of being unwilling to add knobs to allow manual tuning of settings we have no idea how to auto-tune and no plans to auto-tune. Perhaps one step further is required. To change some settings so that it can be auto-tuned better. There are some even more drastic steps that would have to be taken and I believe that Microsoft engineers had to take them. Steps back. For instance, if there is an issue with inability to find out how much of a table is in the cache, perhaps postgresql should have an option to turn off cached reads/writes completely and thus allow DBA to regulate that using the shared_buffers setting. It doesn't sound great, but if you think about it I'm sure there are people willing to use it, if that adds a bit more auto-tunning to the server. I would even go a step further, and say that I believe that some people will embrace it on the basis that they can constraint the amount of memory PostgreSQL uses on their server as a whole, and that includes caches. In my current work place/camp we have many deployments of the same system, over different types of machines, each with different customer data that vary so much that queries need to be rather generic. Postgresql shows its strength with planner doing a good job for different variants of data, however we do a very little tweaking to the configuration parameters. Just because it is just too hard to overlook all of them. I guess that the systems could behave much better, but no one is going to tweak settings for 50 different installations over 50 different type of data and 50 different sets of hardware. If there was even a tiny amount of automation provided in the postgresql, I would welcome it with open arms. What do you have in mind? All I'm trying to say, that whilst you guys focus mostly on single database server installations PostgreSQL has also a great user base that use it as part of a product that is deployed on different sized machines, and with same model but different data variation. We don't sell the product to the people and let them take care of it, but rather sell the service - you would say. But we also don't have a DBA per customer that would look solely at the knob tweaking side of things. So my argument here is, that there isn't always a person who would know tables and databases by their characteristics and thus be able to tweak settings manually. That probably is just a one of many examples where it makes sense, and probably their primary property is that there's no DBA overlooking whole database and thus being able to tune it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] double and numeric conversion
if (p1 buf)
++ * --p1;
else {
++ * --p1; ???
does it even compile ?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] double and numeric conversion
On 3 Mar 2010, at 17:41, Grzegorz Jaskiewicz wrote:
if (p1 buf)
++ * --p1;
else {
++ * --p1; ???
does it even compile ?
Oh, I can see, that it is *(--p1)++ ,mea culpa.
Which doesn't change the fact, that the code is rather messy imo.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 vs. 9.0, Postgres vs. PostgreSQL
think also how people use SQL word , when calling ms sql server. They would just say 'sql server' , and to some I had to explain that the little greedy company didn't actually invented sql, hence it should be called ms sql server... so, -1 for dropping SQL word from me. ... and maybe the shed should be yellow, and with flat roof... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 vs. 9.0
On 21 Jan 2010, at 09:37, Dave Page wrote: In an attempt to pre-empt the normally drawn-out discussions about what the next version of PostgreSQL will be numbered. the core team have discussed the issue and following a lenghty debate lasting literally a few minutes decided that the next release shall be Wait for it 9.0. darn, looks like PostgreSQL developers can only count to 4 :) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] DEFAULT of domain ignored in plpgsql (8.4.1)
On 21 Nov 2009, at 02:56, Josh Berkus wrote: Would a patch that changes that have any chance of being accepted? Or is the gain (not having to repeat the DEFAULT clause, and being able to maintain it at one place instead of many) considered too small compared to the risk of breaking existing code? I don't think there's a lot of risk of code breakage; few people use domains, fewer use them with defaults, and you might be the only one using them as variable types. And there are going to be more substantial backwards compat issues with the lexer changes anyway. As long as we remember to flag the compatibility issue in the release notes, I don't see it as a problem. we use domains with defaults, a lot. That's one of the purposes of domains, to have certain type, constraint, and default. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs head doesn't pass make check on one of the machines here
On 13 Nov 2009, at 19:39, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On fre, 2009-11-13 at 15:05 +0100, Grzegorz Jaśkiewicz wrote: As per Tom's - yes, this laptop has LANG set to UTF8 Polish. Setting it back to EN actually makes this error go away. The Polish locale isn't supported by the regression tests. With only one result-ordering difference, it seems like we could easily support that if there were enough demand. I'd want somebody running a buildfarm machine in Polish locale, though, to catch future breakages. Setting the LANG variable doesn't change behaviour, unfortunately - and it passes on other computers. But changing it on the machine with Polish locale does make it pass, weird. (I barely use Polish locales, even tho I was born long time ago in that country, so it is hard to test it for me - on daily basis. But I will try to re-test it every so often). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] cvs head doesn't pass make check on one of the machines here
consistently fails when compiled on ubuntu 9.10 here (on mini 10v). regression.diffs.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Syntax for partitioning
On 29 Oct 2009, at 02:15, Itagaki Takahiro wrote:
I'd like to improve partitioning feature in 8.5.
Kedar-san's previous work is wonderful, but I cannot see any updated
patch.
http://archives.postgresql.org/message-id/bd8134a40906080702s96c90a9q3bbb581b9bd0d...@mail.gmail.com
So, I'll take over the work if there are no ones to do it.
I'm thinking to add syntax support first. Table partitioning was
proposed many times, but it is still not applied into core.
The reason is it is too difficult to make perfect partitioning
feature at once. I think syntax support is a good start.
First, I will add syntax for CREATE TABLE, ALTER TABLE ADD/DROP
PARTITION.
The syntax is borrowed from from Oracle and MySQL. Their
characteristics
are using LESS THAN in range partitioning. The keyword PARTITION
is
added to the full-reserved keyword list to support ADD/DROP PARTITION.
Those syntax is merely a syntax sugar for INHERITS with CHECK.
Declarations
are translated into CHECK constraints. I have a plan to adjust
pg_dump to
dump definitions of partitioning in the correct format, but the actual
implementation will be still based on constraint exclusion. In
addition,
hash partitioning is not implemented; syntax is parsed but not
implemented
error are raised for now.
Here is syntax I propose:
ALTER TABLE table_name ADD PARTITION name ...;
ALTER TABLE table_name DROP PARTITION [IF EXISTS] name [CASCADE |
RESTRICT];
Range partitioning:
CREATE TABLE table_name ( columns )
PARTITION BY RANGE ( a_expr )
(
PARTITION name VALUES LESS THAN [(] const [)],
PARTITION name VALUES LESS THAN [(] MAXVALUE [)] -- overflow
partition
);
List partitioning:
CREATE TABLE table_name ( columns )
PARTITION BY LIST ( a_expr )
(
PARTITION name VALUES [IN] ( const [, ...] ),
PARTITION name VALUES [IN] [(] DEFAULT [)] -- overflow
partition
);
Hash partitioning:
CREATE TABLE table_name ( columns )
PARTITION BY HASH ( a_expr )
PARTITIONS num_partitions;
CREATE TABLE table_name ( columns )
PARTITION BY HASH ( a_expr )
(
PARTITION name,
...
);
Note:
* Each partition can have optional WITH (...) and TABLESPACE clauses.
* '(' and ')' are optional to support both Oracle and MySQL syntax.
Comments welcome.
+1000
Thanks !
(most anticipated feature for 8.5, here, next to replication [well, I
am interested in multi master, but that's not going to happen :P ] )
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing config files in a directory
On 24 Oct 2009, at 14:41, Magnus Hagander wrote: Per discussion at the developer meeting back in Ottawa, attached is an initial patch that implements reading a directory of configuration files instead of just one. The idea being that something like a tuning tool, or pgadmin, for example can drop and modify files in this directory instead of modifying the main config file (which can be very hard to machine-parse). The idea is the same as other software like apache that parses multiple files. Files are parsed in alphabetical order so it's predictable, and you can make sure some files override others etc. Comments, before I go do the final polishing? :-) I don't know what the discussion topics were, since I was not there. But primary question is, cannot that be achieved with simple includes in postgresql.conf ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
new version under: http://zlew.org/postgresql_static_check/scan-build-2009-10-03-1/ What's strange, is the increase of 48.2 percent in reports, that happened about two weeks before (weekend before the previous one). enjoy. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
http://llvm.org/bugs/show_bug.cgi?id=4979 will see, one issue is already fixed. I'll retry when the second one is too. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 14 Sep 2009, at 06:04, Tom Lane wrote: Looks like the clang guys still have some work to do. Thanks Tom, reported to clang dev's . meanwhile, since quite a lot stuff went in over weekend, and since Yesterday, new report at: http://zlew.org/postgresql_static_check/scan-build-2009-09-14-1/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for automating partitions in PostgreSQL 8.4 Beta 2
Anyone knows what's the latest on that patch ? To be honest, this was the thing that I was looking forward most in 8.5 ... (and probably not only me alone). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
usual round of updates to the scan report. Today's report available at: http://zlew.org/postgresql_static_check/scan-build-2009-09-12-1/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
clang developers were quick to iron out their bugs, here's Today report: http://zlew.org/postgresql_static_check/scan-build-2009-09-04-1/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
new report: http://zlew.org/postgresql_static_check/scan-build-2009-09-02-1/ archive one dir up, as usual (with index of all previous reports). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 2 Sep 2009, at 21:38, Alvaro Herrera wrote: Grzegorz Jaskiewicz escribió: new report: http://zlew.org/postgresql_static_check/scan-build-2009-09-02-1/ archive one dir up, as usual (with index of all previous reports). What's with the analyzer failures? Did you submit bug reports about them? honestly, why would you even ask me such a question My currently open bugs: http://llvm.org/bugs/show_bug.cgi?id=4867 http://llvm.org/bugs/show_bug.cgi?id=4832 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 29 Aug 2009, at 18:05, Greg Stark wrote: Oh, I think I see what's happening. Our assertions can still be turned off at run-time with the variable assert_enabled. Index: src/include/postgres.h === RCS file: /projects/cvsroot/pgsql/src/include/postgres.h,v retrieving revision 1.92 diff -b -u -r1.92 postgres.h --- src/include/postgres.h 1 Jan 2009 17:23:55 - 1.92 +++ src/include/postgres.h 30 Aug 2009 11:17:50 - @@ -639,6 +639,7 @@ */ extern PGDLLIMPORT bool assert_enabled; +#define assert_enabled (1) /* * USE_ASSERT_CHECKING, if defined, turns on all the assertions. @@ -666,7 +667,7 @@ * Isn't CPP fun? */ #define TrapMacro(condition, errorType) \ - ((bool) ((! assert_enabled) || ! (condition) || \ + ((bool) ( ! (condition) || \ (ExceptionalCondition(CppAsString(condition), (errorType), \ __FILE__, __LINE__ @@ -689,8 +690,10 @@ Trap(!(condition), BadState) #endif /* USE_ASSERT_CHECKING */ +#undef assert_enabled + extern int ExceptionalCondition(const char *conditionName, const char *errorType, -const char *fileName, int lineNumber); + const char *fileName, int lineNumber) __attribute__ ((analyzer_noreturn)); like that ? This is another excerpt from my local mods, that I use before running clang-checker over it. but looking at Assert() macros in code (it expands macros if you hoover mouse pointer over one) - it still keeps 'assert_enabled' literal there. damn... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
okay, I think I nailed the assert part right. (3rd iteration, about time...). as usual: http://zlew.org/postgresql_static_check/scan-build-2009-08-30-2/ archive one dir up. the current patch attached. postgres_checker_patch.patch.bz2 Description: BZip2 compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 30 Aug 2009, at 15:46, Greg Stark wrote: So three of the four dead initialization warnings are legitimate -- if minor -- errors. Attached is a patch to remove the redundant initializations. well, all I can do is this: http://llvm.org/bugs/show_bug.cgi?id=4832 I find it hard to belive tho, that it only found 4 possible positives ;) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
with Greg's suggested palloc and friends patch: http://zlew.org/postgresql_static_check/scan-build-2009-08-30-3 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 30 Aug 2009, at 18:07, Greg Stark wrote: On Sun, Aug 30, 2009 at 5:58 PM, Grzegorz Jaskiewiczg...@pointblue.com.pl wrote: with Greg's suggested palloc and friends patch: http://zlew.org/postgresql_static_check/scan-build-2009-08-30-3 Argh. That didn't help at all. hm, I suppose instead of (exit(1),NULL) we could just put ((void*)1) there? But I think Tom's right. Worse, I think until it can do inter-procedural analysis these messages will always be nearly all false positives. Many if not most of our functions take pointers or data structures which contain pointers as arguments or return values. Most of the time those arguments and return values cannot contain NULL pointers and the code doesn't bother to check that every single time. sure, I can try. Btw, I got response to my bug from llvm devs, and they fully agree on all that. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
http://zlew.org/postgresql_static_check/scan-build-2009-08-30-4/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
please don't tell me that this is bogus: http://zlew.org/postgresql_static_check/scan-build-2009-08-30-4/report-7JaICX.html#EndPath ? How come ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 30 Aug 2009, at 19:14, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: please don't tell me that this is bogus: http://zlew.org/postgresql_static_check/scan-build-2009-08-30-4/report-7JaICX.html#EndPath Yes, it's bogus. ctid is never passed as NULL. It might point at an invalid itempointer (one with ip_posid == 0). Look at the only call site. so why do we check if the pointer is valid ? This seems to indicate that clang is stupider than I would have hoped. Isn't it capable of doing the same types of analysis that gcc does when inlining? well, it is usable, but that doesn't mean complete. That's why I am trying to work both ways to provide some info to clang-checker devs, and you guys - with the reports it generates. on the side note, xcode in snow leopard uses it under the hood to do 'build analyze', and it helped me to locate few potential issues in my iphone code. Now, of course my code is times less complicated than postgresql's, but still - it is potentially useful. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
new one at http://zlew.org/postgresql_static_check/scan-build-2009-08-29-3/ archive at : http://zlew.org/postgresql_static_check/postgresql_static_check_29thAugust2009.tar.xz as always, comments are welcomed. And constructive explanation of any wrong-results will be relayed to clang-checker developer(s). hth. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 29 Aug 2009, at 17:35, Greg Stark wrote: We still have things like this showing division by zero: Assert(activeTapes 0); 1913 slotsPerTape = (state-memtupsize - state-mergefirstfree) / activeTapes; It looks like if you marked ExceptionalCondition() as never returning then it should hide this. well, it is marked as such , here's excerpt from differences to head: extern int ExceptionalCondition(const char *conditionName, const char *errorType, -const char *fileName, int lineNumber); +const char *fileName, int lineNumber) __attribute__((analyzer_noreturn)); -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 24 Aug 2009, at 14:40, Peter Eisentraut wrote: On mån, 2009-08-24 at 00:42 +0100, Grzegorz Jaskiewicz wrote: --enable-cassert, enabled, and also added exit_* in pg_dump to list of functions that never return. A few more functions to mark noreturn: DateTimeParseError(), and die_horribly() in pg_dump done. new scan at: http://zlew.org/postgresql_static_check/scan-build-2009-08-27-2/ archive at: http://zlew.org/postgresql_static_check/postgresql_static_check_27thAugust2009.tar.xz If people find it useful (altho, I've only seen single commit as result of that checker, and nothing fancy either) - I can write a script that would update it on daily basis. what you people say ? New Patch : Index: src/Makefile.global.in === RCS file: /projects/cvsroot/pgsql/src/Makefile.global.in,v retrieving revision 1.258 diff -u -b -r1.258 Makefile.global.in --- src/Makefile.global.in 26 Aug 2009 22:24:42 - 1.258 +++ src/Makefile.global.in 27 Aug 2009 11:54:36 - @@ -205,7 +205,10 @@ endif endif # not PGXS +ifndef CC CC = @CC@ +endif + GCC = @GCC@ SUN_STUDIO_CC = @SUN_STUDIO_CC@ CFLAGS = @CFLAGS@ Index: src/bin/pg_dump/pg_backup.h === RCS file: /projects/cvsroot/pgsql/src/bin/pg_dump/pg_backup.h,v retrieving revision 1.52 diff -u -b -r1.52 pg_backup.h --- src/bin/pg_dump/pg_backup.h 11 Jun 2009 14:49:07 - 1.52 +++ src/bin/pg_dump/pg_backup.h 27 Aug 2009 11:54:37 - @@ -150,7 +150,7 @@ extern void exit_horribly(Archive *AH, const char *modulename, const char *fmt,...) -__attribute__((format(printf, 3, 4))); +__attribute__((format(printf, 3, 4))) __attribute__ ((analyzer_noreturn)); /* Lets the archive know we have a DB connection to shutdown if it dies */ Index: src/bin/pg_dump/pg_backup_archiver.h === RCS file: /projects/cvsroot/pgsql/src/bin/pg_dump/pg_backup_archiver.h,v retrieving revision 1.82 diff -u -b -r1.82 pg_backup_archiver.h --- src/bin/pg_dump/pg_backup_archiver.h7 Aug 2009 22:48:34 - 1.82 +++ src/bin/pg_dump/pg_backup_archiver.h27 Aug 2009 11:54:37 - @@ -325,7 +325,7 @@ /* Used everywhere */ extern const char *progname; -extern void die_horribly(ArchiveHandle *AH, const char *modulename, const char *fmt,...) __attribute__((format(printf, 3, 4))); +extern void die_horribly(ArchiveHandle *AH, const char *modulename, const char *fmt,...) __attribute__((format(printf, 3, 4))) __attribute__((analyzer_noreturn)); extern void warn_or_die_horribly(ArchiveHandle *AH, const char *modulename, const char *fmt,...) __attribute__((format(printf, 3, 4))); extern void write_msg(const char *modulename, const char *fmt,...) __attribute__((format(printf, 2, 3))); Index: src/bin/pg_dump/pg_dump.h === RCS file: /projects/cvsroot/pgsql/src/bin/pg_dump/pg_dump.h,v retrieving revision 1.156 diff -u -b -r1.156 pg_dump.h --- src/bin/pg_dump/pg_dump.h 2 Aug 2009 22:14:52 - 1.156 +++ src/bin/pg_dump/pg_dump.h 27 Aug 2009 11:54:37 - @@ -481,7 +481,7 @@ extern void *pg_realloc(void *ptr, size_t size); extern void check_conn_and_db(void); -extern void exit_nicely(void); +extern void exit_nicely(void) __attribute__((analyzer_noreturn)); extern void parseOidArray(const char *str, Oid *array, int arraysize); Index: src/include/postgres.h === RCS file: /projects/cvsroot/pgsql/src/include/postgres.h,v retrieving revision 1.92 diff -u -b -r1.92 postgres.h --- src/include/postgres.h 1 Jan 2009 17:23:55 - 1.92 +++ src/include/postgres.h 27 Aug 2009 11:54:37 - @@ -691,6 +691,6 @@ extern int ExceptionalCondition(const char *conditionName, const char *errorType, -const char *fileName, int lineNumber); + const char *fileName, int lineNumber) __attribute__ ((analyzer_noreturn)); #endif /* POSTGRES_H */ Index: src/include/utils/datetime.h === RCS file: /projects/cvsroot/pgsql/src/include/utils/datetime.h,v retrieving revision 1.75 diff -u -b -r1.75 datetime.h --- src/include/utils/datetime.h11 Jun 2009 14:49:13 - 1.75 +++ src/include/utils/datetime.h27 Aug 2009 11:54:37 - @@ -300,7 +300,7 @@ int *dtype, struct pg_tm * tm, fsec_t *fsec); extern void DateTimeParseError(int dterr, const char *str, - const char *datatype); + const char *datatype) __attribute__((__noreturn__)); extern int DetermineTimeZoneOffset(struct pg_tm * tm, pg_tz *tzp); Index: src/include/utils/elog.h
Re: [HACKERS] Patches for static check on geo_ops.c
On 27 Aug 2009, at 10:46, Paul Matthews wrote: Grzegorz Jaskiewicz wonderful static checker coughed up 5 errors in geo_ops.c. None of them of any particular excitement or of earth shattering nature. A patch is attached below that should correct these. (The more little issue we eliminate, the more the large ones will stand out.) I am flattered, but I am merely a user of it - running it against postgresql's source. The checker is written by wonderful llvm developers, so you should thank them (and apple, for sponsoring development of the checker). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
heh, sorry folks for the noise again :/ There was a fair amount of false positives there - due to nature of Assert() macro. Mainly, since assert_enabled is a runtime variable, not a macro (which I sadly overlooked). So, hardcoding it to (1) (using CPP) removed quite few false positives. New results at: http://zlew.org/postgresql_static_check/scan-build-2009-08-27-4/ archive at: http://zlew.org/postgresql_static_check/postgresql_static_check_27thAugust2009_2.tar.xz Please tell me, if you think that it can be improved more. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] clang's static checker report.
So, after successful, and helpful Saturday with llvm's clang static checker, I decided to run it against postgresql's source code. Result can be seen at: http://zlew.org/postgresql_static_check/scan-build-2009-08-23-5/ . One directory below is the tar file, with the report. I am sure there's plenty of false positives, but I am also quite sure there's many real errors on that list. As I have rather bad experiences with any patches sent here - I hope that's least I can help with. To run clang-check I had to change one of the makefiles slightly, as the postgresql's build system seems to ignore $CC variable completely , always sticking to the one chosen by the configure script. The changed file is ./src/Makefile.global. Around line 210 I included ifdef CC, like that: ifndef CC CC = gcc -no-cpp-precomp GCC = yes endif Which later allowed me to run scan-build make without issues. hope that's helpfull. thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 23 Aug 2009, at 17:41, Greg Stark wrote: On Sun, Aug 23, 2009 at 4:57 PM, Grzegorz Jaskiewiczg...@pointblue.com.pl wrote: I am sure there's plenty of false positives, but I am also quite sure there's many real errors on that list. Do you know how to teach clang about functions which never return? That seems to be causing most of the false positives because it doesn't recognize that our error checks stop execution and avoid the use of the unitialized variables afterwards. I am not the clang developer, so I honestly have no idea how to do it. But as far as I checked report myself, there's couple 'division by zero', and 'null reference' errors that looked plausible to someone as unfamiliar with the postgresql's source as myself. Like with all static checkers, this one will generate a lot of false positives, and it is the inevitable cost of using such a tool having to go through all errors and sieve out positives yourself. You probably refer to the functions that never return. Sadly, even tho llvm clang is capable of doing so (one of its strengths is linking optimization) - the checker is unable to cross reference files, or so it seems. Well, like I said - l hope at least part of that report is useful. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 23 Aug 2009, at 17:41, Greg Stark wrote: Do you know how to teach clang about functions which never return? http://clang-analyzer.llvm.org/annotations.html#attr_noreturn -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
http://zlew.org/postgresql_static_check/scan-build-2009-08-23-5/report-MAVb5D.html#EndPath for a very positive one - at least from strict language point of view. consider: float f = 1; f++; printf(%f\n, f); -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
about the false positives around 'null reference'. I'll try sticking exit(1)'s at the end of each macro - and see if that makes most of them go away. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
memory leak: http://zlew.org/postgresql_static_check/scan-build-2009-08-23-5/report-46wcmJ.html#EndPath -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
this one should contain substantialy less false positives, because error functions were marked as the 'never exit' points: http://zlew.org/postgresql_static_check/scan-build-2009-08-23-9/ for the record, here's patch that marks elog, etc as dead ends: Index: src/include/utils/elog.h === RCS file: /projects/cvsroot/pgsql/src/include/utils/elog.h,v retrieving revision 1.101 diff -u -b -r1.101 elog.h --- src/include/utils/elog.h11 Jun 2009 14:49:13 - 1.101 +++ src/include/utils/elog.h23 Aug 2009 19:20:55 - @@ -112,8 +112,8 @@ #define TEXTDOMAIN NULL extern bool errstart(int elevel, const char *filename, int lineno, -const char *funcname, const char *domain); -extern void errfinish(int dummy,...); + const char *funcname, const char *domain) __attribute__((analyzer_noreturn)); +extern void errfinish(int dummy,...) __attribute__((analyzer_noreturn)); extern int errcode(int sqlerrcode); @@ -197,7 +197,7 @@ elog_finish(int elevel, const char *fmt,...) /* This extension allows gcc to check the format string for consistency with the supplied arguments. */ -__attribute__((format(printf, 2, 3))); +__attribute__((format(printf, 2, 3))) __attribute__((analyzer_noreturn)); /* Support for attaching context information to error reports */ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 23 Aug 2009, at 20:31, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: for the record, here's patch that marks elog, etc as dead ends: That does not look like the right thing at all, since now the checker will believe that elog(NOTICE) and such don't return. I think you need to use Martijn's suggestion instead. Still, there are few worrying finds on that list as it is anyway. I hope you guys will find it useful. I'll modify macro according to Martijn's suggesion, and rerun it again. My laptop is pretty slow, so it will be probably another 1-1.5h before I'll get it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
ok folks, here's the last one for Today: http://zlew.org/postgresql_static_check/scan-build-2009-08-23-29/ tar ball with report can be downloaded from here: http://zlew.org/postgresql_static_check/postgresql_static_check_23rdAugust2009.tar.xz (compressed with lzma's xz tool). here's the patch for elog stuff: Index: src/include/utils/elog.h === RCS file: /projects/cvsroot/pgsql/src/include/utils/elog.h,v retrieving revision 1.101 diff -u -b -r1.101 elog.h --- src/include/utils/elog.h11 Jun 2009 14:49:13 - 1.101 +++ src/include/utils/elog.h23 Aug 2009 22:16:05 - @@ -104,7 +104,7 @@ */ #define ereport_domain(elevel, domain, rest) \ (errstart(elevel, __FILE__, __LINE__, PG_FUNCNAME_MACRO, domain) ? \ -(errfinish rest) : (void) 0) +(errfinish rest) : (void) 0), (elevel = ERROR) ? exit(1) : 0 #define ereport(elevel, rest) \ ereport_domain(elevel, TEXTDOMAIN, rest) @@ -190,7 +190,7 @@ * elog(ERROR, portal \%s\ not found, stmt-portalname); *-- */ -#define elog elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO), elog_finish +#define elog(A, ...) elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO), elog_finish(A, __VA_ARGS__), (A = ERROR) ? exit(1) : 0 extern void elog_start(const char *filename, int lineno, const char *funcname); extern void hopefully satisfying everyone. Hope to see few fixes out of that ;) thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang's static checker report.
On 24 Aug 2009, at 00:15, Greg Stark wrote: On Sun, Aug 23, 2009 at 11:16 PM, Grzegorz Jaskiewiczg...@pointblue.com.pl wrote: ok folks, here's the last one for Today: http://zlew.org/postgresql_static_check/scan-build-2009-08-23-29/ This does look better. The first one I looked at looks like a legitimate bug. The nice thing is that this seems to be picking up a lot of error handling cases that we don't bother to have regression tests for. true One more request though. Can you configure with --enable-assertions so --enable-cassert, enabled, and also added exit_* in pg_dump to list of functions that never return. new report's at: http://zlew.org/postgresql_static_check/scan-build-2009-08-24-2/ the archive is at : http://zlew.org/postgresql_static_check/postgresql_static_check_24thAugust2009.tar.xz So that the overall 'static check' patch now looks like this: Index: src/bin/pg_dump/pg_backup.h === RCS file: /projects/cvsroot/pgsql/src/bin/pg_dump/pg_backup.h,v retrieving revision 1.52 diff -u -b -r1.52 pg_backup.h --- src/bin/pg_dump/pg_backup.h 11 Jun 2009 14:49:07 - 1.52 +++ src/bin/pg_dump/pg_backup.h 23 Aug 2009 23:31:43 - @@ -150,7 +150,7 @@ extern void exit_horribly(Archive *AH, const char *modulename, const char *fmt,...) -__attribute__((format(printf, 3, 4))); +__attribute__((format(printf, 3, 4))) __attribute__((analyzer_noreturn)); /* Lets the archive know we have a DB connection to shutdown if it dies */ Index: src/bin/pg_dump/pg_dump.h === RCS file: /projects/cvsroot/pgsql/src/bin/pg_dump/pg_dump.h,v retrieving revision 1.156 diff -u -b -r1.156 pg_dump.h --- src/bin/pg_dump/pg_dump.h 2 Aug 2009 22:14:52 - 1.156 +++ src/bin/pg_dump/pg_dump.h 23 Aug 2009 23:31:43 - @@ -481,7 +481,7 @@ extern void *pg_realloc(void *ptr, size_t size); extern void check_conn_and_db(void); -extern void exit_nicely(void); +extern void exit_nicely(void) __attribute__((analyzer_noreturn)); extern void parseOidArray(const char *str, Oid *array, int arraysize); Index: src/include/postgres.h === RCS file: /projects/cvsroot/pgsql/src/include/postgres.h,v retrieving revision 1.92 diff -u -b -r1.92 postgres.h --- src/include/postgres.h 1 Jan 2009 17:23:55 - 1.92 +++ src/include/postgres.h 23 Aug 2009 23:31:43 - @@ -691,6 +691,6 @@ extern int ExceptionalCondition(const char *conditionName, const char *errorType, -const char *fileName, int lineNumber); + const char *fileName, int lineNumber) __attribute__((analyzer_noreturn)); #endif /* POSTGRES_H */ Index: src/include/utils/elog.h === RCS file: /projects/cvsroot/pgsql/src/include/utils/elog.h,v retrieving revision 1.101 diff -u -b -r1.101 elog.h --- src/include/utils/elog.h11 Jun 2009 14:49:13 - 1.101 +++ src/include/utils/elog.h23 Aug 2009 23:31:43 - @@ -104,7 +104,7 @@ */ #define ereport_domain(elevel, domain, rest) \ (errstart(elevel, __FILE__, __LINE__, PG_FUNCNAME_MACRO, domain) ? \ -(errfinish rest) : (void) 0) +(errfinish rest) : (void) 0), (elevel = ERROR) ? exit(1) : 0 #define ereport(elevel, rest) \ ereport_domain(elevel, TEXTDOMAIN, rest) @@ -190,7 +190,7 @@ * elog(ERROR, portal \%s\ not found, stmt-portalname); *-- */ -#define elog elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO), elog_finish +#define elog(A, ...) elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO), elog_finish(A, __VA_ARGS__), (A = ERROR) ? exit(1) : 0 extern void elog_start(const char *filename, int lineno, const char *funcname); That's it folks for Today, gotta go to sleep. Have fun... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] boolean in C
Hi folks, Today I got bitten a bit, trying to write C function for postgresql, that also includes some of company's internal stuff. Needles to say, our stuff defines BOOL, as well as postgresql's c.h include file. Now, for us, we will probably change it, but is there any reason for postgresql nowadays not to use stdbool.h, apart from fact, that no one made an effort ? Having said that, wouldn't it be easy as just replacing all TRUE/FALSE/BOOL with lowercase ones, and including stdbool from c.h ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
On 16 Jul 2009, at 12:52, Bernd Helmle wrote: --On 16. Juli 2009 11:12:34 +0100 Grzegorz Jaskiewicz g...@pointblue.com.pl wrote: Now, for us, we will probably change it, but is there any reason for postgresql nowadays not to use stdbool.h, apart from fact, that no one made an effort ? Having said that, wouldn't it be easy as just replacing all TRUE/FALSE/BOOL with lowercase ones, and including stdbool from c.h ? If i remember correctly, older SUN compilers doesn't have it. Don't know if that is still an issue yet, but we need to take care for non- gcc compilers i think. For those guys, we could just put in c.h as it is now... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
oh, another thing. stdbool is C99 standard feature. Not gcc extension. Just in case, someone thinks otherwise. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
On 16 Jul 2009, at 14:20, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: oh, another thing. stdbool is C99 standard feature. We are still targeting C89, not C99. Another reason not to depend on stdbool is that, so far as I can see, the standard does not promise that type _Bool has size = 1 byte. We have to have that because of on-disk compatibility requirements. I think the latter is easily fixable, or forceable to be one byte. Why C89, and not C99 ? Virtually all compilers for last 4 years have/ had C99 support. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
On 16 Jul 2009, at 14:53, Peter Eisentraut wrote: On Thursday 16 July 2009 16:23:31 Grzegorz Jaskiewicz wrote: On 16 Jul 2009, at 14:20, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: oh, another thing. stdbool is C99 standard feature. We are still targeting C89, not C99. Another reason not to depend on stdbool is that, so far as I can see, the standard does not promise that type _Bool has size = 1 byte. We have to have that because of on-disk compatibility requirements. I think the latter is easily fixable, or forceable to be one byte. How do you plan to do that? by casting it to 1 byte type such as char ? I don't think anyone will add 3rd state to boolean in stdbool, at least not any time soon :) And it is pretty annoying, when your product also has its own BOOLean defined... Why C89, and not C99 ? Virtually all compilers for last 4 years have/ had C99 support. Well, I think we want to run on systems that are older than 4 years, too. Sure, but that's probably less than 1% of all systems. The 4 years was a guess, I think its much more than that. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
On 16 Jul 2009, at 15:17, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: That's hardly going to improve readability for anyone. Also, it will flat out not work for the catalog struct declarations. When we say bool relhasindex; the compiler had better think that that's a one-byte field. Sure, but I would certainly hope, there's not too many places where you actually convert it from disc representation, to internal and vice versa. And it is pretty annoying, when your product also has its own BOOLean defined... IOW you're not using stdbool either? Well, saying that I don't is quite an overstatement. It was decided long before I started working for that customer, and is full of problems like that. But still, it would be nice for postgresql to at least not cause problems like that. Having said that, I will probably fix it on customer's side, but I wanted to see if you guys will be happy with patch that changes that in postgresql. thanks . -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for automating partitions in PostgreSQL 8.4 Beta 2
gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith - Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing - fwrapv -bundle -multiply_defined suppress regress.o - bundle_loader ../../../src/backend/postgres -L../../../src/port -o regress.so cp ../../../contrib/spi/refint.so refint.so cp ../../../contrib/spi/autoinc.so autoinc.so gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith - Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing - fwrapv pg_regress.o pg_regress_main.o -L../../../src/port -Wl,- dead_strip_dylibs -lpgport -lz -lreadline -lm -o pg_regress make -C config all make[1]: Nothing to be done for `all'. /bin/sh: /Users/gj/Projects/postgres-head/pgsql/partition.sh: No such file or directory make: *** [all] Error 127 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for automating partitions in PostgreSQL 8.4 Beta 2
make -C catalog all ( echo src/backend/catalog/catalog.o src/backend/catalog/dependency.o src/backend/catalog/heap.o src/backend/catalog/index.o src/backend/ catalog/indexing.o src/backend/catalog/namespace.o src/backend/catalog/ aclchk.o src/backend/catalog/pg_aggregate.o src/backend/catalog/ pg_constraint.o src/backend/catalog/pg_conversion.o src/backend/ catalog/pg_depend.o src/backend/catalog/pg_enum.o src/backend/catalog/ pg_inherits.o src/backend/catalog/pg_largeobject.o src/backend/catalog/ pg_namespace.o src/backend/catalog/pg_operator.o src/backend/catalog/ pg_proc.o src/backend/catalog/pg_shdepend.o src/backend/catalog/ pg_type.o src/backend/catalog/storage.o src/backend/catalog/ toasting.o ) objfiles.txt make[3]: *** No rule to make target `../../../src/include/catalog/ pg_partition.h', needed by `postgres.bki'. Stop. make[2]: *** [catalog-recursive] Error 2 (that's on mac os x). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issue with contrib
On 7 Jun 2009, at 03:27, Bruce Momjian wrote: Grzegorz Jaskiewicz wrote: On 6 Jun 2009, at 19:50, Tom Lane wrote: We have five days. I don't think we need testing, per se. The first step should be to diff the 8.3 and 8.4 versions of the various contrib .sql.in files and determine what changed. Any module whose .sql.in file hasn't changed is definitely safe. I can tell you already that dblink has changed, and I had to drop it before migration, otherwise everything went fine. Migration of 57GB data in place took about 1 minute. The good news is that the INSTALL instructions where clear enough for the tester to understand that uninstalling dblink from the old cluster and reinstalling it in the new cluster would work. Yes, but I forgot about one database, and had to do it all over again, luckily I first tested it without --link... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issue with contrib
On 6 Jun 2009, at 19:50, Tom Lane wrote: We have five days. I don't think we need testing, per se. The first step should be to diff the 8.3 and 8.4 versions of the various contrib .sql.in files and determine what changed. Any module whose .sql.in file hasn't changed is definitely safe. I can tell you already that dblink has changed, and I had to drop it before migration, otherwise everything went fine. Migration of 57GB data in place took about 1 minute. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GCC 4.4 compiler warnings
and a quick one on mac os x, which looks like it has something to do with COMPLETE_WITH_LIST() and macros like that, and maybe with completion_matches on mac os x. I don't get these on linux. This is from gcc4.4 with -O3 (which obviously causes gcc to find more potential warnings). tab-complete.c: In function 'psql_completion': • tab-complete.c:666: warning: implicit declaration of function 'completion_matches' • tab-complete.c:666: warning: assignment makes pointer from integer without a cast • tab-complete.c:670: warning: assignment makes pointer from integer without a cast • tab-complete.c:675: warning: assignment makes pointer from integer without a cast • tab-complete.c:683: warning: assignment makes pointer from integer without a cast • tab-complete.c:699: warning: assignment makes pointer from integer without a cast • tab-complete.c:709: warning: assignment makes pointer from integer without a cast • tab-complete.c:720: warning: assignment makes pointer from integer without a cast • tab-complete.c:730: warning: assignment makes pointer from integer without a cast • tab-complete.c:742: warning: assignment makes pointer from integer without a cast • tab-complete.c:752: warning: assignment makes pointer from integer without a cast • tab-complete.c:762: warning: assignment makes pointer from integer without a cast • tab-complete.c:777: warning: assignment makes pointer from integer without a cast • tab-complete.c:785: warning: assignment makes pointer from integer without a cast • tab-complete.c:794: warning: assignment makes pointer from integer without a cast • tab-complete.c:804: warning: assignment makes pointer from integer without a cast • tab-complete.c:814: warning: assignment makes pointer from integer without a cast • tab-complete.c:824: warning: assignment makes pointer from integer without a cast • tab-complete.c:834: warning: assignment makes pointer from integer without a cast • tab-complete.c:843: warning: assignment makes pointer from integer without a cast • tab-complete.c:851: warning: assignment makes pointer from integer without a cast • tab-complete.c:856: warning: assignment makes pointer from integer without a cast • tab-complete.c:862: warning: assignment makes pointer from integer without a cast • tab-complete.c:871: warning: assignment makes pointer from integer without a cast • tab-complete.c:876: warning: assignment makes pointer from integer without a cast • tab-complete.c:889: warning: assignment makes pointer from integer without a cast • tab-complete.c:899: warning: assignment makes pointer from integer without a cast • tab-complete.c:909: warning: assignment makes pointer from integer without a cast • tab-complete.c:918: warning: assignment makes pointer from integer without a cast • tab-complete.c:925: warning: assignment makes pointer from integer without a cast • tab-complete.c:935: warning: assignment makes pointer from integer without a cast • tab-complete.c:941: warning: assignment makes pointer from integer without a cast • tab-complete.c:948: warning: assignment makes pointer from integer without a cast • tab-complete.c:957: warning: assignment makes pointer from integer without a cast • tab-complete.c:963: warning: assignment makes pointer from integer without a cast • tab-complete.c:975: warning: assignment makes pointer from integer without a cast • tab-complete.c:979: warning: assignment makes pointer from integer without a cast • tab-complete.c:985: warning: assignment makes pointer from integer without a cast • tab-complete.c:994: warning: assignment makes pointer from integer without a cast • tab-complete.c:1000: warning: assignment makes pointer from integer without a cast • tab-complete.c:1009: warning: assignment makes pointer from integer without a cast • tab-complete.c:1018: warning: assignment makes pointer from integer without a cast • tab-complete.c:1028: warning: assignment makes pointer from integer without a cast • tab-complete.c:1035: warning: assignment makes pointer from integer without a cast • tab-complete.c:1045: warning: assignment makes pointer from integer without a cast • tab-complete.c:1056: warning: assignment makes pointer from integer without a cast • tab-complete.c:1066: warning: assignment makes pointer from integer without a cast • tab-complete.c:1075: warning: assignment makes pointer from integer without a cast • tab-complete.c:1082: warning: assignment makes pointer from integer without a
Re: [HACKERS] GCC 4.4 compiler warnings
On 26 Apr 2009, at 16:22, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: tab-complete.c:666: warning: implicit declaration of function 'completion_matches' Are you sure you have a real installation of readline? OSX is notorious for providing a bogus one, particularly in older release series. Sure, it behaves quite strange - gotta say. So I'll try to recompile against vanilla readline, and see if it still causes problems. But what about all those warnings I posted here earlier ? Those were from fedora 9, and vanillia gcc 4.4 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HashJoin w/option to unique-ify inner rel
On 25 Apr 2009, at 04:52, Robert Haas wrote: blow the hash-join plan out of the water anyway... but Stephen Frost was telling me at JDcon East that he sometimes sets it to something like 8GB when he's the only user on his apparently-quite-awesome hardware...) For the record, because most queries have 5-6 joins here, I always set it up to 32MB on production. We don't have more than 100-150 connections, so it plays well on normal 32bit machine with 4GB. If what you wrote about hash-join is confirmed by others, than I am pretty much +100 for fixing it. (just my penny). thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GCC 4.4 compiler warnings
reloptions.c: In function âextractRelOptionsâ: reloptions.c:731: warning: value computed is not used heapam.c: In function âheapgettupâ: heapam.c:458: warning: value computed is not used heapam.c:458: warning: value computed is not used heapam.c: In function âheapgettup_pagemodeâ: heapam.c:726: warning: value computed is not used heapam.c:726: warning: value computed is not used heapam.c: In function âheap_tuple_attr_equalsâ: heapam.c:2936: warning: value computed is not used heapam.c:2936: warning: value computed is not used heapam.c:2937: warning: value computed is not used heapam.c:2937: warning: value computed is not used tuptoaster.c: In function âtoast_fetch_datumâ: tuptoaster.c:1391: warning: value computed is not used tuptoaster.c:1393: warning: value computed is not used tuptoaster.c: In function âtoast_fetch_datum_sliceâ: tuptoaster.c:1607: warning: value computed is not used tuptoaster.c:1609: warning: value computed is not used heap.c: In function âMergeWithExistingConstraintâ: heap.c:2016: warning: value computed is not used aclchk.c: In function âExecGrant_Databaseâ: aclchk.c:1198: warning: value computed is not used aclchk.c:1198: warning: value computed is not used aclchk.c: In function âExecGrant_Tablespaceâ: aclchk.c:1918: warning: value computed is not used aclchk.c:1918: warning: value computed is not used aclchk.c: In function âpg_tablespace_aclmaskâ: aclchk.c:2751: warning: value computed is not used aclchk.c:2751: warning: value computed is not used parse_func.c: In function âParseFuncOrColumnâ: parse_func.c:76: warning: âdeclared_arg_typesâ may be used uninitialized in this function parse_func.c:80: warning: ânvargsâ may be used uninitialized in this function parse_func.c:79: warning: âretsetâ may be used uninitialized in this function parse_func.c:77: warning: âargdefaultsâ may be used uninitialized in this function parse_func.c:69: warning: âfuncidâ may be used uninitialized in this function parse_func.c:68: warning: ârettypeâ may be used uninitialized in this function dbcommands.c: In function âAlterDatabaseSetâ: dbcommands.c:1448: warning: value computed is not used dbcommands.c:1448: warning: value computed is not used dbcommands.c: In function âAlterDatabaseOwnerâ: dbcommands.c:1562: warning: value computed is not used dbcommands.c:1562: warning: value computed is not used proclang.c: In function âfind_language_templateâ: proclang.c:369: warning: value computed is not used proclang.c:369: warning: value computed is not used proclang.c:374: warning: value computed is not used proclang.c:374: warning: value computed is not used proclang.c:379: warning: value computed is not used proclang.c:379: warning: value computed is not used tablecmds.c: In function âdecompile_conbinâ: tablecmds.c:7006: warning: value computed is not used tablecmds.c:7006: warning: value computed is not used tablespace.c: In function âAlterTableSpaceOwnerâ: tablespace.c:883: warning: value computed is not used tablespace.c:883: warning: value computed is not used trigger.c: In function âRelationBuildTriggersâ: trigger.c:1183: warning: value computed is not used typecmds.c: In function âAlterDomainAddConstraintâ: typecmds.c:1948: warning: value computed is not used typecmds.c:1948: warning: value computed is not used typecmds.c: In function âGetDomainConstraintsâ: typecmds.c:2390: warning: value computed is not used user.c: In function âRenameRoleâ: user.c:1075: warning: value computed is not used user.c:1075: warning: value computed is not used spi.c: In function âSPI_getvalueâ: spi.c:786: warning: value computed is not used spi.c:786: warning: value computed is not used numeric.c: In function ânumericvar_to_int4â: numeric.c:2030: warning: âvalâ may be used uninitialized in this function numeric.c: In function ânumeric_int2â: numeric.c:2120: warning: âvalâ may be used uninitialized in this function numeric.c: In function ânumeric_int8â: numeric.c:2073: warning: âresultâ may be used uninitialized in this function ruleutils.c: In function âpg_get_triggerdefâ: ruleutils.c:570: warning: value computed is not used catcache.c: In function âCatalogCacheComputeTupleHashValueâ: catcache.c:231: warning: value computed is not used catcache.c:241: warning: value computed is not used catcache.c:251: warning: value computed is not used catcache.c:261: warning: value computed is not used catcache.c: In function âCatalogCacheFlushRelationâ: catcache.c:738: warning: value computed is not used catcache.c: In function âSearchCatCacheâ: catcache.c:1150: warning: value computed is not used catcache.c:1150: warning: value computed is not used catcache.c: In function âSearchCatCacheListâ: catcache.c:1400: warning: value computed is not used catcache.c:1400: warning: value computed is not used relcache.c: In function âRelationBuildRuleLockâ: relcache.c:649: warning:
Re: [HACKERS] GCC 4.4 compiler warnings
if that's without -O3, than please retry it with. It will give even more. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Automating Partitions in PostgreSQL - Query on syntax
Does sql standard defines it ? and another question, what about updating existing partitions, with no need to drop/recreate ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Replacing plpgsql's lexer
On 19 Apr 2009, at 17:42, Tom Lane wrote: The attached proposed patch rips out plpgsql's handling of comments and string literals, and puts in scanner rules that are extracted from the core lexer (but simplified in a few places where we don't need all the complexity). The net user-visible effects should be: Comments? Will it also mean, that queries are going to be analyzed deeper ? Ie, afaik I am able now to create plpgsql function, that tries to run query accessing non existent table, or columns. Or, if I rename column/table/relation now, views, etc are getting updated - but not plpgsql functions. Will that change with your patch ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Replacing plpgsql's lexer
On 19 Apr 2009, at 18:28, Greg Stark wrote: On Sun, Apr 19, 2009 at 6:24 PM, Grzegorz Jaskiewicz g...@pointblue.com.pl wrote: Will it also mean, that queries are going to be analyzed deeper ? Ie, afaik I am able now to create plpgsql function, that tries to run query accessing non existent table, or columns. Or, if I rename column/table/relation now, views, etc are getting updated - but not plpgsql functions. Will that change with your patch ? The scanner isn't responsible for anything like this. It just braeks the input up into tokens. So its responsible for determining where strings start and end and where tble names start and end but doesn't actually look up the name anywhere -- that's up to the parser and later steps. So no. ok, thanks. To be honest, That would be the great feature. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4b1: Query returning results in different order to 8.3
On 18 Apr 2009, at 22:22, Tom Lane wrote: This is mentioned in the release notes, but I suppose we'd better promote it to the observe the following incompatibilities list... This is a really funny one, because people naturally expect UNION [ALL] to stay in the same order. Unlike the table, order here cannot change by inserts/updates, etc. I am sure many, even well experienced will stumble upon that one. Me is guessing, that UNION [ALL] performance just had to be improved for CTEs ? Or was it something completely separate. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3
Btw, There was a EXPLAIN XML summer of code project, wasn't there ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3
On 16 Apr 2009, at 16:21, David Fetter wrote: On Thu, Apr 16, 2009 at 06:12:10AM +0100, Simon Riggs wrote: I think the way to do this is to introduce plan output in XML If we're going with a serialization, which I think would be an excellent idea, how about one that's light-weight and human-readable like JSON? +1 xml/json is machine readable. I don't think, personaly that explain (analyze) is not easy to read by human, quite contrary. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3
On 16 Apr 2009, at 19:41, Merlin Moncure wrote: Is that because of how the output is formatted though, or because the concepts are difficult to express? (I agree though, json is better especially for structures that are possibly highly nested). What I mean is that what postgresql displays currently as explain(analyze[verbose]) is clear and understandable. Also, it is getting better and better from version to version. So I don't personally agree, that it is unreadable - and I am up for (and I am sure many users like me are) JSON, or XML output. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Closing some 8.4 open items
On 11 Apr 2009, at 08:01, Hitoshi Harada wrote: 2009/4/11 David Fetter da...@fetter.org: On Sat, Apr 11, 2009 at 03:48:33PM +0900, Hitoshi Harada wrote: Yeah, but all the window functions are stored in pg_proc. So are aggregate functions, and they have their own separate way of being addressed in psql :) Aggregate functions are stored in pg_aggregate. And they are combinations of plain function which is stored in pg_proc. Maybe trigger functions should be displayed separately too than ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Closing some 8.4 open items
On 11 Apr 2009, at 13:33, Hitoshi Harada wrote: Maybe trigger functions should be displayed separately too than ? You don't catch the point. The aggregate entries in pg_proc have prosrc = 'aggregate_dummy', which means they're dummy and the entities are stored in pg_aggregate. Triggers in pg_proc are dummy? No, they are actually plain functions with trigger return type. yes, that's from strictly insider's point of view. Based on the implementation of that in postgresql, but you guys talk about user perspective, after all - psql is for users, not only for postgresql hackers. So the better question would be, can you use window, aggregate, trigger functions the same way other procedures ? I guess the answer is no :) Hence, if classify - than I would suggest to do it completely and fair, and not judge it only from postgresql-hacker perspective. I am hardly the postgresql-hacker myself, so it is my opinion from user perspective, that also understands where your opinion comes from. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] a few crazy ideas about hash joins
On 3 Apr 2009, at 19:44, Lawrence, Ramon wrote: I would be especially interested in using a shared memory hash table that *all* backends can use - if the table is mostly read-only, as dimension tables often are in data warehouse applications. That would give zero startup cost and significantly reduced memory. I think that's a non-starter due to visibility issues and handling inserts and updates. Even just reusing a hash from one execution in a later execution of the same plan would be tricky since we would have to expire it if the snapshot changes. If your data set is nearly read-only, materialized views would be a better way to go and would require no hash join changes. I think what he means is that some of the tables in join are effectively read-only. So materialized views are nono here. Unless you mean just a partial ones. I have to say, that frankly I got same problem, and plausibly my schemas could benefit from changes discussed here. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gcc: why optimize for size flag is not the default
On 11 Mar 2009, at 13:51, Marko Kreen wrote: Linux kernel is moving to use -Os everywhere. AFAIK their argument is that kernel code should not be doing anything CPU-intensive, thus minimal cache usage is more important than unrolled loops. This also seems to hint that -Os is not really appropriate to Postgres. Although it would be good fit for eg. PgBouncer. while it might be right in case of linux kernel (which I won't agree totally with personally), I don't see any reason to compare it with postgresql. Kernel is extensively use by everything in system, hence their reasoning. Postgresql is an application. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cardinality()
On 1 Mar 2009, at 00:52, Andrew Dunstan wrote: We seem to have acquired a cardinality() function with almost no discussion, and it has semantics that are a bit surprising to me. I should have thought cardinality(array) would be the total number of elements in the array. Instead, it seems it is a synonym for array_length(array,1). Is that *really* what the standard says? any difference between array_upper(array,1), and cardinality ? Standart just says something like: cardinality (a collection): - The number of elements in that collection. - Those elements need not necessarily have distinct values. - The objects to which this concept applies includes tables and the values of collection types. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] would it be a lot of work, to add optimizations accross unions ?
Say I have: select foo ( select foo from bar1 union all select foo from bar2 union all select foo from bar3 ... ) a order by foo desc limit X; (and I can give you few other examples around the same 'note', say with when foo=N in outer subselect) Would anyone consider such optimization, when postgres will apply the same condition to inner queries, providing that their size is substantial? Same would actually apply for different subqueries, without union/ intersect/etc: select foo( select foo from bar1 ) a where foo in (x,y,z) order by foo desc limit N (of course this is just a simplified example). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] would it be a lot of work, to add optimizations accross unions ?
On 28 Feb 2009, at 11:37, Gregory Stark wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: Say I have: select foo ( select foo from bar1 union all select foo from bar2 union all select foo from bar3 ... ) a order by foo desc limit X; (and I can give you few other examples around the same 'note', say with when foo=N in outer subselect) Would anyone consider such optimization, when postgres will apply the same condition to inner queries, providing that their size is substantial? Well you haven't said what optimization you're looking for here. I posted a patch to look for an ordered path for members of a union a while back but it still needed a fair amount of work before it was usable. The LIMIT can't be pushed into the union unless we do have ordered paths so that's further down the line. (And even then it would only work if there are no conditions on there append path.) And I believe we already do push down where clauses like foo=N. ok, in this case - I have few tables, few M rows each - and I need to find out highest id - for instance. doing it like that, would cause postgres to seq scan all of them, one by one. Obviously, I could just rewrite subselects, to include order+limit, but I was wondering, whether that job couldn't be done on planner's level. I belive limit it self can't be pushed down, but with order by - why not ? if you look for X results, limiting all selects to X, will guarantee the same result - providing of course it is used with order by. (which, in my opinion - should be illegal without order by, anyway). Same would actually apply for different subqueries, without union/ intersect/etc: select foo( select foo from bar1 ) a where foo in (x,y,z) order by foo desc limit N huh? Just a simple example, perhaps oversimplified. The thing is, in case like that - planner could merge two queries together. thanks Gregory. -- GJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 features presentation
tbh, I would add much more facts from internal changes, to improve efficiency. Because that wouldn't be very convincing for , say my managment (but than, what is..), etc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] The science of optimization in practical terms?
yet more arguments, to let postgresql estimate those automatically. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 64 bit PostgreSQL 8.3.6 build on AIX 5300-09-02-0849 with IBM XL C/C++ 10.1.0.1 - initdb fails (could not dump unrecognized node type: 650)
On 9 Feb 2009, at 16:04, Tom Lane wrote: Hmm. I think -qnoansialias corresponds to gcc's -fno-strict-aliasing, which we *know* is necessary to build a working Postgres on recent gcc versions. I have not checked the exact symptoms of -fstrict-aliasing recently, but what you're reporting is definitely consistent with the idea that the compiler is improperly reordering some assignments, which is basically what the aliasing business is about. So that switch seems like the critical issue here. Just for the record Tom, I am building postgresql on my test box with vectoring (-ftree-vectorize) and -O3, which pretty much turns the strict-aliasing flag off. it compiles, passes default tests, passes my tests, and works beautifully. Vectoring is pretty much only used in numeric code. That's on 32bit machine, mac os x. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is a plan for lmza commpression in pg_dump
On 7 Feb 2009, at 21:08, daveg wrote: That this comes up much to often suggests that there is more than near zero interest. Why can only one compression library can be considered? We use multiple readline implementations, for better or worse. I don't see anything wrong with using standard unix pipes... and do it in truly unix and scalable way ! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] add_path optimization
On 2 Feb 2009, at 14:50, Robert Haas wrote: well, true - but also, statically allocated table, without any predefined size (with #DEFINE) , and no boundary check - is bad as well. I suppose , this code is easy enough to let it be with your changes, but I would still call it not pretty. Well, it might merit a comment. :) What I'd really like to do is develop some tests based on a publicly available dataset. Any suggestions? I would say, it wouldn't hurt to do benchmarking/profiling regression tests on real hardware - but someone will have to generate quite substantial amount of data, so we could test it on small queries, up to 20+ join/sort/window function/aggregation queries, with various indexes, and data types. The more real the data, the better. I could make some of my stuff public - but without the lookup tables (id-some real data - like, names, surnames, mac addr, etc). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] add_path optimization
On 1 Feb 2009, at 21:35, Robert Haas wrote: On Sun, Feb 1, 2009 at 3:25 PM, Grzegorz Jaskiewicz g...@pointblue.com.pl wrote: I don't like the fact that you hardcoded that here. I know that you are trying to pass on few calls in one go here, but still... ugly. Well, I think you'll find that using a dynamically sized data structure destroys the possibility of squeezing any additional performance out of this part of the code. The nice thing about fixed-size data structures is that they cost essentially nothing to stack-allocate; you just move the stack pointer and away you go. We should in fact be looking for MORE places where we can avoid the use of constructs like List, since the second-highest CPU hog in my tests was AllocSetAlloc(), beaten out only by add_path(). With this patch applied, AllocSetAlloc() moves up to first. well, true - but also, statically allocated table, without any predefined size (with #DEFINE) , and no boundary check - is bad as well. I suppose , this code is easy enough to let it be with your changes, but I would still call it not pretty. Hmm, well I didn't either, but there's this handy tool called gprof that you might want to try out. I wouldn't be wasting my time patching this part of the code if it didn't make a difference, and in fact if you do 10% of the amount of benchmarking that I did in the process of creating this patch, you will find that it in fact does make a difference. To be honest, I really didn't had a time to run it down with your patch and gprof. I believe that you did that already, hence your suggestion, right ? Actually - if you did profile postgresql with bunch of queries, I wouldn't mind to see results of it - I don't know whether it makes sense to send that to the list (I would think it does), but if it is too big, or something - you could send it to me in private. It's already static to that .c file, so the compiler likely will inline it. In fact, I suspect you will find that removing the static keyword from the implementation of that function in CVS HEAD is itself sufficient to produce a small but measurable slowdown in planning of large join trees, exactly because it will defeat inlining. that depends on many things, including whether optimizations are on or not. Because that function basically consists of two ifs essentially - it could easily be turned into two separate inlines/macros - that would remove any function's specific overhead (stack alloc, etc, etc). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updated backslash consistency patch
On 1 Feb 2009, at 11:07, Grzegorz Jaskiewicz wrote: On 1 Feb 2009, at 10:58, David Fetter wrote: I think Stephen meant that they should add a column to the output. Stephen? it's already there. oh sorry, column - as in psql printout's column yeah, doable too - lemme see if I can do it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updated backslash consistency patch
On 1 Feb 2009, at 14:21, Stephen Frost wrote: So if you have alot of roles granted, and even a couple attributes, it gets ugly.. true. I thought perhaps you guys want to go for table chart USER\t PERM1\t PERM2\t PERM3 foo\to\tx\tx\o bar\tx\to\to\x Personaly that would work for me, but that's probably the very argument the whole thread was devoted to. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 1 Feb 2009, at 10:25, Gregory Stark wrote: I'm sorry if I was unclear. It needs to be in keywords.c but can probably be marked as UNRESERVED_KEYWORD there rather than RESERVED_KEYWORD. In other words there are two places where you have to indicate whether it's reserved or not, keywords.c and gram.y. Ok, ACKed. So far I got mostly critique here, even tho - I haven't started much, which is quite sad in a way - because it is not very pro-creative, but I'll still continue on with the patch - whatever the outcome. Any change to the grammar meets the question of whether it conflicts with the standard. That's just the way it is and doesn't reflect on you or your work. Sure, there's much broad problem with it too. Wether it should grant/revoke SELECT to all user defined tables? In all schemas, except for pg_catalog and information schema (the later, I believe is already SELECT granted for all users). Hence my question yesterday, how can I make sure I got all of these oids. I was suggested to use SearchSysCache* stuff to grab oids, but honestly, I wouldn't mind to get some directions on that from you guys here. thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updated backslash consistency patch
On 1 Feb 2009, at 10:58, David Fetter wrote: I think Stephen meant that they should add a column to the output. Stephen? it's already there. try: psql -E \du ;] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] add_path optimization
disclaimer: I don't know that bit of postgresql code, in fact - this
is the first time I see it.
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***
*** 473,478 match_unsorted_outer(PlannerInfo *root,
--- 473,481
if (nestjoinOK)
{
+ Path *paths[5];
I don't like the fact that you hardcoded that here. I know that you
are trying to pass on few calls in one go here, but still... ugly.
static int
compare_fuzzy_path_costs(Path *path1, Path *path2, int *startup_cost)
{
*startup_cost = (s == 0) ? t : s;
Why not *startup_cost = s, and let the caller decide which value it
wants to use ?
or just return both, from single call (which would ?
...
return t;
}
To be fair, I don't see compare_fuzzy_path_costs change to save too
much of time in planner.
I would myself probably convert that function into two defines/inline
funcs, but that's just me.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] adding stuff to parser, question
Hey folks,
I am trying to add GRANT SELECT ON ALL TABLES to postgres, as it has
been quite few times now - where I had to write a procedure for that.
I kind of looked around, and more or less know how to approach it. But
I am stuck on parser :), yes - parser.
Can someone walk me through adding new rules to parser ?
so far I have this:
Index: parser/gram.y
===
RCS file: /projects/cvsroot/pgsql/src/backend/parser/gram.y,v
retrieving revision 2.656
diff -u -r2.656 gram.y
--- parser/gram.y 22 Jan 2009 20:16:05 - 2.656
+++ parser/gram.y 31 Jan 2009 16:44:57 -
@@ -494,7 +494,7 @@
STATISTICS STDIN STDOUT STORAGE STRICT_P STRIP_P SUBSTRING
SUPERUSER_P
SYMMETRIC SYSID SYSTEM_P
- TABLE TABLESPACE TEMP TEMPLATE TEMPORARY TEXT_P THEN TIME TIMESTAMP
+ TABLE TABLES TABLESPACE TEMP TEMPLATE TEMPORARY TEXT_P THEN TIME
TIMESTAMP
TO TRAILING TRANSACTION TREAT TRIGGER TRIM TRUE_P
TRUNCATE TRUSTED TYPE_P
@@ -4301,6 +4301,13 @@
n-objs = $2;
$$ = n;
}
+ | ALL TABLES
+ {
+ PrivTarget *n = (PrivTarget *)
palloc(sizeof(PrivTarget));
+ n-objtype = ACL_OBJECT_RELATION;
+ n-objs = NULL;
+ $$ = n;
+ }
| SEQUENCE qualified_name_list
{
PrivTarget *n = (PrivTarget *)
palloc(sizeof(PrivTarget));
Index: parser/keywords.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/parser/keywords.c,v
retrieving revision 1.209
diff -u -r1.209 keywords.c
--- parser/keywords.c 1 Jan 2009 17:23:45 - 1.209
+++ parser/keywords.c 31 Jan 2009 16:44:57 -
@@ -373,6 +373,7 @@
{sysid, SYSID, UNRESERVED_KEYWORD},
{system, SYSTEM_P, UNRESERVED_KEYWORD},
{table, TABLE, RESERVED_KEYWORD},
+ {table, TABLES, RESERVED_KEYWORD},
{tablespace, TABLESPACE, UNRESERVED_KEYWORD},
{temp, TEMP, UNRESERVED_KEYWORD},
{template, TEMPLATE, UNRESERVED_KEYWORD},
But that seems to be not nearly enough, for psql to recognize GRANT
SELECT ON ALL TABLES TO foo.
Please help me out, with stuff I am missing here. I never had any
expierence with bison, or any other lexical parsers for that matter.
Thanks. :)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 31 Jan 2009, at 16:46, Grzegorz Jaskiewicz wrote:
+ {table, TABLES, RESERVED_KEYWORD},
Gaaah, a typo...
:(
(another useless post to -hackers, by myself).
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 31 Jan 2009, at 17:17, Gregory Stark wrote:
Grzegorz Jaskiewicz g...@pointblue.com.pl writes:
You're going to kick yourself, but:
{table, TABLE, RESERVED_KEYWORD},
+ {table, TABLES, RESERVED_KEYWORD},
^
I don't see any reason offhand why it should have to be a reserved
word
though. You should be able to make it an UNRESERVED_KEYWORD. Oh, and
you'll
want to add it to the list of tokens in unreserved_keyword in gram.y
as well.
I am really novice with parsers here, so - I felt like I have to do
it, in order to make it work. It just wasn't working without that bit
in keywords.c.
I shall try your way, thanks :)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
from http://wiki.postgresql.org/wiki/Todo: [E] inline: 10px-UntickedTick.svg.pngAllow GRANT/REVOKE permissions to be applied to all schema objects with one command The proposed syntax is: GRANT SELECT ON ALL TABLES IN public TO phpuser; GRANT SELECT ON NEW TABLES IN public TO phpuser; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 31 Jan 2009, at 17:30, Andrew Dunstan wrote: But the syntax you posted does not do this at all. Where does it restrict the grant to a single schema, like the syntax above? I am just starting the attempt here, obviously since I admit that my parser skills are next to none - I didn't address such issue. So far, testing this change - I can do issue GRANT SELECT ON ALL TABLES TO xxx, and it parses well. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 31 Jan 2009, at 17:28, David Fetter wrote: On Sat, Jan 31, 2009 at 05:24:15PM +, Grzegorz Jaskiewicz wrote: from http://wiki.postgresql.org/wiki/Todo: I see the TODO item, but I don't see anything in the SQL standard, which I think is something we should explore before making a potentially incompatible change. Personally, I don't think we should follow SQL 2008 to the letter. I am sure, many ppl miss that ability - I know I do, so I wanted to implement that right in the core. Wether SQL standard has something of same functionality but different syntax or not, well - I would love to know too - but I never read SQL standard - to be honest. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 1 Feb 2009, at 00:05, Joshua Tolley wrote:
to add new syntax, you might consider writing a function instead. This
function might take parameters such as the privilege to grant and
the user to
grant it to, and be called something like this:
SELECT my_grant_function('someuser', 'someprivilege');
Well, if you read my first post - I did wrote such a function, and it
works just fine. But still - since that was in TODO, I figured I might
give it a go.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 31 Jan 2009, at 17:17, Gregory Stark wrote: I don't see any reason offhand why it should have to be a reserved word though. You should be able to make it an UNRESERVED_KEYWORD. Oh, and you'll want to add it to the list of tokens in unreserved_keyword in gram.y as well. removing it from keywords.c and adding it to unserserved_keywords crowd didn't make it... so I'll stick with keywords.c for timebeing. So far I got mostly critique here, even tho - I haven't started much, which is quite sad in a way - because it is not very pro-creative, but I'll still continue on with the patch - whatever the outcome. ta. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] adding stuff to parser, question
On 31 Jan 2009, at 17:22, David Fetter wrote: Is this part of the SQL:2008? If not, is there something else that is? As far as I can see in the 'free' draft, no. Which is quite funny, cos 'DATABASE' keyword isn't there too.. Anyway. Looks like m$'s sybase clone got it, so I figure - at least some ppl figured it would be a nice feature ;) Can someone lead me into, how should I grab all Oid's for tables (in all user defined schemas, public. and others in db - except for system ones) in /src/backend/catalog/namespace.c please ? (which would probably be the best place to do it.) so far I figured, that adding another ACL_OBJECT define would be the best option, instead of passing NIL in objnames, for which there's an assert at begin of objectNamesToOids(). Reason I am asking about the backend/catalog approach, is because of all structures, and existing code (which I only got to go through Today for first time), I don't see any existing example, that would enumerate 'everything' in specific category. Thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updated backslash consistency patch
On 23 Jan 2009, at 00:03, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: Seeing this list reminded me of a pet-peeve.. \du and \dg actually show the same info, that's fine, but neither of them show the rolcanlogin value. +1 for fixing that. was it that easy, or I got it wrong? :) psql_du_conlogin.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More FOR UPDATE/FOR SHARE problems
On 2009-01-26, at 17:34, Kevin Grittner wrote: . It may not surprise someone who is intimately familiar with PostgreSQL internals for a single SELECT statement to see PART of a transactions work, but it would surprise most users, and is certainly not compliant with the standard. +1000 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, dropping a tablespace
On 2009-01-25, at 09:04, Simon Riggs wrote: On Sat, 2009-01-24 at 21:58 +0200, Heikki Linnakangas wrote: When replaying a DROP TABLE SPACE, you first try to remove the directory, and if that fails, you assume that it's because it's in use as a temp tablespace in a read-only transaction. That sounds like you think there another conclusion? What if subdirectory of that directory is owned by root ? Say I create /home/gj/tablespace1 . Surely emptying it is possible, but should it remove the dir altogether ? It is possible to create tablespace in that directory, even so postgres user doesn't own /home/gj/ directory. So why shouldn't it be possible to drop it ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Time to finalize patches for 8.4 beta
On 2009-01-24, at 15:33, Bruce Momjian wrote: The PostgreSQL community is considering including security enhancements in Postgres 8.4, e.g. row-level permissions and SE-Linux security. However, to evaluate the patch and its usefulness, we need security experts who want to use this capability or have used it in other databases. With that sort of features, usually what happens in other projects, is that despite their evaluation during beta period - they are explicitly marked and considered as 'beta', even when product reaches release status. Maybe PostgresSql should follow that tactic too. After all, security stuff needs to be tested and exposed for sometime before considered rock solid, and trustworthy. Which of course doesn't mean, that it shouldn't start in beta's. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fixes for compiler warnings
On 2009-01-18, at 09:56, Peter Eisentraut wrote: On Sunday 18 January 2009 08:28:51 Tom Lane wrote: Yeah, the risk this is trying to guard against is variables containing % unexpectedly. Even if that's not possible, it requires some work to verify and it's a bit fragile. I didn't look at the specific cases yet but in general I think this is a good policy. -Wformat-security warns about printf(var); but not about printf(var, a); I don't understand that; the crash or exploit potential is pretty much the same in both cases. not at all. First case allows you to pass in var from outside, with your, well crafted format strings. Please read more about subject, before you say something that silly. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery Test Framework
On 2009-01-12, at 16:48, Dave Page wrote: On Mon, Jan 12, 2009 at 4:37 PM, Joshua D. Drake j...@commandprompt.com wrote: On Mon, 2009-01-12 at 11:18 -0500, Tom Lane wrote: Basically I think we are up against the same type of project management decision we've had several times before: are we willing to slip the 8.4 release schedule for however long it will take for hot standby and the other replication-related features to be ready? I would certainly not like to see 8.4 slip. I would. PostgreSQL is not a commercial application which has to be released on schedule to satisfy shareholders - it's an Open Source project that aims to provide it's users with useful features. We have two extremely useful features here (hot standby and sync replication) which together will make this a killer release for many people - we can delay a month or two as required to polish and get them ready for release, or decide we're willing to wait another 12 - 14 months for them to be available for end users. I'd much rather see them included than deferred (particularly hot standby, parts of which have been awaiting review for months now anyway, through no fault of Simons). +1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] merry christmas folks
WITH RECURSIVE tree(b, l, lv) AS
(
(
WITH RECURSIVE t(b, l) AS
(
select b/11.6, 0 AS l from (select
generate_series(0,30)::float8 b union all select generate_series(30,0,
-1)::float8 b) ziew
UNION ALL
select (b*1.06), l+1 FROM t WHERE l 3
)
select b, l, 1 AS lv from t order by l asc
)
UNION ALL
select b*1.1, l+(lv*62) AS l , lv+1 FROM tree WHERE lv 4
)
select array_to_string(array_agg(SUBSTRING(' ',
LEAST(GREATEST(b::integer,1),6), 1)),'') from tree group by l order by
l asc;
;]
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
