Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Simon Riggs
On Sun, 2009-01-04 at 21:03 +1300, Mark Kirkwood wrote: bench=# select now(),count(*) from history; ERROR: could not open relation base/16384/16394: No such file or directory Thanks for the report. I'm attempting to recreate now. -- Simon Riggs www.2ndQuadrant.com PostgreSQL

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Mark Kirkwood
Simon Riggs wrote: On Wed, 2008-12-17 at 15:21 +, Simon Riggs wrote: http://wiki.postgresql.org/wiki/Hot_Standby now contains a link to latest version of this patch. v6 of Hot Standby now uploaded to Wiki (link above), with these changes: * Must ignore_killed_tuples and never

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Mark Kirkwood
Simon Riggs wrote: On Sun, 2009-01-04 at 21:03 +1300, Mark Kirkwood wrote: bench=# \d history Table public.history Column |Type | Modifiers +-+--- tid| integer | bid| integer

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Simon Riggs
On Sun, 2009-01-04 at 22:18 +1300, Mark Kirkwood wrote: bench=# select now(),count(*) from history; ERROR: could not open relation base/16384/16394: No such file or directory I'm guessing something tied up with the fact that history has no rows to start with... Good guess,

Re: [HACKERS] lazy_truncate_heap()

2009-01-04 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-01-01 at 12:00 +0200, Heikki Linnakangas wrote: Greg Stark wrote: On 31 Dec 2008, at 13:21, Simon Riggs si...@2ndquadrant.com wrote: Both of these bugs are minor, but the effect of either/both of them is to cause more AccessExclusiveLocks than we might expect.

Re: [HACKERS] New patch for Column-level privileges

2009-01-04 Thread Stephen Frost
Markus, * Markus Wanner (mar...@bluegap.ch) wrote: Stephen Frost wrote: ..in the attached patch. Thanks, I've reviewed this patch again. Thanks! Currently, column-level privileges are not honored when JOINs are involved (you must have the necessary table-level privileges, as you do

Re: [HACKERS] parallel restore

2009-01-04 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I propose to commit this unless someone wants more time for reviewing. A moment's eyeballing of the patch finds rather a lot of debugging cruft yet (inserted printfs, #ifdef WIN32blah, etc); also I think the addition to

Re: [HACKERS] New patch for Column-level privileges

2009-01-04 Thread Markus Wanner
Hello Stephen, Stephen Frost wrote: I'm going to look into it but it's a bit complicated. I was hoping someone who's more familiar with those parts would be able to look at it. Good to hear. I've just been asking, because it remained unclear to me. I don't think that's the right approach,

Re: [HACKERS] parallel restore

2009-01-04 Thread Simon Riggs
On Mon, 2008-12-29 at 18:42 -0500, Andrew Dunstan wrote: Attached is the latest parallel restore patch. I think this is getting fairly close. Includes help text, docco and some extra error checking. Very brief review. Hopefully the --truncate-before-load option works in both parallel mode

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Simon Riggs
On Sun, 2009-01-04 at 21:03 +1300, Mark Kirkwood wrote: bench=# \d history Table public.history Column |Type | Modifiers +-+--- tid| integer | bid| integer |

Re: [HACKERS] pg_stats queries versus per-database encodings

2009-01-04 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: We could attack this by including source database's encoding in the shared-memory entries, and performing a conversion on the fly when reading out the data. However, what happens if the conversion fails? The most

Re: [HACKERS] parallel restore

2009-01-04 Thread Alvaro Herrera
Andrew Dunstan wrote: Attached is the latest parallel restore patch. I think this is getting fairly close. Some random comments - please #define the return type of prestore(). Also, that it returns in one platform and exits in another seems weird as an API. I think it should return in

Re: [HACKERS] New patch for Column-level privileges

2009-01-04 Thread Tom Lane
Markus Wanner mar...@bluegap.ch writes: Stephen Frost wrote: BTW: how are long constant strings expected to be formatted? Are those allowed to exceed 80 columns, or are they expected to be split like so Honestly, I think I've seen both done. Yeah, that's why I'm asking. IMHO, the trouble

Re: [HACKERS] parallel restore

2009-01-04 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: I propose to commit this unless someone wants more time for reviewing. A moment's eyeballing of the patch finds rather a lot of debugging cruft yet (inserted printfs, #ifdef WIN32blah, etc); also I think the addition to include/port.h belongs in

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Simon Riggs
On Sun, 2009-01-04 at 22:13 +1300, Mark Kirkwood wrote: Simon Riggs wrote: Is the file actually missing? i.e. ls -l mydatadir/base/16384/16394* Yeah - $ ls -l $PGDATA/base/16384/16394* ls: /data0/pgslave/8.4/base/16384/16394*: No such file or directory What else is missing? Files,

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Mark Kirkwood
Mark Kirkwood wrote: Simon Riggs wrote: On Sun, 2009-01-04 at 21:03 +1300, Mark Kirkwood wrote: bench=# \d history Table public.history Column |Type | Modifiers +-+--- tid| integer |

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Heikki Linnakangas
Simon Riggs wrote: On Sun, 2009-01-04 at 22:18 +1300, Mark Kirkwood wrote: bench=# select now(),count(*) from history; ERROR: could not open relation base/16384/16394: No such file or directory I'm guessing something tied up with the fact that history has no rows to start with...

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Greg Stark
What I ifind interesting about this is that whereas I had been concerned that adding hot standby late in the development cycle might be destabilize the tree and add lots of time to the release cycle it seems having it might actually increase our ability to see problems in the recovery code

[HACKERS] 8.4 - psql output for \l

2009-01-04 Thread hubert depesz lubaczewski
is it going to stay that way? i find it actually worse than 8.3 behaviour: (dep...@[local]:5840) 11:43:40 [depesz] # \l List of databases Name| Owner | Encoding | Collation |Ctype| Access privileges

Re: [HACKERS] New patch for Column-level privileges

2009-01-04 Thread Markus Wanner
Hello Stephen, Stephen Frost wrote: ..in the attached patch. Thanks, I've reviewed this patch again. Please find attached an updated patch for column-level privileges which incorporates Alvaro's suggested changes and is updated to the latest CVS HEAD. Cool, applies cleanly and compiles

Re: [HACKERS] lazy_truncate_heap()

2009-01-04 Thread Simon Riggs
On Sun, 2009-01-04 at 13:01 +0200, Heikki Linnakangas wrote: Why does an AccessExclusiveLock lead to cancelled queries or pausing WAL application? I thought it'd just block other queries trying to acquire a conflicting lock in the standby, just like holding an AccessExclusiveLock on the

Re: [HACKERS] pg_stats queries versus per-database encodings

2009-01-04 Thread Heikki Linnakangas
Tom Lane wrote: I notice that the pg_stat_statements patch is applying pg_mbcliplen() to query strings, in the fond illusion that it knows what encoding they are in. This brings up a bigger issue, namely that pg_stat_activity isn't exactly encoding-proof either --- whatever encoding is in use

Re: [HACKERS] Latest version of Hot Standby patch: unexpected error querying standby

2009-01-04 Thread Heikki Linnakangas
Heikki Linnakangas wrote: I can reproduce that too with CVS HEAD, so it's clearly a bug. I probably introduced it with the recent smgr changes; I'll try to hunt it down. Now that was an embarrassingly simple bug: --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@

Re: [HACKERS] dblink vs SQL/MED

2009-01-04 Thread Martin Pihlak
Joe Conway wrote: Two specific questions on this approach: 1. This implies that the exact same dblink_connstr_check() is performed on a predefined foreign server and user mapping as a raw connstr -- is this desireable? I'm not entirely clear on the intended purpose and use of foreign

[HACKERS] dblink vs SQL/MED - security and implementation details

2009-01-04 Thread Joe Conway
(changed the subject to hopefully get a few more eyes looking at this thread) Martin Pihlak wrote: I'd vote for allowing aribitrary connect strings -- ordinary users cannot create servers and user mappings unless explicitly granted the privileges. It probably should be noted in the

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Alvaro Herrera
Alvaro Herrera wrote: Some notes about this patch: - the string type handling (basically all the new code) is untested. I'll have a look tomorrow at the btree test code I sent the other day to add a string option and see how it goes. Okay, it was basically fine except for the attached

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Alvaro Herrera
Simon Riggs wrote: Custom variable classes are often useful, but they are system wide. It would be good to be able to use table-level options and have them work very similarly to something we already have. Table-level options are just an obvious normalisation of how we handle parameters.

Re: [HACKERS] contrib/pg_stat_statements 1226

2009-01-04 Thread Tom Lane
I wrote: I'm not enamored of saved_file either, it seems like the wrong part of speech somehow. Maybe save_in_file? Actually ... what is the point of letting users control the filename at all? It seems like the only useful nondefault value is the empty string (to suppress storing the stats).

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Alvaro Herrera
Alvaro Herrera wrote: Okay, it was basically fine except for the attached minor correction. Warning: I intend to commit this patch fairly soon! This is the patch in its final form. I have included a few macros to simplify the writing of amoptions routines. -- Alvaro Herrera

Re: [HACKERS] Significantly larger toast tables on 8.4?

2009-01-04 Thread James Mansion
Peter Eisentraut wrote: c. Are there any well-known pitfalls/objections which would prevent me from changing the algorithm to something more efficient (read: IO-bound)? copyright licenses and patents Would it be possible to have a plugin facility? I guess the most likely candidate

Re: [HACKERS] contrib/pg_stat_statements 1226

2009-01-04 Thread Tom Lane
ITAGAKI Takahiro itagaki.takah...@oss.ntt.co.jp writes: Here is an updated version of contrib/pg_stat_statements patch. I've committed this with significant revisions. Other than the points already mentioned in previous messages: * I removed the proposed changes to the behavior of the core

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Alex Hunsaker
On Sun, Jan 4, 2009 at 15:01, Alvaro Herrera alvhe...@commandprompt.com wrote: Alvaro Herrera wrote: Okay, it was basically fine except for the attached minor correction. Warning: I intend to commit this patch fairly soon! This is the patch in its final form. I have included a few macros to

[HACKERS] [PATCH] ALTER TABLE SET (compress_max_size... = )

2009-01-04 Thread Alex Hunsaker
This patch lets you control 3 pg_lzcompress knobs on a per table basis (note requires reloptions.patch) compress_max_size: Controls the maximum size to be considered for TOAST compression. compress_min_rate: Minimum compression rate (0-100%) required for TOAST compression to be used.

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: This is the patch in its final form. I have included a few macros to simplify the writing of amoptions routines. Minor gripes: * Does initialize_reloptions() need to be exported? It seems to be only called within parseRelOptions(). It's far

Re: [HACKERS] WIP patch for basic window frame support

2009-01-04 Thread Hitoshi Harada
2008/12/31 Tom Lane t...@sss.pgh.pa.us: No docs or regression tests yet, but it seems to work ... please check against Oracle and DB2 behavior. Supported cases are RANGE UNBOUNDED PRECEDING -- same as AND CURRENT ROW ROWS UNBOUNDED PRECEDING-- same as AND CURRENT

Re: [HACKERS] WIP patch for basic window frame support

2009-01-04 Thread Hitoshi Harada
2009/1/5 Tom Lane t...@sss.pgh.pa.us: Hitoshi Harada umi.tan...@gmail.com writes: 2008/12/31 Tom Lane t...@sss.pgh.pa.us: RANGE UNBOUNDED PRECEDING -- same as AND CURRENT ROW ROWS UNBOUNDED PRECEDING-- same as AND CURRENT ROW Is this true? 7.11 syntax rule 9 says so. AFAICS

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread KaiGai Kohei
Alvaro Herrera wrote: Alvaro Herrera wrote: Okay, it was basically fine except for the attached minor correction. Warning: I intend to commit this patch fairly soon! This is the patch in its final form. I have included a few macros to simplify the writing of amoptions routines. Thanks

[HACKERS] QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)

2009-01-04 Thread Stephen R. van den Berg
I asked the author of the QuickLZ algorithm about licensing... Sounds like he is willing to cooperate. This is what I got from him: On Sat, Jan 3, 2009 at 17:56, Lasse Reinhold l...@quicklz.com wrote: Hi Stephen, That sounds really exciting, I'd love to see QuickLZ included into PostgreSQL.

Re: [HACKERS] Significantly larger toast tables on 8.4?

2009-01-04 Thread Stephen R. van den Berg
James Mansion wrote: Peter Eisentraut wrote: c. Are there any well-known pitfalls/objections which would prevent me from changing the algorithm to something more efficient (read: IO-bound)? copyright licenses and patents Would it be possible to have a plugin facility? I guess the most

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Alvaro Herrera
KaiGai Kohei wrote: (1) Who/Where should allocate a string area? + /* Note that this assumes that the variable is already allocated! */ + #define HANDLE_STRING_RELOPTION(optname, var, option) \ + if (HAVE_RELOPTION(optname, option))\ + {

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread Alvaro Herrera
Alex Hunsaker escribió: On Sun, Jan 4, 2009 at 15:01, Alvaro Herrera alvhe...@commandprompt.com wrote: Alvaro Herrera wrote: Okay, it was basically fine except for the attached minor correction. Warning: I intend to commit this patch fairly soon! This is the patch in its final form.

[HACKERS] Export IsUnderPostmaster for pg_stat_statements on win32

2009-01-04 Thread ITAGAKI Takahiro
Hi, I compiled pg_stat_statements on mingw, and got the following error. PGDLLIMPORT for IsUnderPostmaster seems to be needed on win32. Info: resolving _IsUnderPostmaster by linking to __imp__IsUnderPostmaster (auto-import) fu01.o:(.idata$2+0xc): undefined reference to

Re: [HACKERS] QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)

2009-01-04 Thread Alvaro Herrera
On Sat, Jan 3, 2009 at 17:56, Lasse Reinhold l...@quicklz.com wrote: That sounds really exciting, I'd love to see QuickLZ included into PostgreSQL. I'd be glad to offer support and add custom optimizations, features or hacks or whatever should turn up. My only concern is to avoid

[HACKERS] PROPOSAL: geqo improvement

2009-01-04 Thread marcin mank
Hello, List. There are cases when GEQO returns a very bad plan in some rare executions of a query. To decrease likehood of this happening, I propose: When GEQO detects that what it found is in fact a miserable plan it restarts the search. Simple math shows that if the probability of a bad plan

[HACKERS] Many loaded library logs by preload libraries

2009-01-04 Thread ITAGAKI Takahiro
Hi, If we set shared_preload_libraries or local_preload_libraries to load some modules, loaded library messages are logged in server log every new connections and autovacuum workers. LOG: loaded library pg_stat_statements Messages by shared_preload_libraries are logged only once on

Re: [HACKERS] PROPOSAL: geqo improvement

2009-01-04 Thread Robert Haas
2009/1/4 marcin mank marcin.m...@gmail.com: GEQO would decide that the plan is bad when the calculated cost of the plan would exceed the time spent planning so far a fixed number of times (100 ? a configurable parameter ?) . I think a function infering cost from time spent could be calculated

Re: [HACKERS] PROPOSAL: geqo improvement

2009-01-04 Thread marcin mank
It sounds like you're proposing to compare the time spent planning to the estimated execution time. AFAICS, those things are unrelated, so I'm not sure what you hope to figure out by comparing them. The idea is: If we are to spend a LOT of resources executing the query, we might as well burn

Re: [HACKERS] WIP patch for basic window frame support

2009-01-04 Thread Tom Lane
Hitoshi Harada umi.tan...@gmail.com writes: 2008/12/31 Tom Lane t...@sss.pgh.pa.us: RANGE UNBOUNDED PRECEDING -- same as AND CURRENT ROW ROWS UNBOUNDED PRECEDING-- same as AND CURRENT ROW Is this true? 7.11 syntax rule 9 says so. AFAICS general rule 5b doesn't discuss the case

Re: [HACKERS] Export IsUnderPostmaster for pg_stat_statements on win32

2009-01-04 Thread Alvaro Herrera
ITAGAKI Takahiro wrote: Hi, I compiled pg_stat_statements on mingw, and got the following error. PGDLLIMPORT for IsUnderPostmaster seems to be needed on win32. Applied, thanks. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication,

Re: [HACKERS] generic reloptions improvement

2009-01-04 Thread KaiGai Kohei
Alvaro Herrera wrote: KaiGai Kohei wrote: (1) Who/Where should allocate a string area? + /* Note that this assumes that the variable is already allocated! */ + #define HANDLE_STRING_RELOPTION(optname, var, option) \ + if (HAVE_RELOPTION(optname, option))

Re: [HACKERS] contrib/pg_stat_statements 1226

2009-01-04 Thread ITAGAKI Takahiro
Thank you for many works. Tom Lane t...@sss.pgh.pa.us wrote: 1. The proposed change to track system/user CPU time presents an enormous cost, and no argument has been made to show that there is any comparable benefit. 2. I'm unconvinced by the proposed changes to accumulate backend-local

[HACKERS] SPI nesting in plperl

2009-01-04 Thread Tom Lane
I tried fixing this http://archives.postgresql.org/pgsql-general/2009-01/msg00030.php by inserting SPI_push/SPI_pop calls around plperl's use of InputFunctionCall and OutputFunctionCall. Unfortunately it soon turned into a mess, because there are various control paths through that code and some

Re: [HACKERS] Many loaded library logs by preload libraries

2009-01-04 Thread Tom Lane
ITAGAKI Takahiro itagaki.takah...@oss.ntt.co.jp writes: If we set shared_preload_libraries or local_preload_libraries to load some modules, loaded library messages are logged in server log every new connections and autovacuum workers. LOG: loaded library pg_stat_statements Yeah, I was