Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Heikki Linnakangas
Split of an internal node works like this: 1. Gather all the existing tuples on the page, plus the new tuple being inserted. 2. Call picksplit on the tuples, to divide them into pages 3. Go through all tuples on the buffer associated with the page, and divide them into buffers on the new

Re: [HACKERS] Possible Bug in pg_upgrade

2011-08-11 Thread Peter Eisentraut
On ons, 2011-08-10 at 18:53 -0400, Tom Lane wrote: Dave Byrne dby...@mdb.com writes: Attached is a patch that skips orphaned temporary relations in pg_upgrade if they are lingering around. It works for 9.0 - 9.1 upgrades, however I wasn't able to tell when pg_class.relistemp was added so

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-11 Thread Peter Eisentraut
On ons, 2011-08-10 at 14:19 -0400, Tom Lane wrote: One thing that might be reasonable would be to move the digest() functions digest(data text, type text) returns bytea digest(data bytea, type text) returns bytea from pgcrypto into core, ... which this approach would create,

Re: [HACKERS] XMLATTRIBUTES vs. values of type XML

2011-08-11 Thread Peter Eisentraut
On fre, 2011-07-29 at 11:37 +0200, Florian Pflug wrote: On Jul28, 2011, at 22:51 , Peter Eisentraut wrote: On ons, 2011-07-27 at 23:21 +0200, Florian Pflug wrote: On Jul27, 2011, at 23:08 , Peter Eisentraut wrote: Well, offhand I would expect that passing an XML value to XMLATTRIBUTES

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Alexander Korotkov
On Thu, Aug 11, 2011 at 10:21 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Split of an internal node works like this: 1. Gather all the existing tuples on the page, plus the new tuple being inserted. 2. Call picksplit on the tuples, to divide them into pages 3. Go

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-11 Thread Magnus Hagander
On Thu, Aug 11, 2011 at 09:06, Peter Eisentraut pete...@gmx.net wrote: On ons, 2011-08-10 at 14:19 -0400, Tom Lane wrote: One thing that might be reasonable would be to move the digest() functions     digest(data text, type text) returns bytea     digest(data bytea, type text) returns

Re: [HACKERS] XMLATTRIBUTES vs. values of type XML

2011-08-11 Thread Florian Pflug
On Aug11, 2011, at 09:16 , Peter Eisentraut wrote: On fre, 2011-07-29 at 11:37 +0200, Florian Pflug wrote: On Jul28, 2011, at 22:51 , Peter Eisentraut wrote: On ons, 2011-07-27 at 23:21 +0200, Florian Pflug wrote: On Jul27, 2011, at 23:08 , Peter Eisentraut wrote: Well, offhand I would expect

Re: [HACKERS] pgstat wait timeout warnings

2011-08-11 Thread Bernd Helmle
--On 10. August 2011 21:54:06 +0300 Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So my theory is that if the I/O is really busy, write() on the stats file blocks for more than 5 seconds, and you get the timeout. I've seen it on customer instances with very high INSERT

Re: [HACKERS] pgstat wait timeout warnings

2011-08-11 Thread Andres Freund
On Thursday, August 11, 2011 11:49:12 Bernd Helmle wrote: --On 10. August 2011 21:54:06 +0300 Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So my theory is that if the I/O is really busy, write() on the stats file blocks for more than 5 seconds, and you get the timeout.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Alexander Korotkov
On Wed, Aug 10, 2011 at 11:45 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: unloadNodeBuffers() is now dead code. processEmptyingStack calls it. LEAF_PAGES_STATS_* are unused now. Removed. Should avoid calling smgrnblocks() on every tuple, the overhead of that could

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Heikki Linnakangas
On 10.08.2011 22:44, Alexander Korotkov wrote: Manual and readme updates. Thanks, I'm reviewing these now. Do we want to expose the level-step and buffersize parameters to users? They've been useful during testing, but I'm thinking we should be able to guess good enough values for them

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-11 Thread Marko Kreen
On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: I would like to see whether there is support for adding sha1 and sha2 functions into the core. I can't get excited about that, but could put up with it as long as there wasn't scope

Re: [HACKERS] compiling 9.2 : WinXp+mingw

2011-08-11 Thread Andrew Dunstan
On 08/11/2011 08:58 AM, pasman pasmański wrote: Hi. Compiling 9.1 sources from packed tgz file works ok. When compile sources loaded from git repository, configure pass without errors, config.log looks ok, but in src/include/pg_config.h aren't any defines, only undefines. It seems like

Re: [HACKERS] compiling 9.2 : WinXp+mingw

2011-08-11 Thread pasman pasmański
Its problem on my computer. But prepacked souces compile good. Maybe git need specific configuration for postgres ? Or some buggy sed/gawk/grep skip pg_config.h file? 2011/8/11, pasman pasmański pasma...@gmail.com: Hi. Compiling 9.1 sources from packed tgz file works ok. When compile

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Heikki Linnakangas
On 10.08.2011 22:44, Alexander Korotkov wrote: Manual and readme updates. I went through these, and did some editing and rewording. Attached is an updated README, and an updated patch of the doc changes. Let me know if I screwed up something. -- Heikki Linnakangas EnterpriseDB

Re: [HACKERS] small issue with host names in hba

2011-08-11 Thread Robert Haas
On Tue, Aug 9, 2011 at 2:16 PM, Peter Eisentraut pete...@gmx.net wrote: When a host name is used in pg_hba.conf, then we call pg_getnameinfo_all() to get the host name for the client's IP address, either in postmaster.c or in hba.c, whichever happens first.  But if the IP address has no host

Re: [HACKERS] compiling 9.2 : WinXp+mingw

2011-08-11 Thread Andrew Dunstan
On 08/11/2011 09:42 AM, pasman pasmański wrote: Its problem on my computer. But prepacked souces compile good. Maybe git need specific configuration for postgres ? Or some buggy sed/gawk/grep skip pg_config.h file? 1. A: Because we read from top to bottom, left to right. Q: Why

Re: [HACKERS] pgstat wait timeout warnings

2011-08-11 Thread Tom Lane
Andres Freund and...@anarazel.de writes: --On 10. August 2011 21:54:06 +0300 Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So my theory is that if the I/O is really busy, write() on the stats file blocks for more than 5 seconds, and you get the timeout. Yes, I have seen it

Re: [HACKERS] pgstat wait timeout warnings

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 10:30 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@anarazel.de writes: --On 10. August 2011 21:54:06 +0300 Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So my theory is that if the I/O is really busy, write() on the stats file blocks for

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-11 Thread Tom Lane
Marko Kreen mark...@gmail.com writes: On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... which this approach would create, because digest() isn't restricted to just those algorithms.  I think it'd be better to just invent two new functions, which also avoids issues for

Re: [HACKERS] psql: display of object comments

2011-08-11 Thread Robert Haas
2011/8/9 Shigeru Hanada shigeru.han...@gmail.com: I'm the author of that patch, sorry for confusion.  May I explain the background of implementing those command? :) Basically, during implementing foreign table support, I tried to follow the existing design. I found two backslash command

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-11 Thread Andrew Dunstan
On 08/11/2011 10:46 AM, Tom Lane wrote: Marko Kreenmark...@gmail.com writes: On Wed, Aug 10, 2011 at 9:19 PM, Tom Lanet...@sss.pgh.pa.us wrote: ... which this approach would create, because digest() isn't restricted to just those algorithms. I think it'd be better to just invent two new

[HACKERS] Backup's from standby

2011-08-11 Thread senthilnathan
What is the basic reason for restricting backup at standby server(pg_start_backup)? We are doing the following steps to take backup from standby which(backups) might be used for standalone startups. 1.Create file system snapshot(LVM) 2.take the back up of data directory including wal files

Re: [HACKERS] psql: display of object comments

2011-08-11 Thread Robert Haas
On Mon, Aug 8, 2011 at 6:01 PM, Josh Kupershmidt schmi...@gmail.com wrote: On Mon, Aug 8, 2011 at 4:34 PM, Robert Haas robertmh...@gmail.com wrote: OK, I've now committed most of this, with some additions to the documentation.  Remaining bits attached. Looks good, thanks. And now I've

Re: [HACKERS] psql document fix about showing FDW options

2011-08-11 Thread Robert Haas
2011/8/9 Shigeru Hanada shigeru.han...@gmail.com: postgres=# \d pgbench_accounts       Foreign table public.pgbench_accounts  Column  |     Type      | Modifiers |    Options --+---+---+---  aid      | integer       | not null  | {colname=aid}  bid    

Re: [HACKERS] per-column FDW options, v5

2011-08-11 Thread Robert Haas
2011/8/9 Shigeru Hanada shigeru.han...@gmail.com: (2011/08/09 1:16), Robert Haas wrote: 2011/8/8 Shigeru Hanadashigeru.han...@gmail.com: Currently table-level options are showin in result of \det+ command (only verbose mode), in same style as fdw and foreign servers. But \d is more popular

Re: [HACKERS] Extra check in 9.0 exclusion constraint unintended consequences

2011-08-11 Thread Robert Haas
On Sun, Jul 10, 2011 at 3:29 PM, Jeff Davis pg...@j-davis.com wrote: On Sun, 2011-07-10 at 00:36 -0400, Alvaro Herrera wrote: Is this really a good idea?  I think the note should still be there in 9.1 and beyond (with the version applicability note of course) I see your point, but it also

[HACKERS] plpython crash

2011-08-11 Thread Jean-Baptiste Quenot
Hi there, plpython crashes on me on various 64-bit Ubuntu hosts, see the gdb backtrace at: https://gist.github.com/1140005 Do you believe there was recent bugfixes regarding PLyMapping_ToTuple() ? This is PG 9.0.4 with HEAD of plpython taken in march 2011 and backported. Please tell me if you

Re: [HACKERS] per-column FDW options, v5

2011-08-11 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue ago 11 11:50:40 -0400 2011: 2011/8/9 Shigeru Hanada shigeru.han...@gmail.com: (3) OPTIONS clause style Show FDW options as they were in OPTIONS clause.  Each option is shown as key 'value', and delimited with ','.     Ex)     FDW Options:

Re: [HACKERS] per-column FDW options, v5

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 12:04 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of jue ago 11 11:50:40 -0400 2011: 2011/8/9 Shigeru Hanada shigeru.han...@gmail.com: (3) OPTIONS clause style Show FDW options as they were in OPTIONS clause.  Each option

[HACKERS] Final 48 for PgWest CFP

2011-08-11 Thread Joshua D. Drake
Hello, We are in the final 48 hours of the CFP for PgWest. Let's get those talks in. https://www.postgresqlconference.org/talk_types Sincerely, Joshua D. Drake -- Command Prompt, Inc. - http://www.commandprompt.com/ PostgreSQL Support, Training, Professional Services and Development The

Re: [HACKERS] Backup's from standby

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 1:02 AM, senthilnathan senthilnatha...@gmail.com wrote: What is the basic reason for restricting backup at standby server(pg_start_backup)? It performs a checkpoint. We are doing the following steps to take backup from standby which(backups) might be used for

Re: [HACKERS] small issue with host names in hba

2011-08-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Aug 9, 2011 at 2:16 PM, Peter Eisentraut pete...@gmx.net wrote: But I'm a little confused by what this code is really trying to accomplish: ... I think the intended behavior of NI_NUMERICHOST is to suppress the name lookup, and return the

Re: [HACKERS] Transient plans versus the SPI API

2011-08-11 Thread Dimitri Fontaine
Hannu Krosing ha...@krosing.net writes: Hm, you mean reverse-engineering the parameterization of the query? Yes, basically re-generate the query after (or while) parsing, replacing constants and arguments with another set of generated arguments and printing the list of these arguments at the

Re: [HACKERS] Extra check in 9.0 exclusion constraint unintended consequences

2011-08-11 Thread Jeff Davis
On Thu, 2011-08-11 at 11:58 -0400, Robert Haas wrote: I'm OK with adding a note either to the 9.0 docs only (which means it might be missed by a 9.0 user who only looks at the current docs) or with adding a note to all versions mentioning the difference in behavior with 9.0, but I'm not really

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Alexander Korotkov
On Thu, Aug 11, 2011 at 2:28 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 10.08.2011 22:44, Alexander Korotkov wrote: Manual and readme updates. Thanks, I'm reviewing these now. Do we want to expose the level-step and buffersize parameters to users? They've been

Re: [HACKERS] Reduce WAL logging of INSERT SELECT

2011-08-11 Thread Bruce Momjian
Simon Riggs wrote: On Sat, Aug 6, 2011 at 4:16 AM, Bruce Momjian br...@momjian.us wrote: Well, if the table is created in the same transaction (which is the only case under consideration), no other sessions can write to the table so you are just writing the entire table on commit, rather

Re: [HACKERS] index-only scans

2011-08-11 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 1. The way that nodeIndexscan.c builds up the faux heap tuple is perhaps susceptible to improvement. I thought about building a virtual tuple, but then what do I do with an OID column, if I have one? Or maybe this should be done some

Re: [HACKERS] index-only scans

2011-08-11 Thread Joshua D. Drake
On 08/11/2011 01:57 PM, Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 1. The way that nodeIndexscan.c builds up the faux heap tuple is perhaps susceptible to improvement. I thought about building a virtual tuple, but then what do I do with an OID column, if I

Re: [HACKERS] index-only scans

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 4:57 PM, Greg Sabino Mullane g...@turnstep.com wrote: 1. The way that nodeIndexscan.c builds up the faux heap tuple is perhaps susceptible to improvement.  I thought about building a virtual tuple, but then what do I do with an OID column, if I have one?  Or maybe this

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-11 Thread Tom Lane
I wrote: I still haven't reproduced the behavior here, but I think I see what must be happening: we are getting an sinval reset while attempting to open pg_class_oid_index. After a number of false starts, I've managed to reproduce this behavior locally. The above theory turns out to be wrong,

Re: [HACKERS] index-only scans

2011-08-11 Thread Cédric Villemain
2011/8/11 Robert Haas robertmh...@gmail.com: Please find attached a patch implementing a basic version of index-only scans.  This patch is the work of my colleague Ibrar Ahmed and myself, and also incorporates some code from previous patches posted by Heikki Linnakanagas. Great!. I'm able

Re: [HACKERS] index-only scans

2011-08-11 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Maybe it's time to finally remove the been-deprecated-for-a-while OIDs? I thought about just not supporting that for index-only scans, but system catalogs use them pretty extensively, and it doesn't seem out of the question that that could

Re: [HACKERS] index-only scans

2011-08-11 Thread Andrew Dunstan
On 08/11/2011 09:44 PM, Greg Sabino Mullane wrote: I guess the question that should be asked is we are going to finally remove OIDs someday, right?. If so, and if it's potentially blocking a major new feature, why not now? It seems a bit odd then that we added ALTER TABLE SET WITH OIDS

Re: [HACKERS] psql document fix about showing FDW options

2011-08-11 Thread Shigeru Hanada
(2011/08/12 0:48), Robert Haas wrote: 2011/8/9 Shigeru Hanadashigeru.han...@gmail.com: postgres=# \d pgbench_accounts Foreign table public.pgbench_accounts Column | Type | Modifiers |Options --+---+---+--- aid | integer

Re: [HACKERS] index-only scans

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 5:39 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 2011/8/11 Robert Haas robertmh...@gmail.com: Please find attached a patch implementing a basic version of index-only scans.  This patch is the work of my colleague Ibrar Ahmed and myself, and also

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 5:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: I can reproduce the problem fairly conveniently with this crude hack: diff --git a/src/backend/storage/ipc/sinval.c b/src/backend/storage/ipc/sinval.c index 8499615..5ad2aee 100644 *** a/src/backend/storage/ipc/sinval.c

Re: [HACKERS] psql document fix about showing FDW options

2011-08-11 Thread Robert Haas
2011/8/11 Shigeru Hanada shigeru.han...@gmail.com: Yeah, I have (hopefully) working FDW for PostgreSQL which is based on the one which has been proposed for 9.1, and updates done by Heikki. I've implemented:  * SELECT clause omitting  * WHERE clause pushdown (assuming remote has same

Re: [HACKERS] index-only scans

2011-08-11 Thread Robert Haas
On Thu, Aug 11, 2011 at 9:44 PM, Greg Sabino Mullane g...@turnstep.com wrote: Maybe it's time to finally remove the been-deprecated-for-a-while OIDs? I thought about just not supporting that for index-only scans, but system catalogs use them pretty extensively, and it doesn't seem out of the

[HACKERS] our buffer replacement strategy is kind of lame

2011-08-11 Thread Robert Haas
While I was poking around at the index-only scans patch, I couldn't help noticing that our buffer replacement algorithm leaves something to be desired. Here's the query again: select sum(aid) from sample_data a1 where exists (select * from pgbench_accounts a where a.aid = a1.aid and a.aid !=