Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Heikki Linnakangas
On 17.01.2011 22:33, Tom Lane wrote: Peter Eisentrautpete...@gmx.net writes: On mån, 2011-01-17 at 07:35 +0100, Magnus Hagander wrote: In fact, aren't there cases where the *length test* also fails? Currently, two text values are only equal of strcoll() considers them equal and the bits

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Itagaki Takahiro
On Tue, Jan 18, 2011 at 05:39, Tom Lane t...@sss.pgh.pa.us wrote: I haven't looked at this patch, but it seems to me that it would be reasonable to conclude A != B if the va_extsize values in the toast pointers don't agree. It's a very light-weight alternative of memcmp the byte data, but

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Mark Kirkwood
On 18/01/11 18:04, Tom Lane wrote: David Fetterda...@fetter.org writes: Who's the copyright holder(s)? If it's all individual contributors, Red Hat policy is not in play. Sorry David, it was written on the company's dime. However, I doubt that Red Hat derives any value from this useful

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 08:21, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Jan 18, 2011 at 4:15 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I also find it weird that incoming replication connections are logged by default. In the standby, we also log streaming

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/17/2011 07:58 PM, Kääriäinen Anssi wrote: The issue I saw was this: assume you have an extension foo, containing one function, test(). CREATE EXTENSION foo; DROP FUNCTION test(); -- restricted due to dependency ALTER FUNCTION test() RENAME TO test2; DROP FUNCTION test2(); -- not

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: Robert Haas wrote: a quick-and-dirty attempt to limit the amount of I/O caused by hint bits. I'm still very interested in knowing what people think about that. I found the elimination of the response-time spike promising. I don't think

Re: [HACKERS] Confusing comment in TransactionIdIsInProgress

2011-01-18 Thread Heikki Linnakangas
On 18.01.2011 07:15, Jim Nasby wrote: Shouldn't the comment read If first time through? /* * If not first time through, get workspace to remember main XIDs in. We * malloc it permanently to avoid repeated palloc/pfree overhead. */ if (xids == NULL)

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/17/2011 06:53 PM, Dimitri Fontaine wrote: Usability review: The patch implements a way to create extensions. While the patch is labeled extensions support for pg_dump, it actually implements more. It implements a new way to package and install extension, and changes contrib extensions to

Re: [HACKERS] Determining client_encoding from client locale

2011-01-18 Thread Susanne Ebrecht
On 17.01.2011 20:18, Peter Eisentraut wrote: That may be worth investigating, but I don't think it's related to the present patch. As I already said - not at all. The patch was ok for me. Susanne -- Susanne Ebrecht - 2ndQuadrant PostgreSQL Development, 24x7 Support, Training and Services

Re: [HACKERS] Warning compiling pg_dump (MinGW, Windows XP)

2011-01-18 Thread Pavel Golub
Hello, Andrew. You wrote: AD On 01/17/2011 03:51 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 01/17/2011 07:18 AM, Pavel Golub wrote: So you think I should just ignore these warnings? Because I can't remember the same behaviour on 8.x branches... We've had them all

Re: [HACKERS] MingW + GCC 4.5.2 + Relocate libpq.dll = SegFault

2011-01-18 Thread Pavel Golub
Hello, Charlie. Can you please express your opinion about my request Warning compiling pg_dump (MinGW, Windows XP) to pgsql-hackers on Thu, 13 Jan 2011. Do you have the same warnings using MinGW environment? You wrote: CS I'm compiling postgresql 9.0.2 using msys + mingw + gcc 4.5.2 (latest CS

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Fujii Masao
On Mon, Jan 17, 2011 at 10:30 PM, Fujii Masao masao.fu...@gmail.com wrote: Though I haven't seen the core part of the patch (i.e., ReceiveTarFile, etc..) yet, here is the comments against others. Here are another comments: When I untar the tar file taken by pg_basebackup, I got the following

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread tv
On Jan 17, 2011, at 6:36 PM, Tomas Vondra wrote: 1) Forks are 'per relation' but the distinct estimators are 'per column' (or 'per group of columns') so I'm not sure whether the file should contain all the estimators for the table, or if there should be one fork for each estimator. The

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/18/2011 11:42 AM, Dimitri Fontaine wrote: I've fixed the case by having the code remember the function's extension if any, and restore it along with the other dependencies. The only question here is should CREATE OR REPLACE be allowed. I just realized this could present a new problem. If

Re: [HACKERS] Replication logging

2011-01-18 Thread Fujii Masao
On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander mag...@hagander.net wrote: We should treat log_disconnections the same? We could keep it a boolean, but then only log disconnections for the cases that are mentioned in log_connections? It doesn't make sense to log disconnection for a

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen anssi.kaariai...@thl.fi writes: Is this supposed to be used mainly by contrib and PGXN extensions? When I saw the documentation, I immediately thought that this is a nice way to package my application's stored procedures. If this is not one of the intended usages, it should be

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Cédric Villemain
2011/1/18 Greg Smith g...@2ndquadrant.com: Bruce Momjian wrote: Should we be writing until 2:30 then sleep 30 seconds and fsync at 3:00? The idea of having a dead period doing no work at all between write phase and sync phase may have some merit.  I don't have enough test data yet on some

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Cédric Villemain
2011/1/18 Fujii Masao masao.fu...@gmail.com: On Mon, Jan 17, 2011 at 10:30 PM, Fujii Masao masao.fu...@gmail.com wrote: Though I haven't seen the core part of the patch (i.e., ReceiveTarFile, etc..) yet, here is the comments against others. Here are another comments: When I untar the tar

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/18/2011 12:11 PM, Anssi Kääriäinen wrote: The only question here is should CREATE OR REPLACE be allowed. I just realized this could present a new problem. If I am not mistaken, when loading from dump, you suddenly get the extension's version back, not the one you defined in CREATE OR

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen anssi.kaariai...@thl.fi writes: The only question here is should CREATE OR REPLACE be allowed. I just Yes. Think ALTER EXTENSION UPGRADE, the next patch in the series (already proposed for this CF too). realized this could present a new problem. If I am not mistaken, when

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen anssi.kaariai...@thl.fi writes: Ok, verified at least for CREATE OR REPLACE, ALTER FUNCTION RENAME and ALTER FUNCTION SET search_path. You will get the extensions version back when restoring from plain sql dump, not the CORed function, rename is lost and same for search_path.

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Greg Smith
Robert Haas wrote: Idea #4: For ext3 filesystems that like to dump the entire buffer cache instead of only the requested file, write a little daemon that runs alongside of (and completely indepdently of) PostgreSQL. Every 30 s, it opens a 1-byte file, changes the byte, fsyncs the file, and

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 03:14, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jan 17, 2011 at 10:50 PM, Magnus Hagander mag...@hagander.net wrote: +       printf(_(  -D, --pgdata=directory   receive base backup into directory\n)); +       printf(_(  -T, --tardir=directory    receive base

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 10:49, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jan 17, 2011 at 10:30 PM, Fujii Masao masao.fu...@gmail.com wrote: Though I haven't seen the core part of the patch (i.e., ReceiveTarFile, etc..) yet, here is the comments against others. Here are another

Re: [HACKERS] auto-sizing wal_buffers

2011-01-18 Thread Greg Smith
Fujii Masao wrote: +/* Minimum setting used for a lower bound on wal_buffers */ +#define XLOG_BUFFER_MIN4 Why didn't you use XLOG_BUFFER_MIN instead of XLOGbuffersMin? XLOG_BUFFER_MIN is not used anywhere for now. That's a typo; will fix. + if

Re: [HACKERS] auto-sizing wal_buffers

2011-01-18 Thread Greg Smith
Josh Berkus wrote: I think we can be more specific on that last sentence; is there even any *theoretical* benefit to settings above 16MB, the size of a WAL segment? Certainly there have been no test results to show any. There was the set Marti just reminded about. The old wording

Re: [HACKERS] Warning compiling pg_dump (MinGW, Windows XP)

2011-01-18 Thread Andrew Dunstan
On 01/18/2011 04:40 AM, Pavel Golub wrote: AD We could add -Wno-format to the flags. That makes it shut up, but maybe AD we don't want to use such a sledgehammer. I want to understand the only thing. Are these warnings really dangerous? Or I should just ignore them? As I pointed out

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/18/2011 01:03 PM, Dimitri Fontaine wrote: I'd appreciate a list of yet-to-fix items. What I have is the search_path issue where CREATE EXTENSION foo; can leave it changed for the current session, I intend to fix that later today. Other than that, I have no further already agreed on code

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Alvaro Herrera
Excerpts from Dimitri Fontaine's message of mar ene 18 07:01:55 -0300 2011: Anssi Kääriäinen anssi.kaariai...@thl.fi writes: It used to work this way with \i, obviously. Should the extension patch care about that and how? Do we want to RESET search_path or to manually manage it like we

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Mon, Jan 17, 2011 at 16:27, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-01-17 at 16:20 +0100, Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:18, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 17, 2011 at 8:55 AM, Magnus Hagander mag...@hagander.net wrote: Hmm. I

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 12:40, Magnus Hagander mag...@hagander.net wrote: On Tue, Jan 18, 2011 at 10:49, Fujii Masao masao.fu...@gmail.com wrote: Yeah that sounds like a good idea. Shouldn't be too hard to do (will reuqire a backend patch as well, of course). Should we use -f for fast? Though

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen anssi.kaariai...@thl.fi writes: On 01/18/2011 01:03 PM, Dimitri Fontaine wrote: I'd appreciate a list of yet-to-fix items. What I have is the search_path issue where CREATE EXTENSION foo; can leave it changed for the current session, I intend to fix that later today. After

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 08:40:50 -0300 2011: On Tue, Jan 18, 2011 at 10:49, Fujii Masao masao.fu...@gmail.com wrote: +                                       fprintf(stderr, _(%s: could not write to file '%s': %m\n), %m in fprintf is portable? Hmm. I just

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Alvaro Herrera
Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: Hello Postgres Hackers, In reference to this todo item about clustering system table indexes, ( http://archives.postgresql.org/pgsql-hackers/2004-05/msg00989.php ) I have been studying the system tables

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Alvaro Herrera
Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: Hello Postgres Hackers, BTW whatever you do, don't start a new thread by replying to an existing message and just changing the subject line. It will mess up the threading for some readers, and some might not even see your

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Merlin Moncure
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby j...@nasby.net wrote: On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: Robert Haas  wrote: a quick-and-dirty attempt to limit the amount of I/O caused by hint bits. I'm still very interested in knowing what people think about that. I found the

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread David Fetter
On Tue, Jan 18, 2011 at 09:14:41PM +1300, Mark Kirkwood wrote: On 18/01/11 18:04, Tom Lane wrote: David Fetterda...@fetter.org writes: Who's the copyright holder(s)? If it's all individual contributors, Red Hat policy is not in play. Sorry David, it was written on the company's dime.

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: Ok, thanks for clarifying. I've updated to use strerror(). Guess it's time for another patch, PFA :-) Thanks ... Message nitpick: + if (compresslevel 0) + { + fprintf(stderr, + _(%s: this build

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 15:49, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: Ok, thanks for clarifying. I've updated to use strerror(). Guess it's time for another patch, PFA :-) Thanks ...  Message nitpick: +  

Re: [HACKERS] review: FDW API

2011-01-18 Thread Shigeru HANADA
. Please find attached patches. == patch list == 1) 20110118-no_fdw_perm_check.patch - This patch is not included in last post. This had been proposed on 2011-01-05 first, but maybe has not been reviewd yet. I re-propose this patch for SQL standard conformance. This patch removes permission

Re: [HACKERS] SQL/MED - file_fdw

2011-01-18 Thread Shigeru HANADA
is not for EXPLAIN. I'll try to defer generating explainInfo until EXPLAIN VERBOSE really uses it. It might need new hook point in expalain.c, though. Regards, -- Shigeru Hanada 20110118-file_fdw.patch.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 10:56, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander mag...@hagander.net wrote: We should treat log_disconnections the same? We could keep it a boolean, but then only log disconnections for the cases that are mentioned in

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Tom Lane
David Fetter da...@fetter.org writes: I'm guessing there's a Policy® at Red Hat that software made on its dime be GPL (v2, I'd guess), and that getting an exception would involve convening its board or similarly drastic action. It's company policy, and while it *might* be possible to get an

Re: [HACKERS] Replication logging

2011-01-18 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Is there *any* usecase for setting them differently though? I can't believe we're still engaged in painting this bikeshed. Let's just control it off log_connections and have done. BTW, what about log_disconnections --- does a walsender emit a message

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Itagaki Takahiro itagaki.takah...@gmail.com writes: On Tue, Jan 18, 2011 at 05:39, Tom Lane t...@sss.pgh.pa.us wrote: I haven't looked at this patch, but it seems to me that it would be reasonable to conclude A != B if the va_extsize values in the toast pointers don't agree. It's a very

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Actually, after some IM chats, I think pg_streamrecv should be renamed, probably to pg_walstream (or pg_logstream, but pg_walstream is a lot more specific than that) pg_stream_log pg_stream_backup Those seem better. Tom, would those solve your

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: It's a very light-weight alternative of memcmp the byte data, but there is still the same issue -- we might have different compressed results if we use different algorithm for TOASTing. Which makes it a lightweight waste of

Re: [HACKERS] Replication logging

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I also find it weird that incoming replication connections are logged by default. In the standby, we also log streaming replication successfully connected to primary, which serves much of the same

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: Hello Postgres Hackers, In reference to this todo item about clustering system table indexes, (

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: No, I don't think so.  Has any evidence been submitted that that part of the patch is of benefit? I think you might be mixing up what's actually in the patch with another idea that

Re: [HACKERS] test_fsync open_sync test

2011-01-18 Thread Bruce Momjian
Greg Smith wrote: Bruce Momjian wrote: Is there a value to this test_fsync test? Compare open_sync with different sizes: (This is designed to compare the cost of one large sync'ed write and two smaller sync'ed writes.) open_sync 16k write 242.563

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 11:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: No, I don't think so.  Has any evidence been submitted that that part of the patch is of benefit? I think you

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 10:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Fetter da...@fetter.org writes: I'm guessing there's a PolicyŽ at Red Hat that software made on its dime be GPL (v2, I'd guess), and that getting an exception would involve convening its board or similarly drastic

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 11:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Oh, I misread Itagaki-san's comment to imply that that *was* in the patch.  Maybe I should go read it. Perhaps. :-) While you're at it you might commit it. :-) Yeah, as penance

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 17:31, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Actually, after some IM chats, I think pg_streamrecv should be renamed, probably to pg_walstream (or pg_logstream, but pg_walstream is a lot more specific than that) pg_stream_log

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Cédric Villemain
2011/1/18 Magnus Hagander mag...@hagander.net: On Tue, Jan 18, 2011 at 17:31, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Actually, after some IM chats, I think pg_streamrecv should be renamed, probably to pg_walstream (or pg_logstream, but pg_walstream is a

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 4:53 AM, t...@fuzzy.cz wrote: So the most important question is how to intercept the new/updated rows, and where to store them. I think each backend should maintain it's own private list of new records and forward them only in case of commit. Does that sound

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Cédric Villemain
2011/1/18 Robert Haas robertmh...@gmail.com: On Tue, Jan 18, 2011 at 10:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Fetter da...@fetter.org writes: I'm guessing there's a PolicyŽ at Red Hat that software made on its dime be GPL (v2, I'd guess), and that getting an exception would involve

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 8:24 AM, Merlin Moncure wrote: a few weeks back I hacked an experimental patch that removed the hint bit action completely. the results were very premature and/or incorrect, but my initial findings suggested that hint bits might not be worth the cost from performance

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby j...@nasby.net wrote: - Forks are very possibly a more efficient way to deal with TOAST than having separate tables. There's a fair amount of overhead we pay for the current setup. That seems like

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby j...@nasby.net wrote: On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby j...@nasby.net wrote: - Forks are very possibly a more efficient way to deal with TOAST than having separate tables. There's a fair

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby j...@nasby.net wrote: On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby j...@nasby.net wrote: - Forks are very possibly a more efficient way to deal with TOAST than

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:24 AM, Robert Haas wrote: On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby j...@nasby.net wrote: On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby j...@nasby.net wrote: - Forks are very possibly a more efficient way to deal with TOAST

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:32 AM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby j...@nasby.net wrote: On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby j...@nasby.net wrote: - Forks are very possibly

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby j...@nasby.net wrote: On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: Robert Haas  wrote: a quick-and-dirty attempt to limit the amount of I/O caused by hint bits. I'm still very interested in knowing what people think about that. I found the

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure mmonc...@gmail.com wrote: a few weeks back I hacked an experimental patch that removed the hint bit action completely.  the results were very premature and/or incorrect, but my initial findings suggested that hint bits might not be worth the cost

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:32 PM, Jim Nasby j...@nasby.net wrote: How's that different from what vacuum does on a TOAST table now? TOAST vacuum is currently an entirely separate vacuum. It might run at the same time as the main table vacuum, but it still has all the work that would be

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:03 PM, Magnus Hagander mag...@hagander.net wrote: So it'd be pg_receive_wal and pg_receive_base_backup then? Votes from others? (it's easy to rename so far, so I'll keep plugging away under the name pg_basebackup based on Fujii-sans comments until such a time as we

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
On Jan 18, 2011, at 6:35 AM, Alvaro Herrera wrote: Wow, this is really old stuff. I don't know if this is really of any benefit, given that these catalogs are loaded into syscaches anyway. The benefit is educational primarily. I was looking for a todo list item that would expose me to

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: Hello Postgres Hackers, In reference to this todo item about clustering system table indexes, (

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 11:53:55 -0300 2011: On Tue, Jan 18, 2011 at 15:49, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: Ok, thanks for clarifying. I've updated to use strerror().

Re: [HACKERS] Re: new patch of MERGE (merge_204) a question about duplicated ctid

2011-01-18 Thread Greg Smith
David Fetter wrote: I think I haven't communicated clearly what I'm suggesting, which is that we ship with both an UPSERT and a MERGE, the former being ugly, crude and simple, and the latter festooned with dire warnings about isolation levels and locking. I don't know that I completely

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think you may be confused about what the patch does - currently, pages with hint bit changes are considered dirty, period. Therefore, they are written whenever any other dirty page would be written: by the background writer cleaning scan, at

Re: [HACKERS] psql: Add \dL to show languages

2011-01-18 Thread Andreas Karlsson
Hi Josh, Nope, I do not have any better ideas than DO Blocks?. Everything looks good with the exception one bug now. \dL foo * QUERY ** SELECT l.lanname AS Name, pg_catalog.pg_get_userbyid(l.lanowner) as Owner, l.lanpltrusted AS Trusted FROM pg_catalog.pg_language

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Simon Riggs
On Tue, 2011-01-18 at 18:03 +0100, Magnus Hagander wrote: So it'd be pg_receive_wal and pg_receive_base_backup then? OK for me. Maybe even pg_receive_wal_stream Don't see any reason why command names can't be long. We have many function names already that long. -- Simon Riggs

Re: [HACKERS] [PERFORM] pgbench to the MAXINT

2011-01-18 Thread Greg Smith
Euler Taveira de Oliveira wrote: (i) If we want to support and scale factor greater than 21474 we have to convert some columns to bigint; it will change the test. From the portability point it is a pity but as we have never supported it I'm not too worried about it. Why? Because it will use

[HACKERS] Performance bug in DO blocks

2011-01-18 Thread Tom Lane
I just noticed that if you execute the same DO command over and over within a session, it gets slower and slower. And if you keep it up you'll notice the backend's RAM consumption bloating too. The cause appears to be that we leak the cached plans created for any SQL statements or expressions

Re: [HACKERS] test_fsync open_sync test

2011-01-18 Thread Bruce Momjian
Greg Smith wrote: Bruce Momjian wrote: Is there a value to this test_fsync test? Compare open_sync with different sizes: (This is designed to compare the cost of one large sync'ed write and two smaller sync'ed writes.) open_sync 16k write 242.563

[HACKERS] PgEast: 2011, 2nd call for papers

2011-01-18 Thread Joshua D. Drake
Hey folks, PgEast is being held in NYC this year from 03/22-03-25. Get your papers in, the deadline is soon! http://www.postgresqlconference.org/ Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training,

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 17:33, Robert Haas robertmh...@gmail.com wrote: On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I also find it weird that incoming replication connections are logged by default. In the standby, we also log streaming

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Josh Berkus
To be frank, I really don't care about fixing this behavior on ext3, especially in the context of that sort of hack. That filesystem is not the future, it's not possible to ever really make it work right, and every minute spent on pandering to its limitations would be better spent elsewhere

Re: [HACKERS] texteq/byteaeq: avoid detoast

2011-01-18 Thread Tom Lane
Noah Misch n...@leadboat.com writes: texteq, textne, byteaeq and byteane detoast their arguments, then check for equality of length. Unequal lengths imply the answer trivially; given equal lengths, the functions proceed to compare the actual bytes. We can skip detoasting entirely when the

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I think you may be confused about what the patch does - currently, pages with hint bit changes are considered dirty, period. Therefore, they are written whenever any other dirty page

Re: [HACKERS] Replication logging

2011-01-18 Thread Josh Berkus
All, Just speaking as someone who does a lot of log-crunching for performance review, I don't find having a separate log_connections option for replication terribly useful. It's easy enough for me to just log all connections and filter for the type of connections I want. Even in cases where

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so your questions will be answered.

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Heikki Linnakangas
On 18.01.2011 21:16, Robert Haas wrote: On Tue, Jan 18, 2011 at 1:32 PM, Tom Lanet...@sss.pgh.pa.us wrote: While I was trying to performance-test the texteq patch, it occurred to me that this proposed hint-bit change has got a serious drawback. To wit, that it will totally destroy

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: There's a few remaining TODO comments in the code, which obviously need to be resolved one way or another Not all of these are must haves for 9.1. Here's how they break down: The two in predicate_internals.h mark places which

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:00 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 18.01.2011 21:16, Robert Haas wrote: On Tue, Jan 18, 2011 at 1:32 PM, Tom Lanet...@sss.pgh.pa.us  wrote: While I was trying to performance-test the texteq patch, it occurred to me that this

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:18 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: That's all of them. Our existing code has plenty of TODOs in it already, so I see no problem with continuing to comment places where future enhancements are possible, as long as they don't reflect deficiencies

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: If my back-of-the-envelope math is right, a carefully constructed pessimal load could need up to (max_connections / 2)^2 -- so 100 connections could conceivably require 2500 structures, although such a scenario would be hard to create.

[HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
I have modified test_fsync to use test labels that match wal_sync_method values, and and added more tests for open_sync with different sizes. This should make the program easier for novices to understand. Here is a test run for Ubuntu 11.04: $ ./test_fsync 2000 operations per

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Bruce Momjian
Robert Haas wrote: On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: Hello Postgres Hackers, In reference to this todo item about clustering system table indexes, (

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes: One of the reasons our client wants GIN for the integer[] column so bad is because recreating the GiST integer[] index is quite painful. Before I duped the table, I was just dropping and recreating the index on the original table. It was great

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane t...@sss.pgh.pa.us: Noah Misch n...@leadboat.com writes: On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Mark Kirkwood
On 19/01/11 05:51, Robert Haas wrote: I'm not sure why they'd care, but it certainly doesn't seem worth spending the amount of time arguing about it that we are. David and Mark are, of course, free to spend their time petitioning Red Hat for relicensing if they are so inclined, but they aren't

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:16 PM, Simone Aiken sai...@ulfheim.net wrote: When I'm learning a new system I like to first learn how to use it, second learn its data model, third start seriously looking at the code. So that Todo is ideal for my learning method. Sure - my point is just that we

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread A.M.
On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: I have modified test_fsync to use test labels that match wal_sync_method values, and and added more tests for open_sync with different sizes. This should make the program easier for novices to understand. Here is a test run for Ubuntu

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes: These numbers are a bit crazy-making, but the upshot is that Gist is slow out of the gate, but with data cached, it's pretty speedy. With indexscan and bitmapscan disabled, these queries all took 300-400 ms. So GIN was never better performing than

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2011/1/18 Tom Lane t...@sss.pgh.pa.us: I looked at this patch and found it fairly awkward.  What is the point of adding an additional flag to every variable, as opposed to just forcibly detoasting during assignment? But detoasting on assignment

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread David E. Wheeler
On Jan 18, 2011, at 1:39 PM, Tom Lane wrote: At the moment my opinion is that gist__int_ops is too broken to be usable, and it's also too uncommented to be fixable by anyone other than the original author. That seems to jibe with your comments from last year:

  1   2   >