Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Heikki Linnakangas
On 20.01.2011 04:36, Robert Haas wrote: ... Even better, the code changes would be confined to ANALYZE rather than spread out all over the system, which has positive implications for robustness and likelihood of commit. Keep in mind that the administrator can already override the ndistinct

Re: [HACKERS] Extending opfamilies for GIN indexes

2011-01-20 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Actually the other way around. An opclass is the subset of an opfamily that is tightly bound to an index. The build methods have to be associatable with an index, so they're part of the index's opclass. The query methods could be loose in the opfamily. I

Re: [HACKERS] WIP: RangeTypes

2011-01-20 Thread Jeff Davis
New patch. I added a lot of generic range functions, and a lot of operators. There is still more work to do, this is just an updated patch. The latest can be seen on the git repository, as well: http://git.postgresql.org/gitweb?p=users/jdavis/postgres.git;a=log;h=refs/heads/rangetypes Regards,

Re: [HACKERS] REVIEW: WIP: plpgsql - foreach in

2011-01-20 Thread Pavel Stehule
2011/1/20 Stephen Frost sfr...@snowman.net: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 19, 2011 at 6:04 PM, Stephen Frost sfr...@snowman.net wrote: I'm going to mark this returned to author with feedback. That implies you don't think it should be considered further for this

[HACKERS] REPLICATION privilege and shutdown

2011-01-20 Thread Fujii Masao
Hi, Both the user with REPLICATION privilege and the superuser can call pg_stop_backup. But only superuser can connect to the server to cancel online backup during shutdown. The non-superuser with REPLICATION privilege cannot. Is this behavior intentional? Or just oversight? Regards, -- Fujii

Re: [HACKERS] REPLICATION privilege and shutdown

2011-01-20 Thread Heikki Linnakangas
On 20.01.2011 11:41, Fujii Masao wrote: Hi, Both the user with REPLICATION privilege and the superuser can call pg_stop_backup. But only superuser can connect to the server to cancel online backup during shutdown. The non-superuser with REPLICATION privilege cannot. Is this behavior

Re: [HACKERS] REPLICATION privilege and shutdown

2011-01-20 Thread Magnus Hagander
On Thu, Jan 20, 2011 at 10:50, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.01.2011 11:41, Fujii Masao wrote: Hi, Both the user with REPLICATION privilege and the superuser can call pg_stop_backup. But only superuser can connect to the server to cancel online backup

Re: [HACKERS] REPLICATION privilege and shutdown

2011-01-20 Thread Fujii Masao
On Thu, Jan 20, 2011 at 6:50 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I think we need to consider the situation after the multiple streaming base backups patch goes in. After that we can change pg_stop_backup() so that you need superuser privileges to run it again -

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Simon Riggs
On Wed, 2011-01-19 at 17:46 -0500, Noah Misch wrote: First, I'd like to note that the thread for this patch had *four* me-too responses to the use case. That's extremely unusual; the subject is definitely compelling to people. It addresses the bad behavior of natural attempts to

Re: [HACKERS] Transaction-scope advisory locks

2011-01-20 Thread Marko Tiikkaja
On 2011-01-17 9:28 AM +0200, Itagaki Takahiro wrote: Here is a short review for Transaction scoped advisory locks: https://commitfest.postgresql.org/action/patch_view?id=518 Thanks for reviewing! == Features == The patch adds pg_[try_]advisory_xact_lock[_shared] functions. The function names

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-01-20 Thread Heikki Linnakangas
On 19.01.2011 16:01, Joachim Wieland wrote: On Wed, Jan 19, 2011 at 7:47 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Here are the latest patches all of them also rebased to current HEAD. Will update the commitfest app as well. What's the idea of storing the file sizes

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Magnus Hagander
On Thu, Jan 20, 2011 at 05:23, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Jan 19, 2011 at 9:37 PM, Magnus Hagander mag...@hagander.net wrote: Great. Thanks for the quick update! Here are another comments: Here are comments against the documents. The other code looks good. Thanks!

Re: [HACKERS] Include WAL in base backup

2011-01-20 Thread Magnus Hagander
On Thu, Jan 20, 2011 at 05:03, Stephen Frost sfr...@snowman.net wrote: Greetings, * Magnus Hagander (mag...@hagander.net) wrote: For now, you need to set wal_keep_segments to make it work properly, but if you do the idea is that the tar file/stream generated in the base backup will include

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Magnus Hagander
On Thu, Jan 20, 2011 at 12:42, Magnus Hagander mag...@hagander.net wrote: On Thu, Jan 20, 2011 at 05:23, Fujii Masao masao.fu...@gmail.com wrote: It's helpful to document what to set to allow pg_basebackup connection. That is not only the REPLICATION privilege but also max_wal_senders and

Re: [HACKERS] bug in SignalSomeChildren

2011-01-20 Thread Bernd Helmle
--On 22. Dezember 2010 15:51:09 +0900 Fujii Masao masao.fu...@gmail.com wrote: How about doing target != ALL test at the head for the most common case (target == ALL)? That's an idea, but the test you propose implements it incorrectly. Thanks! I revised the patch. I had a look at this

Re: [HACKERS] SQL/MED - file_fdw

2011-01-20 Thread Itagaki Takahiro
On Wed, Jan 19, 2011 at 00:34, Shigeru HANADA han...@metrosystems.co.jp wrote: Attached patch requires FDW API patches and copy_export-20110114.patch. Some minor comments: * Can you pass slot-tts_values and tts_isnull directly to NextCopyFrom()? It won't allocate the arrays; just fill the array

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-01-20 Thread Joachim Wieland
On Thu, Jan 20, 2011 at 6:07 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It's part of the overall idea to make sure files are not inadvertently exchanged between different backups and that a file is not truncated. In the future I'd also like to add a checksum to the TOC so

Re: [HACKERS] [GENERAL] streaming replication feature request

2011-01-20 Thread Bruce Momjian
Scott Ribe wrote: How about supporting something like: wal_keep_segments = '7d' [ moved to hackers] Sorry for the late reply. That is a very interesting idea. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com

Re: [HACKERS] Moving test_fsync to /contrib?

2011-01-20 Thread Bruce Momjian
Greg Smith wrote: Alvaro Herrera wrote: I don't understand why it would be overkill. Are you saying people would complain because you installed a 25 kB executable that they might not want to use? Just for fun I checked /usr/bin and noticed that I have a pandoc executable, weighing 17

Re: [HACKERS] Moving test_fsync to /contrib?

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 9:13 AM, Bruce Momjian br...@momjian.us wrote: OK, I am ready to move test_fsync to /contrib.  Is pg_test_fsync the best name?  pg_check_fsync?  pg_fsync_performance?  pg_verify_fsync? I don't see too much reason to rename it more than necessary, so how about

Re: [HACKERS] ALTER TYPE 1: recheck index-based constraints

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 12:57 AM, Noah Misch n...@leadboat.com wrote: There are two distinct questions here: Agreed. (1) Should reindex_relation receive boolean facts from its callers by way of two boolean arguments, or by way of one flags vector? The former seems best when you want every

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-20 Thread Robert Haas
On Wed, Jan 19, 2011 at 4:27 PM, Simone Aiken sai...@ulfheim.net wrote: In my experience size increases related to documentation are almost always worth it.  So I'm prejudiced right out of the gate.  I was wondering if every pg_ table gets copied out to every database ..  if there is already a

Re: [HACKERS] sepgsql contrib module

2011-01-20 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue ene 20 00:10:55 -0300 2011: You have to write it with a line of dashes on the first and last lines, if you don't want it reformatted as a paragraph. It might be worth actually running pgindent over contrib/selinux and then check over the results.

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Bruce Momjian
Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:27, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-01-17 at 16:20 +0100, Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:18, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 17, 2011 at 8:55 AM, Magnus Hagander

Re: [HACKERS] sepgsql contrib module

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 9:59 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of jue ene 20 00:10:55 -0300 2011: You have to write it with a line of dashes on the first and last lines, if you don't want it reformatted as a paragraph.  It might be worth

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 10:01 AM, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:27, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-01-17 at 16:20 +0100, Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:18, Robert Haas robertmh...@gmail.com

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Csaba Nagy
Hi Tomas, On Wed, 2011-01-19 at 23:13 +0100, Tomas Vondra wrote: No, the multi-column statistics do not require constant updating. There are cases where a sampling is perfectly fine, although you may need a bit larger sample. Generally if you can use a multi-dimensional histogram, you don't

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Bruce Momjian
Robert Haas wrote: On Thu, Jan 20, 2011 at 10:01 AM, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:27, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-01-17 at 16:20 +0100, Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:18, Robert Haas

Re: [HACKERS] sepgsql contrib module

2011-01-20 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Hmm, I don't think pgindent is run regularly on contrib as it is on the core code. News to me. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Is there a way to build PostgreSQL client libraries with MinGW

2011-01-20 Thread XiaoboGu
Hi, We are using R to work with 64bit PostgreSQL client libraries, and to avoid compiler compatibility issues the R development community suggest using the same compiler for both the main application and dlls. So do you have any experience to build libpq.dll using MinGW 64 bit. Thanks.

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 10:15 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Jan 20, 2011 at 10:01 AM, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: On Mon, Jan 17, 2011 at 16:27, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-01-17 at 16:20

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-01-20 Thread Heikki Linnakangas
On 20.01.2011 15:46, Joachim Wieland wrote: On Thu, Jan 20, 2011 at 6:07 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The header is there to identify a file, it contains the header that every other pgdump file contains, including the internal version number and the unique

Re: [HACKERS] Moving test_fsync to /contrib?

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 9:13 AM, Bruce Momjian br...@momjian.us wrote: OK, I am ready to move test_fsync to /contrib.  Is pg_test_fsync the best name?  pg_check_fsync?  pg_fsync_performance?  pg_verify_fsync? I don't see too much reason to rename it

Re: [HACKERS] Include WAL in base backup

2011-01-20 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: - Why not initialize logid and logseg like so?:        int logid = startptr.xlogid;        int logseg = startptr.xrecoff / XLogSegSize;  Then use those in your elog?  Seems cleaner to me. Hmm. Yes. Agreed. Marginal complaint here: int is the

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-01-20 Thread Florian Pflug
On Jan20, 2011, at 16:22 , Heikki Linnakangas wrote: You can put files in the archive in a certain order if you list them explicitly in the tar command line, like tar cf backup.tar toc.dat It's hard to know the right order, though. In practice you would need to do tar tf backup.tar

Re: [HACKERS] Transaction-scope advisory locks

2011-01-20 Thread Tom Lane
Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes: Another thing I now see is this: BEGIN; SELECT pg_advisory_xact_lock(1); -- do something here -- upgrade to session lock SELECT pg_advisory_lock(1); COMMIT; This seems useful, since the xact lock would be automatically released if

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Bruce Momjian
Robert Haas wrote: I read the description as: + ? ?You can also use the xref linkend=app-pgbasebackup tool to take + ? ?the backup, instead of manually copying the files. This tool will take + ? ?care of the functionpg_start_backup()/, copy and + ? ?functionpg_stop_backup()/ steps

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: On Thu, Jan 20, 2011 at 10:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure why that's the right solution. Why do you think that we should not create the tablespace under the $PGDATA directory? I'm not surprised that people mounts the filesystem

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 19, 2011 at 10:16 PM, Stephen Frost sfr...@snowman.net wrote: This patch looked good, in general, to me.  I added a few documentation updates and a comment, but it's a very straight-forward patch as far as I can tell.  Passes all

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Simon Riggs
On Wed, 2011-01-19 at 22:16 -0500, Robert Haas wrote: That's another way of saying the patch is not anywhere close to being done. My patch is materially incomplete. Certainly we may see that as grounds for rejection, which I would not and could not argue with. It is a popular feature, so I

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Magnus Hagander
On Thu, Jan 20, 2011 at 16:45, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: I read the description as: + ? ?You can also use the xref linkend=app-pgbasebackup tool to take + ? ?the backup, instead of manually copying the files. This tool will take + ? ?care of the

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 11:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 19, 2011 at 10:16 PM, Stephen Frost sfr...@snowman.net wrote: This patch looked good, in general, to me.  I added a few documentation updates and a comment, but it's a

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: The main problem I've got with this patch is that there's no place to shoehorn the information into the textual EXPLAIN format without breaking a lot of expectations (and hence code --- it's insane to imagine that any significant amount of client-side

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Also, it won't actually work unless the server has replication configured (wal_level!=minimal, max_wal_senders0, and possibly some setting for wal_keep_segments), which has been the main point of the naming discussion thus far. Now, you know what

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 11:55 AM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: The main problem I've got with this patch is that there's no place to shoehorn the information into the textual EXPLAIN format without breaking a lot of expectations (and hence code

Re: [HACKERS] CommitFest wrap-up

2011-01-20 Thread Robert Haas
On Tue, Dec 21, 2010 at 10:49 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 21, 2010 at 11:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 15, 2010 at 11:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: - Writeable CTEs - I think we need Tom

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Marko Tiikkaja
On 2011-01-20 7:07 PM +0200, Robert Haas wrote: Looking at the patch, I have to say I had hoped this was going to show nfiltered in both the estimated and actual cases, which it doesn't. Now maybe that's more work than we want to put in, but it would be nice to have. That would be fantastical,

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 11:55 AM, Stephen Frost sfr...@snowman.net wrote: While I agree completely about the general if you're going to break, break it big approach, but I don't particularly care for holding output strings from EXPLAIN to the same

Re: [HACKERS] CommitFest wrap-up

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 21, 2010 at 10:49 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 21, 2010 at 11:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 15, 2010 at 11:29 AM, Tom Lane t...@sss.pgh.pa.us

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 11:59 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Also, it won't actually work unless the server has replication configured (wal_level!=minimal, max_wal_senders0, and possibly some setting for wal_keep_segments), which

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 12:57 PM, Magnus Hagander mag...@hagander.net wrote: On Jan 20, 2011 6:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 11:55 AM, Stephen Frost sfr...@snowman.net wrote: While I agree completely about the

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Magnus Hagander
On Jan 20, 2011 6:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 11:55 AM, Stephen Frost sfr...@snowman.net wrote: While I agree completely about the general if you're going to break, break it big approach, but I don't

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Noah Misch
On Thu, Jan 20, 2011 at 10:07:23AM +, Simon Riggs wrote: On Wed, 2011-01-19 at 17:46 -0500, Noah Misch wrote: First, I'd like to note that the thread for this patch had *four* me-too responses to the use case. That's extremely unusual; the subject is definitely compelling to

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Robert Haas robertmh...@gmail.com writes: I agree; we make bigger changes than this all the time. No, we don't. Alright, do we want to go down the road of adding new things to the XML/JSON/YAML/Whatever-else format that isn't displayed in the TEXT

Re: [HACKERS] Moving test_fsync to /contrib?

2011-01-20 Thread Josh Berkus
On 1/20/11 6:15 AM, Robert Haas wrote: On Thu, Jan 20, 2011 at 9:13 AM, Bruce Momjian br...@momjian.us wrote: OK, I am ready to move test_fsync to /contrib. Is pg_test_fsync the best name? pg_check_fsync? pg_fsync_performance? pg_verify_fsync? I don't see too much reason to rename it

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I think that the basic problem with wal_level is that to increase it you need to somehow ensure that all the backends have the new setting, and then checkpoint. Right now, the backends get the value through the GUC machinery, and so there's no

Re: [HACKERS] ALTER TYPE 1: recheck index-based constraints

2011-01-20 Thread Noah Misch
On Thu, Jan 20, 2011 at 09:26:29AM -0500, Robert Haas wrote: My main beef with the Boolean flags is that this kind of thing is not too clear: reindex_relation(myrel, false, false, true, true, false, true, false, false, true); Unless you have an excellent memory, you can't tell what

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: How much has that code been updated from one release to the next? Just an FYI, I talked to depesz on IRC (please chime in if you disagree with any of this) and he indicated that he's had to update the code from time to time, mostly because the parser

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 1:47 PM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Robert Haas robertmh...@gmail.com writes: I agree; we make bigger changes than this all the time. No, we don't. Alright, do we want to go down the road of adding new things to

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 2:10 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I think that the basic problem with wal_level is that to increase it you need to somehow ensure that all the backends have the new setting, and then checkpoint.  Right now,

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread hubert depesz lubaczewski
On Thu, Jan 20, 2011 at 02:48:59PM -0500, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: How much has that code been updated from one release to the next? Just an FYI, I talked to depesz on IRC (please chime in if you disagree with any of this) and he indicated that he's

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Sending them a signal seems like a promising approach, but the trick is guaranteeing that they've actually acted on it before you start the checkpoint. How much using a latch here would help? Or be overkill? -- Dimitri Fontaine http://2ndQuadrant.fr

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Heikki Linnakangas
On 20.01.2011 22:15, Dimitri Fontaine wrote: Robert Haasrobertmh...@gmail.com writes: Sending them a signal seems like a promising approach, but the trick is guaranteeing that they've actually acted on it before you start the checkpoint. How much using a latch here would help? Or be

Re: [HACKERS] exceptions not present in plpy with Python 3

2011-01-20 Thread Peter Eisentraut
On lör, 2010-12-18 at 18:56 +0100, Jan Urbański wrote: there seems to be a problem in the way we add exceptions to the plpy module in PL/Python compiled with Python 3k. Try this: DO $$ plpy.SPIError $$ language plpython3u; I'm not a Python 3 expert, but I nicked some code from the Internet

Re: [HACKERS] pl/python refactoring

2011-01-20 Thread Peter Eisentraut
On ons, 2011-01-19 at 10:06 +0900, Hitoshi Harada wrote: - This is not in the patch, but around line 184 vis versa in comment seems like typo. Fixed. - A line break should be added before PLy_add_exception() after static void I'll add that when I get to the patch. - This is also not in the

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
hubert depesz lubaczewski dep...@depesz.com writes: On Thu, Jan 20, 2011 at 02:48:59PM -0500, Stephen Frost wrote: He also mentioned that he didn't feel it was terribly complicated or that it'd be difficult to update for this. Looking over the code, it's got a simple regex for matching that

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 3:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: The main functional problem I see with this format is that it assumes there is one and only one filter step associated with every plan node. That is just plain wrong.  Many don't have any, and there are important cases where

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: I think filtered is pretty clear and like it... I find it ambiguous. [Takes sip of filtered water.] How about excluded? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

[HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
Hackers, One of our clients is seeing an unusual issue with statements which are waiting going into sleep forever and never completing. This first e-mail is for a has anyone else seen this problem before? while we try to collect additional information for a diagnosis. This problem only happens

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 2:10 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I think that the basic problem with wal_level is that to increase it you need to somehow ensure that all the backends have the

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of jue ene 20 18:05:15 -0300 2011: One of our clients is seeing an unusual issue with statements which are waiting going into sleep forever and never completing. This first e-mail is for a has anyone else seen this problem before? while we try to

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Tom Lane
Noah Misch n...@leadboat.com writes: Heikki's suggestion seemed straightforward, so much so that I couldn't figure why nobody had done it. That would usually mean I'm missing something. If you're willing to substitute an incompatible table, it's not clear why you don't just do

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 03:06, Nathan Boley napsal(a): And actually it does not depend on ndistinct for the columns only, it depends on ndistinct estimates for the combination of columns. So improving the ndistinct estimates for columns is just a necessary first step (and only if it works reasonably

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 3:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, is it just me, or is the terminology number filtered pretty confusing/ambiguous in itself?  It doesn't seem at all clear to me whether that's the number of rows passed by the

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
I have seen it -- on 8.1 too. On our case it was caused by an insert that was doing lots of toast insertions, so it needed to grab the extension lock frequently for the toast table; and this was slowed down by a largish shared_buffers setting, somehow (8.1 doesn't have lock partitioning, so

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Simon Riggs
On Thu, 2011-01-20 at 13:14 -0500, Noah Misch wrote: When DDL has taken AccessExclusiveLock and a query waits for it, it's the Right Thing for that query to wake up and proceed based on the complete, final state of that committed DDL. Aside from the waiting itself, the query should behave

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: One of our clients is seeing an unusual issue with statements which are waiting going into sleep forever and never completing. This first e-mail is for a has anyone else seen this problem before? while we try to collect additional information for a

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-20 Thread Simone Aiken
After playing with this in benchmarks and researching the weird results I got I'm going to advise dropping the todo for now unless something happens to change how postgres handles clustering. You guys probably already grokked this so I am just recording it for the list archives. The primary

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 03:36, Robert Haas napsal(a): On Wed, Jan 19, 2011 at 5:13 PM, Tomas Vondra t...@fuzzy.cz wrote: Regarding the crash scenario - if the commit fails, just throw away the local estimator copy, it's not needed. I'm not sure how to take care of the case when commit succeeds and the

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
I would take that to mean that it's waiting on the client. You mean that the client timed out and isn't accepting data from the query anymore? Shouldn't Postgres time out on that after a while? In one case, the orphaned statement was 16 hours old before we killed it. If it's relevant, the

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 09:10, Heikki Linnakangas napsal(a): It seems that the suggested multi-column selectivity estimator would be more sensitive to ndistinct of the individual columns. Is that correct? How is it biased? If we routinely under-estimate ndistinct of individual columns, for example, does

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.01.2011 03:05, Kevin Grittner wrote: If we don't do something like this, do we just provide REPEATABLE READ on the standby as the strictest level of transaction isolation? If so, do we generate an error on a request for

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 11:05, Csaba Nagy napsal(a): Hi Tomas, On Wed, 2011-01-19 at 23:13 +0100, Tomas Vondra wrote: No, the multi-column statistics do not require constant updating. There are cases where a sampling is perfectly fine, although you may need a bit larger sample. Generally if you can

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Josh Berkus
Kevin, So, based on a more complete description of the issues, any more opinions on whether to generate the error, as suggested by Heikki? If it's a choice between generating an error and letting users see inconsistent data, I'll take the former. Does anyone think this justifies the

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: I would take that to mean that it's waiting on the client. You mean that the client timed out and isn't accepting data from the query anymore? No, if the backend is in RECV state, it's waiting for the client to *send* it something. (Although if this is

[HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
Hello list, I wanted to test the waters on how receptive people might be to an extension that would allow Postgres to support two passwords for a given role. I have recently encountered a case where this would be highly useful when performing rolling password upgrades across many client

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Andrew Dunstan
On 01/20/2011 05:28 PM, Daniel Farina wrote: Hello list, I wanted to test the waters on how receptive people might be to an extension that would allow Postgres to support two passwords for a given role. I have recently encountered a case where this would be highly useful when performing

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 4:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Noah Misch n...@leadboat.com writes: Heikki's suggestion seemed straightforward, so much so that I couldn't figure why nobody had done it.  That would usually mean I'm missing something. If you're willing to substitute an

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
On Thu, Jan 20, 2011 at 2:45 PM, Andrew Dunstan and...@dunslane.net wrote: Have you thought of trying to use an external auth source like LDAP for such a scheme? I have thought about that, although LDAP is the only one that came to mind (I don't know a whole lot of systems in detail, only by

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Simon Riggs
On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: The idea is that whenever we see a valid snapshot which would yield a truly serializable view of the data for a READ ONLY transaction, we add a WAL record with that snapshot information. You haven't explained why this approach is the

Re: [HACKERS] [GENERAL] Large object corruption during 'piped' pg_restore

2011-01-20 Thread Tom Lane
Bosco Rama postg...@boscorama.com writes: If 'standard_conforming_strings = on' is set in our DB (which is required for our app) then the piped restore method (e.g. pg_restore -O backup.dat | psql) results in the large objects being corrupted. All servers and client tools involved are PG

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Noah Misch
On Thu, Jan 20, 2011 at 09:36:11PM +, Simon Riggs wrote: I agree that the DDL behaviour is wrong and should be fixed. Thank you for championing that alternative view. Swapping based upon names only works and is very flexible, much more so than EXCHANGE could be. A separate utility

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 20, 2011 at 4:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: If you're willing to substitute an incompatible table, it's not clear why you don't just do begin; drop table t; alter table t_new

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: The idea is that whenever we see a valid snapshot which would yield a truly serializable view of the data for a READ ONLY transaction, we add a WAL record with that snapshot information. You

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Marko Tiikkaja
On 1/20/2011 12:47 PM, Tom Lane wrote: So the line I'm thinking we should pursue is to visually associate the new counter with the filter condition, either like Filter Cond: (x 42) (nfiltered = 123) or Filter Cond: (x 42) Rows Filtered: 123 I'd prefer the latter.

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Florian Pflug
On Jan21, 2011, at 00:11 , Simon Riggs wrote: It's not clear to me what the reason is that this doesn't just work on HS already. If you started there it might help. The problem is that snapshots taken on the master sometimes represent a state of the database which cannot occur under any

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
I wrote: Why not? We already generate appropriate snapshots for this in SSI, so is the problem in getting the appropriate information into the WAL stream or in having a request for a snapshot within a serializable transaction while running in hot standby the problem? I dropped few words.

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
On 1/20/11 2:26 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: I would take that to mean that it's waiting on the client. You mean that the client timed out and isn't accepting data from the query anymore? No, if the backend is in RECV state, it's waiting for the client to

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Tom Lane
Daniel Farina drfar...@acm.org writes: I wanted to test the waters on how receptive people might be to an extension that would allow Postgres to support two passwords for a given role. Not very. Why don't you just put two roles in the same group? regards, tom lane --

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: The idea is that whenever we see a valid snapshot which would yield a truly serializable view of the data for a READ ONLY transaction, we add a WAL record with that snapshot information. You

  1   2   >