Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Amit kapila
On Tuesday, November 13, 2012 3:11 AM Simon Riggs wrote: On 12 November 2012 21:26, Merlin Moncure wrote: >> I couldn't disagree more. The patch is small, logical, and fixes an >> awful problem, namely that --single mode is basically unusable. As to >> your wider point (namely, that you can't c

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Amit kapila
On Monday, November 12, 2012 8:31 PM Merlin Moncure wrote: On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila wrote: > On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: > On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: > On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: > On Frid

[HACKERS] Memory leaks in record_out and record_send

2012-11-12 Thread Tom Lane
I looked into the problem complained of here: http://archives.postgresql.org/pgsql-general/2012-11/msg00279.php which turns out to have nothing to do with joins and everything to do with the fact that record_out() leaks memory like mad. It leaks both the strings returned by the per-column output f

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-12 Thread Amit kapila
On Monday, November 12, 2012 9:56 PM Alvaro Herrera wrote: Robert Haas escribió: > On Tue, Jul 31, 2012 at 8:09 AM, Amit kapila wrote: >> >> I think I can see all of those things being potentially useful. There >> >> are a couple of pending patches that will revise the WAL format >> >> slightly;

[HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-12 Thread Amit kapila
On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila wrote: > On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: >> On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila >> wrote: >> > On Thursday, November 08, 2012 2:04 PM Heikki Linnakangas wrote: >>

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-12 Thread Amit kapila
On Monday, November 12, 2012 12:07 PM Greg Smith wrote: On 11/9/12 11:59 PM, Amit kapila wrote: >> Please let me know if there are any objections or problems in above method >> of implementation, >> else I can go ahead to prepare the patch for the coming CF. > It may be the case that the locking

[HACKERS] support for LDAP URLs

2012-11-12 Thread Peter Eisentraut
Here is a patch to support RFC 2255 LDAP URLs in pg_hba.conf. So, instead of, say host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapsearchattribute=uid you could write host ... ldap lapurl="ldap://ldap.example.net/dc=example,dc=net?uid?sub"; Apache and probably oth

Re: [HACKERS] Doc patch to See Also: CREATE TABLE AS in CREATE TABLE docs

2012-11-12 Thread Peter Eisentraut
On Sun, 2012-09-23 at 21:28 -0500, Karl O. Pinc wrote: > Patch to add CREATE TABLE AS to the See Also: section > of the CREATE TABLE docs. committed -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pg

Re: [HACKERS] Index only scans wiki page

2012-11-12 Thread Peter Geoghegan
On 13 November 2012 01:03, Jeff Janes wrote: > https://wiki.postgresql.org/wiki/Index-only_scans > > This page is seriously out of date. It suggests they are not yet > implemented, but only being talked about. Attached is a piece I wrote on the feature. That might form the basis of a new wiki pa

[HACKERS] Index only scans wiki page

2012-11-12 Thread Jeff Janes
https://wiki.postgresql.org/wiki/Index-only_scans This page is seriously out of date. It suggests they are not yet implemented, but only being talked about. Would someone who knows a lot about the subject (which is why I'm sending this hackers, not web) like to take a whack at updating this? O

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Josh Berkus
Jeff, > OK, so here's my proposal for a first patch (changes from Simon's > patch): > > * Add a flag to the postgres executable indicating that it should use > checksums on everything. This would only be valid if bootstrap mode is > also specified. > * Add a multi-state checksums flag in pg_c

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Greg Smith
On 11/12/12 3:44 AM, Markus Wanner wrote: Sorry if that has been discussed before, but can't we do without that bit at all? It adds a checksum switch to each page, where we just agreed we don't event want a per-database switch. Once you accept that eventually there need to be online conversion

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Jeff Davis
On Mon, 2012-11-12 at 20:44 +0100, Markus Wanner wrote: > As described before in this thread, I think we might be able to do > without the "has checksum"-bit, as yet another simplification. But I > don't object to adding it, either. I see. For a first patch, I guess that's OK. Might as well make i

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Greg Smith
On 11/12/12 4:44 AM, Craig Ringer wrote: Is it absurd to suggest using another bitmap, like the FSM or visibility map, to store information on page checksumming while checksumming is enabled but incomplete? I spent some time thinking about that last week. One problem with it is that the bitm

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Robert Haas
On Mon, Nov 12, 2012 at 5:17 PM, Simon Riggs wrote: > On 12 November 2012 16:51, Robert Haas wrote: > >> Although there may be some workloads that access very large numbers of >> tables repeatedly, I bet that's not typical. > > Transactions with large numbers of DDL statements are typical at > up

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Simon Riggs
On 12 November 2012 16:51, Robert Haas wrote: > Although there may be some workloads that access very large numbers of > tables repeatedly, I bet that's not typical. Transactions with large numbers of DDL statements are typical at upgrade (application or database release level) and the execution

[HACKERS] vacuumlo - use a cursor

2012-11-12 Thread Andrew Dunstan
vacuumlo is rather simpleminded about dealing with the list of LOs to be removed - it just fetches them as a straight resultset. For one of my our this resulted in an out of memory condition. The attached patch tries to remedy that by using a cursor instead. If this is wanted I will add it to t

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Jeff Janes
On Fri, Nov 9, 2012 at 10:50 AM, Jeff Janes wrote: > On Thu, Nov 8, 2012 at 9:50 PM, Tom Lane wrote: >> Jeff Janes writes: >>> Are sure the server you are dumping out of is head? >> >> I experimented a bit with dumping/restoring 16000 tables matching >> Bruce's test case (ie, one serial column a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Simon Riggs
On 12 November 2012 21:26, Merlin Moncure wrote: > I couldn't disagree more. The patch is small, logical, and fixes an > awful problem, namely that --single mode is basically unusable. As to > your wider point (namely, that you can't connect to it, therefore it's > bad), it has got to be refute

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Mon, Nov 12, 2012 at 12:09:08PM -0500, Bruce Momjian wrote: > The second approach would be to simply try to copy the fsm, vm, and > extent files, and ignore any ENOEXIST errors. This allows code > simplification. The downside is that it doesn't pull all files with > matching prefixes --- it re

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Mon, Nov 12, 2012 at 06:14:59PM -0300, Alvaro Herrera wrote: > Bruce Momjian escribió: > > > --- 17,24 > > > > static void transfer_single_new_db(pageCnvCtx *pageConverter, > >FileNameMap *maps, int size); > > ! static int transfer_relfile(pageC

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Mon, Nov 12, 2012 at 1:21 PM, Simon Riggs wrote: > On 10 September 2012 17:50, Tom Lane wrote: > >> The point of the proposal that I am making is to have a simple, >> low-maintenance solution for people who need a single-application >> database. A compromise somewhere in the middle isn't like

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Alvaro Herrera
Bruce Momjian escribió: > It is possible that the poor 16k pg_upgrade value is caused by the poor > 9.2 binary-upgrade number (189.38). Perhaps I need to hack up > pg_upgrade to allow a 9.3 to 9.3 upgrade to test this. Hmm? This already works, since "make check" uses it, right? -- -- Sent

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Alvaro Herrera
Bruce Momjian escribió: > --- 17,24 > > static void transfer_single_new_db(pageCnvCtx *pageConverter, > FileNameMap *maps, int size); > ! static int transfer_relfile(pageCnvCtx *pageConverter, FileNameMap *map, > !

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Mon, Nov 12, 2012 at 03:59:27PM -0500, Bruce Momjian wrote: > > The second approach would be to simply try to copy the fsm, vm, and > > extent files, and ignore any ENOEXIST errors. This allows code > > simplification. The downside is that it doesn't pull all files with > > matching prefixes -

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Mon, Nov 12, 2012 at 12:09:08PM -0500, Bruce Momjian wrote: > OK, I have had some time to think about this. What the current code > does is, for each database, get a directory listing to know about any > vm, fsm, and >1gig extents that exist in the directory. It caches the > directory listing

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Tom Lane
Here's an updated patch that fixes the GIST replay functions as well as the other minor issues that were mentioned. Barring objections, I'll set about back-patching this as far as 9.0. One thing that could use verification is my fix for gistRedoPageSplitRecord. AFAICS, the first page listed in t

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Tom Lane
Simon Riggs writes: > On 11 November 2012 23:24, Tom Lane wrote: >> Practically all WAL record types that touch multiple pages have some >> bug of this type. In addition to btree_xlog_split, I found that >> heap_xlog_update, ginRedoDeletePage, spgRedoAddLeaf, spgRedoMoveLeafs, >> spgRedoAddNode,

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
Jeff, On 11/12/2012 06:52 PM, Jeff Davis wrote: > OK, so here's my proposal for a first patch (changes from Simon's > patch): > > * Add a flag to the postgres executable indicating that it should use > checksums on everything. This would only be valid if bootstrap mode is > also specified. >

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Simon Riggs
On 12 November 2012 14:51, Tom Lane wrote: >> The 9.0 code is broken, however. In >> 9.0, when a child page is split, the parent and new children are kept >> locked until the downlinks are inserted/updated. If a concurrent scan >> comes along and sees that incomplete state, it will miss tuples on

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Simon Riggs
On 11 November 2012 23:24, Tom Lane wrote: > Practically all WAL record types that touch multiple pages have some > bug of this type. In addition to btree_xlog_split, I found that > heap_xlog_update, ginRedoDeletePage, spgRedoAddLeaf, spgRedoMoveLeafs, > spgRedoAddNode, spgRedoSplitTuple, and sp

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Simon Riggs
On 12 November 2012 16:22, Robert Haas wrote: > But I guess that raises the question - should COPY (FREEZE) silently > ignore the option for not-new relfilenodes, or should it error out? > Simon proposed the former, but I'm wondering if the latter would be > better. It's got some complex pre-con

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Simon Riggs
On 10 September 2012 17:50, Tom Lane wrote: > The point of the proposal that I am making is to have a simple, > low-maintenance solution for people who need a single-application > database. A compromise somewhere in the middle isn't likely to be an > improvement for anybody. For instance, if yo

Re: [HACKERS] Pg_upgrade speed for many tables

2012-11-12 Thread Jeff Davis
On Mon, 2012-11-12 at 10:29 -0800, Jeff Janes wrote: > When I'm doing a pg_upgrade with thousands of tables, the shutdown > checkpoint after restoring the dump to the new cluster takes a very > long time, as the writer drains its operation table by opening and > individually fsync-ing thousands of

Re: [HACKERS] Pg_upgrade speed for many tables

2012-11-12 Thread Jeff Janes
On Mon, Nov 5, 2012 at 12:08 PM, Bruce Momjian wrote: > Magnus reported that a customer with a million tables was finding > pg_upgrade slow. I had never considered many table to be a problem, but > decided to test it. I created a database with 2k tables like this: > > CREATE TABLE test19

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-12 Thread Alvaro Herrera
Noah Misch wrote: > On Sun, Nov 11, 2012 at 02:18:11AM -0300, Alvaro Herrera wrote: > > I will > > move all the open patches to CF3, unless someone beats me to it. I > > probably won't be able to get anything done tomorrow, so if somebody has > > a boring Sunday I would appreciate the help. > >

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Jeff Davis
On Sun, 2012-11-11 at 23:55 -0500, Greg Smith wrote: > Adding an initdb option to start out with everything checksummed seems > an uncontroversial good first thing to have available. OK, so here's my proposal for a first patch (changes from Simon's patch): * Add a flag to the postgres executab

Re: [HACKERS] Suggestion for --truncate-tables to pg_restore

2012-11-12 Thread Karl O. Pinc
Hi, Attached is version 4. Version 3 no longer built against head. On 10/16/2012 09:48:06 PM, Karl O. Pinc wrote: > > On 09/23/2012 08:52:07 PM, Karl O. Pinc wrote: > > > On 09/23/2012 12:24:27 AM, Karl O. Pinc wrote: > > > > On 09/23/2012 12:19:07 AM, Karl O. Pinc wrote: > > > > On 09/21/201

Re: [HACKERS] Doc patch: Document names of automatically created constraints and indexes

2012-11-12 Thread Karl O. Pinc
On 11/12/2012 10:40:00 AM, Tom Lane wrote: > "Karl O. Pinc" writes: > > Attached is a patch (against head) which documents the > > name of the constraint and index created when using > > PRIMARY KEY with CREATE TABLE, and the name of the > > index created when using UNIQUE. > Personally I don't

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Sat, Nov 10, 2012 at 12:41:44PM -0500, Bruce Momjian wrote: > On Sat, Nov 10, 2012 at 07:17:34PM +0200, Ants Aasma wrote: > > On Sat, Nov 10, 2012 at 7:10 PM, Bruce Momjian wrote: > > > I am confused why you see a loop. transfer_all_new_dbs() does a > > > merge-join of old/new database names,

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Robert Haas
On Fri, Nov 9, 2012 at 12:50 AM, Tom Lane wrote: > Jeff Janes writes: >> Are sure the server you are dumping out of is head? > > I experimented a bit with dumping/restoring 16000 tables matching > Bruce's test case (ie, one serial column apiece). The pg_dump profile > seems fairly flat, without

Re: [HACKERS] Doc patch: Document names of automatically created constraints and indexes

2012-11-12 Thread Tom Lane
"Karl O. Pinc" writes: > Attached is a patch (against head) which documents the > name of the constraint and index created when using > PRIMARY KEY with CREATE TABLE, and the name of the > index created when using UNIQUE. I haven't read all > the docs recently but I don't believe this is presentl

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Jeff Davis
On Mon, 2012-11-12 at 09:44 +0100, Markus Wanner wrote: > Can we simply write a progress indicator to pg_control or someplace > saying that all pages up to X of relation Y are supposed to have valid > checksums? pg_control would not be the right place for that structure. It's intended to be fixed-

Re: [HACKERS] Fix errcontext() function

2012-11-12 Thread Heikki Linnakangas
On 12.11.2012 17:52, Alvaro Herrera wrote: Hopefully you noticed that contrib is broken. Oops.. fixed. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-12 Thread Alvaro Herrera
Robert Haas escribió: > On Tue, Jul 31, 2012 at 8:09 AM, Amit kapila wrote: > >> I think I can see all of those things being potentially useful. There > >> are a couple of pending patches that will revise the WAL format > >> slightly; not sure how much those are likely to interfere with any > >>

[HACKERS] Doc patch: Document names of automatically created constraints and indexes

2012-11-12 Thread Karl O. Pinc
Hi, Attached is a patch (against head) which documents the name of the constraint and index created when using PRIMARY KEY with CREATE TABLE, and the name of the index created when using UNIQUE. I haven't read all the docs recently but I don't believe this is presently documented. It's unclear t

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Tom Lane
Bruce Momjian writes: > On Fri, Nov 9, 2012 at 12:50:34AM -0500, Tom Lane wrote: >> The hash_seq_search time is probably mostly associated with >> AtEOXact_RelationCache, which is run during transaction commit and scans >> the relcache hashtable looking for tables created in the current >> transa

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Robert Haas
On Mon, Nov 12, 2012 at 11:20 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Nov 9, 2012 at 3:31 PM, Simon Riggs wrote: >>> So what we're talking about here is a new mode for COPY, that when >>> requested will pre-freeze tuples when loading into a newly >>> created/truncated table. If the

Re: [HACKERS] Fwd: question on foreign key lock

2012-11-12 Thread Robert Haas
On Thu, Nov 8, 2012 at 3:45 AM, Filip Rembiałkowski wrote: > maybe this is a better group for this question? > > I can't see why creating foreign key on table A referencing table B, > generates an AccessExclusiveLock on B. > It seems (to a layman :-) ) that only writes to B should be blocked. > >

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Tom Lane
Robert Haas writes: > On Fri, Nov 9, 2012 at 3:31 PM, Simon Riggs wrote: >> So what we're talking about here is a new mode for COPY, that when >> requested will pre-freeze tuples when loading into a newly >> created/truncated table. If the table isn't newly created/truncated >> then we'll just ig

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-12 Thread Robert Haas
On Tue, Jul 31, 2012 at 8:09 AM, Amit kapila wrote: >>> Based on the discussion and suggestions in this mail chain, following >>> features can be implemented: >>> >>> 1. To compute the value of max LSN in data pages based on user input >>> whether he wants it for an individual > >>> file, a par

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Fri, Nov 9, 2012 at 12:50:34AM -0500, Tom Lane wrote: > Jeff Janes writes: > > Are sure the server you are dumping out of is head? > > I experimented a bit with dumping/restoring 16000 tables matching > Bruce's test case (ie, one serial column apiece). The pg_dump profile > seems fairly flat

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Robert Haas
On Fri, Nov 9, 2012 at 3:31 PM, Simon Riggs wrote: > So what we're talking about here is a new mode for COPY, that when > requested will pre-freeze tuples when loading into a newly > created/truncated table. If the table isn't newly created/truncated > then we'll just ignore it and continue. I see

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-12 Thread Bruce Momjian
On Sat, Nov 10, 2012 at 05:59:54PM -0500, Bruce Momjian wrote: > On Sat, Nov 10, 2012 at 02:45:54PM -0800, Jeff Janes wrote: > > On Sat, Nov 10, 2012 at 9:15 AM, Bruce Momjian wrote: > > > On Fri, Nov 9, 2012 at 04:23:40PM -0800, Jeff Janes wrote: > > >> On Fri, Nov 9, 2012 at 3:06 PM, Bruce Momj

[HACKERS] Dumping an Extension's Script

2012-11-12 Thread Dimitri Fontaine
Hi, Please find attached to this email an RFC patch implementing the basics of the pg_dump --extension-script option. After much discussion around the concept of an inline extension, we decided last year that a good first step would be pg_dump support for an extension's script. The approach I've

Re: [HACKERS] Fix errcontext() function

2012-11-12 Thread Alvaro Herrera
Heikki Linnakangas wrote: > On 10.11.2012 11:46, Chen Huajun wrote: > >>Unfortunately not all compilers support varargs macros. I bumped into > >>this in February, see > >>http://archives.postgresql.org/message-id/4f3b72e0.8040...@enterprisedb.com. > >>My last attempt to fix this was > >>at http://

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Andres Freund
On 2012-11-12 10:19:09 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-11-10 16:24:22 -0500, Tom Lane wrote: > >> If any of this stuff were getting used by external modules, changing it > >> would be problematic ... but fortunately, it isn't, because we lack > >> support for plug-in rmg

[HACKERS] Version 4.9 of PostgreSQL buildfarm client released

2012-11-12 Thread Andrew Dunstan
Version 4.9 of the PostgreSQL buildfarm client has been released and is available at Changes since version 4.8: . adjust git status checking to allow for messages from recent git releases . add a module to test FileTextA

Re: [HACKERS] Fix errcontext() function

2012-11-12 Thread Heikki Linnakangas
On 10.11.2012 11:46, Chen Huajun wrote: Unfortunately not all compilers support varargs macros. I bumped into this in February, see http://archives.postgresql.org/message-id/4f3b72e0.8040...@enterprisedb.com. My last attempt to fix this was at http://archives.postgresql.org/pgsql-hackers/2012-04/

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Tom Lane
Andres Freund writes: > On 2012-11-10 16:24:22 -0500, Tom Lane wrote: >> If any of this stuff were getting used by external modules, changing it >> would be problematic ... but fortunately, it isn't, because we lack >> support for plug-in rmgrs. So I'm not worried about back-patching the >> chang

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila wrote: > On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: > On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: > On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: > On Friday, September 07, 2012 11:19 PM Tom Lane wrote: > Heikki Li

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Alvaro Herrera
Greg Smith wrote: > On 11/11/12 2:56 PM, Jeff Davis wrote: > >We could have a separate utility, pg_checksums, that can > >alter the state and/or do an offline verification. And initdb would take > >an option that would start everything out fully protected with > >checksums. > > Adding an initdb op

Re: [HACKERS] [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-12 Thread Fujii Masao
On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila wrote: > On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: >> On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila >> wrote: >> > On Thursday, November 08, 2012 2:04 PM Heikki Linnakangas wrote: >> >> On 19.10.2012 14:42, Amit kapila wrote: >> >> > On T

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Tom Lane
Heikki Linnakangas writes: > On 12.11.2012 01:25, Tom Lane wrote: >> Worse than that, GIST WAL replay seems to be a total disaster from this >> standpoint, and I'm not convinced that it's even fixable without >> incompatible WAL changes. > Hmm. After the rewrite of that logic I did in 9.1 (commit

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Tom Lane
Robert Haas writes: > On Sun, Nov 11, 2012 at 6:24 PM, Tom Lane wrote: >> I already pointed out the inconsistency in heap_xlog_freeze about >> whether a cleanup lock is needed. As is, this patch uses a cleanup >> lock, but I suspect that a regular lock is sufficient --- comments? > Well, accord

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Robert Haas
On Sun, Nov 11, 2012 at 6:24 PM, Tom Lane wrote: > I already pointed out the inconsistency in heap_xlog_freeze about > whether a cleanup lock is needed. As is, this patch uses a cleanup > lock, but I suspect that a regular lock is sufficient --- comments? Well, according to storage/buffer/README

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-12 Thread Tom Lane
Greg Smith writes: > Regardless, exactly how to prevent two concurrent processes from writing > the same file feels like the last step in the small roadmap for what > this feature needs. "Write a temp file and use rename(2) to move it into place" is the standard solution for that.

Re: [HACKERS] why can't plpgsql return a row-expression?

2012-11-12 Thread Asif Rehman
Hi, I have tried to solve this issue. Please see the attached patch. With this patch, any expression is allowed in the return statement. For any invalid expression an error is generated without doing any special handling. When a row expression is used in the return statement then the resultant tu

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
On 11/12/2012 10:44 AM, Craig Ringer wrote: > That'll make it hard for VACUUM, hint-bit setting, etc to > opportunistically checksum pages whenever they're doing a page write anyway. It *is* a hard problem, yes. And the single bit doesn't really solve it. So I'm arguing against opportunistically c

Re: [HACKERS] Identity projection

2012-11-12 Thread Kyotaro HORIGUCHI
Hello, This is new version of identity projection patch. Reverted projectionInfo and ExecBuildProjectionInfo. Identity projection is recognized directly in ExecGroup, ExecResult, and ExecWindowAgg. nodeAgg is reverted because I couldn't make it sane.. The following is the result of performance te

Re: [HACKERS] Reduce palloc's in numeric operations.

2012-11-12 Thread Kyotaro HORIGUCHI
Thanks for comments, > Have to be careful to really not modify the > operands. numeric_out() and numeric_out_sci() are wrong; they > call get_str_from_var(), which modifies the argument. Same with > numeric_expr(): it passes the argument to > numericvar_to_double_no_overflow(), which passes it to

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Craig Ringer
On 11/12/2012 04:44 PM, Markus Wanner wrote: > Jeff, > > On 11/10/2012 12:08 AM, Jeff Davis wrote: >> The bit indicating that a checksum is present may be lost due to >> corruption. > Hm.. I see. > > Sorry if that has been discussed before, but can't we do without that > bit at all? It adds a check

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
On 11/12/2012 05:55 AM, Greg Smith wrote: > Adding an initdb option to start out with everything checksummed seems > an uncontroversial good first thing to have available. +1 So the following discussion really is for a future patch extending on that initial checkpoint support. > One of the reall

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Markus Wanner
Jeff, On 11/10/2012 12:08 AM, Jeff Davis wrote: > The bit indicating that a checksum is present may be lost due to > corruption. Hm.. I see. Sorry if that has been discussed before, but can't we do without that bit at all? It adds a checksum switch to each page, where we just agreed we don't eve

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-12 Thread Heikki Linnakangas
On 12.11.2012 01:25, Tom Lane wrote: Worse than that, GIST WAL replay seems to be a total disaster from this standpoint, and I'm not convinced that it's even fixable without incompatible WAL changes. There are several cases where index insertion operations generate more than one WAL record while