Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Heikki Linnakangas
On 12/03/2013 09:25 AM, Jeff Davis wrote: On Mon, 2013-12-02 at 11:07 +0200, Heikki Linnakangas wrote: There should be no difference between file-based extensions and catalog-based extensions. It's just two different ways to install the same extension. The extension author doesn't need to care

Re: [HACKERS] logical changeset generation v6.7

2013-12-03 Thread Kyotaro HORIGUCHI
Hello, This is rather trivial and superficial comments as not fully gripping functions of this patchset. - Some patches have line offset to master. Rebase needed. Other random comments follows, = 0001: - You assined HeapTupleGetOid(tuple) into relid to read in several points but no

Re: [HACKERS] In-core regression tests for replication, cascading, archiving, PITR, etc.

2013-12-03 Thread Michael Paquier
On Mon, Dec 2, 2013 at 11:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: At the same time, I'm pretty skeptical that any simple regression-test type facility would have caught the bugs we've fixed lately ... The replication bug would have been

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-03 Thread Sawada Masahiko
On Tue, Dec 3, 2013 at 4:28 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 3, 2013 at 3:30 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Tue, Dec 3, 2013 at 12:02 PM, Michael Paquier michael.paqu...@gmail.com wrote: Indeed, I forgot this code path. Completing for

[HACKERS] Expiring tuples from aggregation in window frames efficiently

2013-12-03 Thread David Rowley
Hi folks, I was casting my mind back to an idea that came up when windowing functions were first implemented in 8.4 during some talk about how ROWS BETWEEN might implemented in the future. This has now been implemented but with the implementation there is some quadratic behaviour when tuples move

Re: [HACKERS] Get more from indices.

2013-12-03 Thread Etsuro Fujita
I wrote: I've modified the patch to work in such a way. Also, as ISTM the patch is more complicated than what the patch really does, I've simplified the patch. I've revised the patch a bit. Please find attached the patch. Thanks, Best regards, Etsuro Fujita

Re: [HACKERS] logical changeset generation v6.7

2013-12-03 Thread Kyotaro HORIGUCHI
Hello, this is continued comments. = 0004: To be continued to next mail. = 0005: - In heapam.c, it seems to be better replacing t_self only during logical decoding. - In GetOldestXmin(), the parameter name 'alreadyLocked' would be better being simplly 'nolock' since

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 00:47:07 -0500, Noah Misch wrote: On Sat, Nov 30, 2013 at 01:06:09AM +, Alvaro Herrera wrote: Fix a couple of bugs in MultiXactId freezing Both heap_freeze_tuple() and heap_tuple_needs_freeze() neglected to look into a multixact to check the members against cutoff_xid.

[HACKERS] Skip hole in log_newpage

2013-12-03 Thread Heikki Linnakangas
The log_newpage function, used to WAL-log a full copy of a page, is missing the trick we normally use for full-page images to leave out the unused space on the block. That's pretty trivial to implement, so we should. The place where this matters the most is when building a new B-tree index.

Re: [HACKERS] Skip hole in log_newpage

2013-12-03 Thread Andres Freund
Hi, On 2013-12-03 13:03:41 +0200, Heikki Linnakangas wrote: The log_newpage function, used to WAL-log a full copy of a page, is missing the trick we normally use for full-page images to leave out the unused space on the block. That's pretty trivial to implement, so we should. The place

Re: [HACKERS] Skip hole in log_newpage

2013-12-03 Thread Heikki Linnakangas
On 12/03/2013 01:37 PM, Andres Freund wrote: I haven't looked thoroughly through all callsites, but shouldn't the vacuumlazy callsite use std = true? Well, it's logging an empty page, ie. a page full of zeros. I'm not sure if you'd consider that a standard page. Like the backup-block code in

Re: [HACKERS] Skip hole in log_newpage

2013-12-03 Thread Andres Freund
On 2013-12-03 13:57:04 +0200, Heikki Linnakangas wrote: On 12/03/2013 01:37 PM, Andres Freund wrote: I haven't looked thoroughly through all callsites, but shouldn't the vacuumlazy callsite use std = true? Well, it's logging an empty page, ie. a page full of zeros. I'm not sure if you'd

Re: [HACKERS] Skip hole in log_newpage

2013-12-03 Thread Heikki Linnakangas
On 12/03/2013 02:03 PM, Andres Freund wrote: On 2013-12-03 13:57:04 +0200, Heikki Linnakangas wrote: On 12/03/2013 01:37 PM, Andres Freund wrote: I haven't looked thoroughly through all callsites, but shouldn't the vacuumlazy callsite use std = true? Well, it's logging an empty page, ie. a

[HACKERS] [bug fix] pg_ctl stop times out when it should respond quickly

2013-12-03 Thread MauMau
Hello, I've encountered a small bug and fixed it. I guess this occurs on all major releases. I saw this happen on 9.2 and 9.4devel. Please find attached the patch and commit this. [Problem] If I mistakenly set an invalid value to listen_addresses, say '-1', and start the database

Re: [HACKERS] Skip hole in log_newpage

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 6:03 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: The log_newpage function, used to WAL-log a full copy of a page, is missing the trick we normally use for full-page images to leave out the unused space on the block. That's pretty trivial to implement, so we

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Robert Haas
On Mon, Dec 2, 2013 at 11:26 PM, Noah Misch n...@leadboat.com wrote: On Mon, Dec 02, 2013 at 08:56:03PM -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: ... I propose merely changing the syntax to TABLE FOR ROWS (...). Ugh :-(. Verbose and not exactly intuitive, I think. I don't

Re: [HACKERS] logical changeset generation v6.7

2013-12-03 Thread Andres Freund
On 2013-11-29 01:16:39 -0500, Robert Haas wrote: On Thu, Nov 14, 2013 at 8:46 AM, Andres Freund and...@2ndquadrant.com wrote: [ new patches ] Here's an updated version of patch #2. I didn't really like the approach you took in the documentation, so I revised it. Fair enough. Apart from

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 1:31 AM, Jeff Davis pg...@j-davis.com wrote: On Sun, 2013-12-01 at 15:48 +0100, Dimitri Fontaine wrote: Jeff Davis pg...@j-davis.com writes: I don't see why we are trying to accommodate a case where the author doesn't offer enough full SQL scripts and offers broken

Re: [HACKERS] logical changeset generation v6.7

2013-12-03 Thread Andres Freund
Hi, On 2013-11-28 21:15:18 -0500, Robert Haas wrote: OK, I've committed the patch to adjust the definition of IsSystemRelation()/IsSystemClass() and add IsCatalogRelation()/IsCatalogClass(). Thanks for taking care of this! I kibitzed your decision about which function to use in a few

[HACKERS] [bug fix or improvement?] Correctly place DLLs for ECPG apps in bin folder

2013-12-03 Thread MauMau
Hello, The attached patch implements the below proposal, and moves libecpg.dll, libecpg_compat.dll, and libpgtypes.dll from lib to bin folder on Windows, where they should be. http://www.postgresql.org/message-id/10470B394DB8486F93AC60107CC44C8B@maumau As Andrew-san said, I don't expect it

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Greg Stark
On Mon, Dec 2, 2013 at 7:46 PM, Robert Haas robertmh...@gmail.com wrote: Just tossing an idea out there. What if you could install an extension by specifying not a local file name but a URL. Obviously there's a security issue but for example we could allow only https URLs with verified domain

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: On 3 December 2013 02:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: ISTM that the real solution to this particular problem is to decouple the extensions that are currently in contrib from a specific postgres version. Problem? It's not a bug

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Can we separate this feature out? It's an issue with extensions today, and I'm eager to make some progress after the explosion of differing opinions today. +1 for separating that part out. I thought it was separated, at some point.

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Tom Dunstan (pg...@tomd.cc) wrote: Extensions in contrib live in a weird place. Totally builtin stuff should obviously be dumped without versions, and stuff which is completely separate and follows its own release schedule should obviously be versioned. I guess we consider all modules in

Re: [HACKERS] Trust intermediate CA for client certificates

2013-12-03 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: Uh, this thread actually started with Ian's feature request, and has changed to document the current behavior. Whoops, apologies for that then- I clearly came into it (or perhaps more accurately, was brought into it) after the start of the thread. I'm

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: Stephen mentioned using external tools and/or metadata, but to me that sounds like it would require porting the extension away from what's on PGXN today. Not at all- and that'd be the point. An external tool could take the PGXN extension, run 'make',

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Noah Misch
On Tue, Dec 03, 2013 at 11:56:07AM +0100, Andres Freund wrote: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: On Sat, Nov 30, 2013 at 01:06:09AM +, Alvaro Herrera wrote: Fix a couple of bugs in MultiXactId freezing Both heap_freeze_tuple() and heap_tuple_needs_freeze() neglected

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Greg Stark (st...@mit.edu) wrote: I thought the fundamental problem the in-catalog extensions were trying to solve were the issue with not having access to the filesystem. If that's the case then being able to say create extension from http://... would solve that. That's not really 'solved'

[HACKERS] Recovery bug in GIN, missing full page image

2013-12-03 Thread Heikki Linnakangas
While looking at Alexander's GIN patch, I noticed an ancient bug in the WAL-logging of GIN entry-tree insertions. entryPlaceToPage and dataPlacetoPage functions don't make a full-page image of the page, when inserting a downlink on a non-leaf page. The comment says: /* *

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Mon, Dec 02, 2013 at 08:56:03PM -0500, Tom Lane wrote: Ugh :-(. Verbose and not exactly intuitive, I think. I don't like any of the other options you listed much better. Still, the idea of using more than one word might get us out of the bind that a

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: The test spec additionally covers a (probably-related) assertion failure, new in 9.3.2. Too bad it's too late to do anthing about it for 9.3.2. :(. At least the last seems actually unrelated, I am

Re: [HACKERS] Skip hole in log_newpage

2013-12-03 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: The log_newpage function, used to WAL-log a full copy of a page, is missing the trick we normally use for full-page images to leave out the unused space on the block. That's pretty trivial to implement, so we should. Anyone see a problem

Re: [HACKERS] Proposed feature: Selective Foreign Keys

2013-12-03 Thread Robert Haas
On Mon, Dec 2, 2013 at 6:08 PM, Tom Dunstan pg...@tomd.cc wrote: On 3 Dec 2013, at 03:37, Robert Haas robertmh...@gmail.com wrote: I also like this feature. It would be really neat if a FOREIGN KEY constraint with a WHERE clause could use a *partial* index on the foreign table provided that

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 09:48:23 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: The test spec additionally covers a (probably-related) assertion failure, new in 9.3.2. Too bad it's too late to do anthing about it for 9.3.2. :(.

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: After sleeping on it, your other suggestion of TABLE OF, or possibly TABLE FROM, is starting to grow on me. Who else has an opinion? Alright, for my 2c, I like having this syntax include 'TABLE' simply because it's what folks coming from Oracle might be

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 8:44 AM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: On 3 December 2013 02:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: ISTM that the real solution to this particular problem is to decouple the extensions that are currently in

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 09:16:18 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 11:56:07AM +0100, Andres Freund wrote: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: On Sat, Nov 30, 2013 at 01:06:09AM +, Alvaro Herrera wrote: Any idea how to cheat our way out of that one given the current way

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 8:43 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: What Jeff is proposing is to simplify that down and have PostgreSQL auto discover the upgrade cycle when the version asked for isn't directly available with a creation script. We would keep the behavior depicted

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 09:16:18 -0500, Noah Misch wrote: The test spec additionally covers a (probably-related) assertion failure, new in 9.3.2. Too bad it's too late to do anthing about it for 9.3.2. :(. At least the last seems actually unrelated, I am not sure why it's 9.3.2 only. Alvaro,

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Noah Misch
On Tue, Dec 03, 2013 at 04:08:23PM +0100, Andres Freund wrote: On 2013-12-03 09:16:18 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 11:56:07AM +0100, Andres Freund wrote: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: On Sat, Nov 30, 2013 at 01:06:09AM +, Alvaro Herrera wrote:

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 10:29:54 -0500, Noah Misch wrote: Sorry, my original report was rather terse. I speak of the scenario exercised by the second permutation in that isolationtester spec. The multixact is later than VACUUM's cutoff_multi, so 9.3.1 does not freeze it at all. 9.3.2 does freeze it

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
Hi, On 2013-12-03 10:29:54 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 04:08:23PM +0100, Andres Freund wrote: On 2013-12-03 09:16:18 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 11:56:07AM +0100, Andres Freund wrote: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: On Sat,

[HACKERS] Dynamic configuration via LDAP in postmaster

2013-12-03 Thread Vasily Soshnikov
I need advise about where is best place for adding such features. Currently I found that 'postmaster' have event loop(including handling SIGHUP) inside PostgressMain(postgress.c) for realoding configuration file, based on my investigation my plan is handling ldap events just before SIGHUP. PS I

Review: [HACKERS] ECPG infrastructure changes part 1, was: Re: ECPG fixes

2013-12-03 Thread Antonin Houska
The changes are very well divided into logical units, so I think I could understand the ideas. I'm not too familiar with the ecpg internals, so consider this at least a coding review. git apply: Clean, except for one finding below Build: no errors/warnings Documentation build: no

Re: [HACKERS] pg_upgrade segfaults when given an invalid PGSERVICE value

2013-12-03 Thread Bruce Momjian
On Fri, Nov 29, 2013 at 04:14:00PM -0500, Bruce Momjian wrote: On Thu, Mar 28, 2013 at 03:06:30PM -0400, Steve Singer wrote: So to summarise: Plan A: The first patch I attached for pg_upgrade + documentation changes, and changing the other places that call PQconndefaults() to accept

Re: [HACKERS] Trust intermediate CA for client certificates

2013-12-03 Thread Bruce Momjian
On Mon, Dec 2, 2013 at 05:35:06PM -0500, Andrew Dunstan wrote: On 12/02/2013 04:17 PM, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Sorry, I should have said: Tom is saying that for his openssl version, a client that passed an intermediate certificate had to supply a

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: That's not really 'solved' unless you feel we can depend on that create extension from URL to work at pg_restore time... I wouldn't have guessed that people would accept that, but I've already been wrong about such things in this thread once.

Re: [HACKERS] Add full object name to the tag field

2013-12-03 Thread David Johnston
Robert Haas wrote On Mon, Dec 2, 2013 at 9:49 AM, Asit Mahato lt; rigid.asit@ gt; wrote: Hi all, I am a newbie. I am unable to understand the to do statement given below. Add full object name to the tag field. eg. for operators we need '=(integer, integer)', instead of just '='.

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: That's not really 'solved' unless you feel we can depend on that create extension from URL to work at pg_restore time... I wouldn't have guessed that people would accept that, but I've already been

Re: [HACKERS] Time-Delayed Standbys

2013-12-03 Thread Christian Kruse
Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It applies and compiles w/o errors or warnings. I set up a master and two hot standbys replicating from the master, one with 5 minutes delay and one without delay. After that I created a new database and generated some

Re: [HACKERS] 9.2.1 index-only scans : abnormal heap fetches after VACUUM FULL

2013-12-03 Thread Bruce Momjian
On Thu, Nov 28, 2013 at 05:38:05PM -0500, Bruce Momjian wrote: I wonder if we ought to mark each page as all-visible in raw_heap_insert() when we first initialize it, and then clear the flag when we come across a tuple that isn't all-visible. We could try to set hint bits on the tuple

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: In more normal cases, however, the system can (and probably should) figure out what was intended by choosing the *shortest* path to get to the intended version. For example, if someone ships 1.0, 1.0--1.1, 1.1, and 1.1--1.2, the system should choose

Re: [HACKERS] Dynamic configuration via LDAP in postmaster

2013-12-03 Thread Heikki Linnakangas
On 12/03/2013 05:44 PM, Vasily Soshnikov wrote: I need advise about where is best place for adding such features. Currently I found that 'postmaster' have event loop(including handling SIGHUP) inside PostgressMain(postgress.c) for realoding configuration file, based on my investigation my plan

Re: Review: [HACKERS] ECPG infrastructure changes part 1, was: Re: ECPG fixes

2013-12-03 Thread Boszormenyi Zoltan
Thanks for the review. 2013-12-03 16:48 keltezéssel, Antonin Houska írta: The changes are very well divided into logical units, so I think I could understand the ideas. I'm not too familiar with the ecpg internals, so consider this at least a coding review. git apply: Clean, except for

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
Hi Alvaro, Noah, On 2013-12-03 15:57:10 +0100, Andres Freund wrote: On 2013-12-03 09:48:23 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-12-03 00:47:07 -0500, Noah Misch wrote: The test spec additionally covers a (probably-related) assertion failure, new

Re: [HACKERS] Allowing parallel pg_restore from pipe

2013-12-03 Thread Bruce Momjian
On Wed, Apr 24, 2013 at 03:33:42PM -0400, Andrew Dunstan wrote: On 04/23/2013 07:53 PM, Timothy Garnett wrote: Hi All, Currently the -j option to pg_restore, which allows for parallelization in the restore, can only be used if the input file is a regular file and not, for ex., a pipe.

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: I understand that you once proposed that and it was shot down but I think we need to move past that now that we've seen what the alternative is.. That isn't to say anything about the code or about you specifically, but, for my part, I really don't like

Re: [HACKERS] Time-Delayed Standbys

2013-12-03 Thread Fabrízio de Royes Mello
On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse christ...@2ndquadrant.comwrote: Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It applies and compiles w/o errors or warnings. I set up a master and two hot standbys replicating from the master, one with 5 minutes

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Pavel Stehule
2013/12/3 Stephen Frost sfr...@snowman.net * Tom Lane (t...@sss.pgh.pa.us) wrote: After sleeping on it, your other suggestion of TABLE OF, or possibly TABLE FROM, is starting to grow on me. Who else has an opinion? Alright, for my 2c, I like having this syntax include 'TABLE' simply

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-03 09:48:23 -0500, Tom Lane wrote: Is this bad enough that we need to re-wrap the release? After looking, I think I am revising my opinion. The broken locking behaviour (outside of freeze, which I don't see how we can fix in time), is

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 12:22:33 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-12-03 09:48:23 -0500, Tom Lane wrote: Is this bad enough that we need to re-wrap the release? After looking, I think I am revising my opinion. The broken locking behaviour (outside of

Re: [HACKERS] WITHIN GROUP patch

2013-12-03 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: Please find attached the latest patch for WITHIN GROUP. This patch is after fixing the merge conflicts. I've started to look at this patch now. I have a couple of immediate reactions to the catalog changes: 1. I really hate the way you've overloaded the

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Any idea how to cheat our way out of that one given the current way heap_freeze_tuple() works (running on both primary and standby)? My only idea was to MultiXactIdWait() if !InRecovery but that's extremly grotty. We can't even realistically create

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Magnus Hagander
On Tue, Dec 3, 2013 at 7:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: Any idea how to cheat our way out of that one given the current way heap_freeze_tuple() works (running on both primary and standby)? My only idea was to MultiXactIdWait() if

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread David E. Wheeler
On Dec 3, 2013, at 9:14 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I understand that it can happen, it still really sucks when it does. delusionnal paragraph, censored for lack of humour (incl. sarcasm) I have not followed this project closely, Dimitri, but I for one have

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Noah Misch
On Tue, Dec 03, 2013 at 04:37:58PM +0100, Andres Freund wrote: On 2013-12-03 10:29:54 -0500, Noah Misch wrote: Sorry, my original report was rather terse. I speak of the scenario exercised by the second permutation in that isolationtester spec. The multixact is later than VACUUM's

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Alvaro Herrera
Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Any idea how to cheat our way out of that one given the current way heap_freeze_tuple() works (running on both primary and standby)? My only idea was to MultiXactIdWait() if !InRecovery but that's extremly grotty. We can't even

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Tue, Dec 3, 2013 at 7:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Maybe we should just bite the bullet and change the WAL format for heap_freeze (inventing an all-new record type, not repurposing the old one, and allowing WAL replay to continue to

Re: [HACKERS] Dynamic configuration via LDAP in postmaster

2013-12-03 Thread Vasily Soshnikov
Thank you for the advice, nowadays we(company where I work) use such scheme but that scheme is not always useful at the stage of development of the back-end infrastructure. Also we have found a better solution : have ldap for dynamic configuraion out of the box. So question is following: if you

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Magnus Hagander
On Tue, Dec 3, 2013 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Tue, Dec 3, 2013 at 7:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Maybe we should just bite the bullet and change the WAL format for heap_freeze (inventing an all-new record

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 13:14:38 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 04:37:58PM +0100, Andres Freund wrote: On 2013-12-03 10:29:54 -0500, Noah Misch wrote: Sorry, my original report was rather terse. I speak of the scenario exercised by the second permutation in that

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Alvaro Herrera
Noah Misch wrote: On 2013-12-03 10:29:54 -0500, Noah Misch wrote: Sorry, my original report was rather terse. I speak of the scenario exercised by the second permutation in that isolationtester spec. The multixact is later than VACUUM's cutoff_multi, so 9.3.1 does not freeze it

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 13:11:13 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Any idea how to cheat our way out of that one given the current way heap_freeze_tuple() works (running on both primary and standby)? My only idea was to MultiXactIdWait() if !InRecovery but that's

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Jeff Davis
On Tue, 2013-12-03 at 10:08 +0200, Heikki Linnakangas wrote: Another perspective is that that's already a situation we'd rather not have. Let's not make it worse by introducing a third way to install an extension, which again requires the extension author to package the extension

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Alvaro Herrera
Andres Freund wrote: I wondered about that myself. How would you suggest the format to look like? ISTM per tuple we'd need: * OffsetNumber off * uint16 infomask * TransactionId xmin * TransactionId xmax I don't see why we'd need infomask2, but so far being overly skimpy in that place

Re: [HACKERS] WITHIN GROUP patch

2013-12-03 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom 1. I really hate the way you've overloaded the transvalue to do Tom something that has approximately nothing to do with transition Tom state (and haven't updated catalogs.sgml to explain that, Tom either). Seems like it'd be cleaner to just

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Jeff Davis
On Tue, 2013-12-03 at 09:20 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: Stephen mentioned using external tools and/or metadata, but to me that sounds like it would require porting the extension away from what's on PGXN today. Not at all- and that'd be the point.

[HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Josh Berkus
All, https://lkml.org/lkml/2013/11/24/133 What this means for us: http://citusdata.com/blog/72-linux-memory-manager-and-your-big-data It seems clear that Kernel.org, since 2.6, has been in the business of pushing major, hackish, changes to the IO stack without testing them or even thinking too

Re: [HACKERS] Time-Delayed Standbys

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse christ...@2ndquadrant.com wrote: Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It applies and compiles w/o errors or

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Noah Misch
On Tue, Dec 03, 2013 at 07:26:38PM +0100, Andres Freund wrote: On 2013-12-03 13:14:38 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 04:37:58PM +0100, Andres Freund wrote: I currently don't see fixing the errorneous freezing of lockers (not the updater though) without changing the wal

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 1:44 PM, Josh Berkus j...@agliodbs.com wrote: All, https://lkml.org/lkml/2013/11/24/133 What this means for us: http://citusdata.com/blog/72-linux-memory-manager-and-your-big-data It seems clear that Kernel.org, since 2.6, has been in the business of pushing major,

Re: [HACKERS] Time-Delayed Standbys

2013-12-03 Thread Joshua D. Drake
On 12/03/2013 10:46 AM, Robert Haas wrote: On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse christ...@2ndquadrant.com wrote: Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 13:49:49 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 07:26:38PM +0100, Andres Freund wrote: On 2013-12-03 13:14:38 -0500, Noah Misch wrote: Not fixing it at all is the real no-go. We'd take both of those undesirables before just tolerating the lost locks in 9.3.

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Noah Misch
On Tue, Dec 03, 2013 at 10:03:32AM -0500, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: After sleeping on it, your other suggestion of TABLE OF, or possibly TABLE FROM, is starting to grow on me. Who else has an opinion? Alright, for my 2c, I like having this syntax

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Andres Freund
On 2013-12-03 15:40:44 -0300, Alvaro Herrera wrote: Andres Freund wrote: I wondered about that myself. How would you suggest the format to look like? ISTM per tuple we'd need: * OffsetNumber off * uint16 infomask * TransactionId xmin * TransactionId xmax I don't see why

Re: [HACKERS] Time-Delayed Standbys

2013-12-03 Thread Andres Freund
On 2013-12-03 13:46:28 -0500, Robert Haas wrote: On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse christ...@2ndquadrant.com wrote: Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Joshua D. Drake
On 12/03/2013 10:44 AM, Josh Berkus wrote: All, https://lkml.org/lkml/2013/11/24/133 What this means for us: http://citusdata.com/blog/72-linux-memory-manager-and-your-big-data It seems clear that Kernel.org, since 2.6, has been in the business of pushing major, hackish, changes to the IO

Re: [HACKERS] 9.2.1 index-only scans : abnormal heap fetches after VACUUM FULL

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 11:25 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Nov 28, 2013 at 05:38:05PM -0500, Bruce Momjian wrote: I wonder if we ought to mark each page as all-visible in raw_heap_insert() when we first initialize it, and then clear the flag when we come across a tuple

[HACKERS] Parallel Select query performance and shared buffers

2013-12-03 Thread Metin Doslu
We have several independent tables on a multi-core machine serving Select queries. These tables fit into memory; and each Select queries goes over one table's pages sequentially. In this experiment, there are no indexes or table joins. When we send concurrent Select queries to these tables, query

[HACKERS] Problem with displaying wide tables in psql

2013-12-03 Thread Sergey Muraviov
Hi. Psql definitely have a problem with displaying wide tables. Even in expanded mode, they look horrible. So I tried to solve this problem. Before the patch: postgres=# \x 1 Expanded display (expanded) is on. postgres=# \pset border 2 Border style (border) is 2. postgres=# select * from

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-03 Thread Alvaro Herrera
Andres Freund wrote: On 2013-12-03 13:49:49 -0500, Noah Misch wrote: On Tue, Dec 03, 2013 at 07:26:38PM +0100, Andres Freund wrote: On 2013-12-03 13:14:38 -0500, Noah Misch wrote: Not fixing it at all is the real no-go. We'd take both of those undesirables before just

Re: [HACKERS] Problem with displaying wide tables in psql

2013-12-03 Thread Pavel Stehule
Hello do you know a pager less trick http://merlinmoncure.blogspot.cz/2007/10/better-psql-with-less.html Regards Pavel Stehule 2013/12/3 Sergey Muraviov sergey.k.murav...@gmail.com Hi. Psql definitely have a problem with displaying wide tables. Even in expanded mode, they look horrible.

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Josh Berkus
On 12/03/2013 10:59 AM, Joshua D. Drake wrote: This seems rather half cocked. I read the article. They found a problem, that really will only affect a reasonably small percentage of users, created a test case, reported it, and a patch was produced. Users with at least one file bigger than 50%

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: This sounds like Inline Extensions to me, which was previously proposed. I've not looked at that proposal very carefully, but I agree that what we're describing is a lot closer to 'inline extensions' than 'extension templates'. If I recall, that proposal

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Tue, Dec 03, 2013 at 10:03:32AM -0500, Stephen Frost wrote: Alright, for my 2c, I like having this syntax include 'TABLE' simply because it's what folks coming from Oracle might be looking for. Following from that, to keep it distinct from the spec's

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: When it comes to dump/reload, I'd much rather see a mechanism which uses our deep understanding of the extension's objects (as database objects) to implement the dump/reload than a text blob which is carried forward from major version to major version

Re: [HACKERS] 9.2.1 index-only scans : abnormal heap fetches after VACUUM FULL

2013-12-03 Thread Bruce Momjian
On Tue, Dec 3, 2013 at 02:01:52PM -0500, Robert Haas wrote: On Tue, Dec 3, 2013 at 11:25 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Nov 28, 2013 at 05:38:05PM -0500, Bruce Momjian wrote: I wonder if we ought to mark each page as all-visible in raw_heap_insert() when we first

Re: [HACKERS] Time-Delayed Standbys

2013-12-03 Thread Simon Riggs
On 18 October 2013 19:03, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: The attached patch is a continuation of Robert's work [1]. Reviewing v2... I made some changes: - use of Latches instead of pg_usleep, so we don't have to wakeup regularly. OK - call

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: On Tue, 2013-12-03 at 09:20 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: Stephen mentioned using external tools and/or metadata, but to me that sounds like it would require porting the extension away from what's on PGXN

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-12-03 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Another issue is that if you are used to the Oracle syntax, in which an UNNEST() is presumed, it's not exactly clear that TABLE ROWS, or any other phrase including TABLE, *doesn't* also imply an UNNEST. So to me that's kind of a strike against Stephen's

  1   2   >