Re: [HACKERS] Failback to old master

2014-11-13 Thread Heikki Linnakangas
On 11/12/2014 03:28 PM, Ants Aasma wrote: On Tue, Nov 11, 2014 at 11:52 PM, Maeldron T. maeld...@gmail.com wrote: As far as I remember (I can’t test it right now but I am 99% sure) promoting the slave makes it impossible to connect the old master to the new one without making a base_backup.

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Jeff Davis
Great discussion. Robert, I think you make a lot of very valid points, but philosophically I am in pretty strong agreement with Andres here. On Fri, 2014-10-31 at 14:29 -0400, Robert Haas wrote: So just refusing parallelism as soon as anything has taken an access exclusive lock doesn't sound

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-13 Thread Simon Riggs
On 13 November 2014 00:20, Jim Nasby jim.na...@bluetreble.com wrote: Isn't the real use-case here that if constraints were valid when you dumped then we shouldn't have to *any* re-validate when we load? (Though, we'd have to be careful of that with CHECK because that can call user code...)

[HACKERS] Typos in CREATE TABLE doc

2014-11-13 Thread Etsuro Fujita
It seems to me there are typos in the reference page for CREATE TABLE. Please find attached a patch. Thanks, Best regards, Etsuro Fujita diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 299cce8..ebcb885 100644 --- a/doc/src/sgml/ref/create_table.sgml +++

Re: [HACKERS] Typos in CREATE TABLE doc

2014-11-13 Thread Heikki Linnakangas
On 11/13/2014 12:45 PM, Etsuro Fujita wrote: It seems to me there are typos in the reference page for CREATE TABLE. The structure of the sentence is a bit funky, but it seems correct to me. If you google for should any, you'll get a bunch of pages discussing similar sentences. I would add a

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Simon Riggs
On 12 October 2014 23:00, Tomas Vondra t...@fuzzy.cz wrote: It however seems to be working sufficiently well at this point, enough to get some useful feedback. So here we go. This looks interesting and useful. What I'd like to check before a detailed review is that this has sufficient

Re: [HACKERS] Typos in CREATE TABLE doc

2014-11-13 Thread Etsuro Fujita
(2014/11/13 20:07), Heikki Linnakangas wrote: On 11/13/2014 12:45 PM, Etsuro Fujita wrote: It seems to me there are typos in the reference page for CREATE TABLE. The structure of the sentence is a bit funky, but it seems correct to me. If you google for should any, you'll get a bunch of

Re: [HACKERS] Add CREATE support to event triggers

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:58 PM, Andres Freund and...@2ndquadrant.com wrote: That's already the situation today with all the logical replication solutions. They *constantly* break in the field. Most commonly because of DDL differences. Right. And that's why it's cool that logical decoding can

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 2:26 AM, Jeff Davis pg...@j-davis.com wrote: The current deadlock detector would see a dependency graph like: A2 - B - A1 But with lock groups, it would see: (A1 A2) - B - (A1 A2) which is a cycle, and can be detected regardless of the synchronization method

Re: [HACKERS] Add CREATE support to event triggers

2014-11-13 Thread Andres Freund
On 2014-11-13 07:17:32 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 4:58 PM, Andres Freund and...@2ndquadrant.com wrote: That's already the situation today with all the logical replication solutions. They *constantly* break in the field. Most commonly because of DDL differences.

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Petr Jelinek
On 13/11/14 07:04, Michael Paquier wrote: On Wed, Nov 12, 2014 at 10:06 PM, Petr Jelinek p...@2ndquadrant.com wrote: Brief list of changes: - the commit timestamp record now stores timestamp, lsn and nodeid Now that not only the commit timestamp is stored, calling that commit timestamp,

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Tomas Vondra
Dne 13 Listopad 2014, 12:31, Simon Riggs napsal(a): On 12 October 2014 23:00, Tomas Vondra t...@fuzzy.cz wrote: It however seems to be working sufficiently well at this point, enough to get some useful feedback. So here we go. This looks interesting and useful. What I'd like to check

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Simon Riggs
On 9 November 2014 16:57, Steve Singer st...@ssinger.info wrote: On 11/07/2014 07:07 PM, Petr Jelinek wrote: The list of what is useful might be long, but we can't have everything there as there are space constraints, and LSN is another 8 bytes and I still want to have some bytes for storing

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Petr Jelinek
On 13/11/14 14:18, Simon Riggs wrote: So in summary... the components are * Commit LSN order is useful for applying changes - available by logical decoding * Commit timestamps and nodeid are useful for conflict resolution - available from this patch Both components have been designed in ways

Re: [HACKERS] Failback to old master

2014-11-13 Thread Ants Aasma
On Thu, Nov 13, 2014 at 10:05 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: In this case the old master will request recovery from a point after the timeline switch and the new master will reply with an error. So it is safe to try re-adding a crashed master as a slave, but this might

Re: [HACKERS] WAL format and API changes (9.5)

2014-11-13 Thread Andres Freund
On 2014-11-13 15:33:44 +0200, Heikki Linnakangas wrote: Here's a new version, with big changes again to the record format. Have a look at xlogrecord.h for the details, but in a nutshell: 1. The overall format is now: XLogRecord, per-block headers, header for main data portion, per-block

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-13 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: has any business including. And if that weren't bad enough, it's been included into utils/rel.h (relcache), This is for the definition of RowSecurityDesc. I'm happy to move that to a utils/rowsecurity.h

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Katharina Büchse
On 13.11.2014 14:11, Tomas Vondra wrote: Dne 13 Listopad 2014, 12:31, Simon Riggs napsal(a): On 12 October 2014 23:00, Tomas Vondra t...@fuzzy.cz wrote: It however seems to be working sufficiently well at this point, enough to get some useful feedback. So here we go. This looks interesting

[HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Andrew Dunstan
I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many thousands of lines to the screen). I'd like a way to be able to specify a

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Thom Brown
On 13 November 2014 15:52, Andrew Dunstan and...@dunslane.net wrote: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many thousands of lines to the

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Andres Freund
On 2014-11-13 11:09:06 -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query

[HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Gilles Darold
Hi, There's a segfault when trying to dump global object from a running 7.4.27 with a pg_dumpall of version 9.3.5 but also 9.2.9. $ pg_dumpall -g -h localhost -p 5474 column number -1 is out of range 0..12 Segmentation fault (core dumped) The problem comes from the first columns of

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 7:31 PM, Andres Freund and...@2ndquadrant.com wrote: But I think it won't work realistically. We have a *lot* of infrastructure that refers to indexes using it's primary key. I don't think we want to touch all those places to also disambiguate on some other factor. All

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Andrew Dunstan
On 11/13/2014 11:09 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2014-11-13 Thread Tomas Vondra
Dne 13 Listopad 2014, 16:51, Katharina Büchse napsal(a): On 13.11.2014 14:11, Tomas Vondra wrote: The only place where I think this might work are the associative rules. It's simple to specify rules like (ZIP code implies city) and we could even do some simple check against the data to see

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Michael Banck
On Thu, Nov 13, 2014 at 05:14:47PM +0100, Andres Freund wrote: On 2014-11-13 11:09:06 -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Alvaro Herrera
Amit Kapila wrote: This patch is in Ready for committer stage for more than 1.5 months. I believe this is an important functionality such that without this tar format of pg_basebackup is not usable on Windows. I feel this will add a value to pg_basebackup utility and moreover the need and

Re: [HACKERS] On partitioning

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 1:39 AM, Stephen Frost sfr...@snowman.net wrote: Agreed- a node tree seems a bit too far to make this really work well.. But, I'm curious what you were thinking specifically? I gave a pretty specific example in my email. A node tree which accepts an argument of the

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread David G Johnston
Andrew Dunstan wrote On 11/13/2014 11:09 AM, Tom Lane wrote: Andrew Dunstan lt; andrew@ gt; writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 10:42 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Nov 11, 2014 at 3:52 PM, Peter Geoghegan p...@heroku.com wrote: I'm pretty puzzled by this. Other than our agree to disagree and defer to committer position on the question of whether or not more than

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Peter Geoghegan
On Thu, Nov 13, 2014 at 8:57 AM, Robert Haas robertmh...@gmail.com wrote: My point is: I am not sure I can be defined as a reviewer of this patch or take any credit in this patch review knowing that the latest version submitted is a simple rebase of the version I did my first review on. Hence,

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Robert Haas
On Wed, Nov 12, 2014 at 8:00 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Nov 12, 2014 at 4:54 PM, Peter Geoghegan p...@heroku.com wrote: Attached patch moves the Levenshtein distance implementation into core. Oops. Somehow managed to send a *.patch.swp file. :-) Here is the actual

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 2:45 AM, Josh Berkus j...@agliodbs.com wrote: On 11/12/2014 10:06 AM, Robert Haas wrote: hat *appears* to be happening is that the pause_at_recovery_target, followed by the restart, on the replica causes it to advance one commit on timeline 1. But *not all the time*;

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston david.g.johns...@gmail.com wrote: Because I might be quite happy with 100 or 200 lines I can just scroll in my terminal's scroll buffer, but want to use the pager for more than that. This is useful especially if I want to scroll back and see

Re: [HACKERS] Inconsistencies in documentation of row-level locking

2014-11-13 Thread Alvaro Herrera
Michael Paquier wrote: Hi all, Currently all the row-level lock modes are described in the page for SELECT query: http://www.postgresql.org/docs/devel/static/explicit-locking.html#LOCKING-ROWS However, after browsing the documentation, I noticed in the page describing all the explicit

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-13 Thread Peter Geoghegan
On Thu, Nov 13, 2014 at 9:36 AM, Robert Haas robertmh...@gmail.com wrote: Committed. I changed varstr_leven() to varstr_levenshtein() because abbrvs cn mk the code hrd to undstnd. And to grep. Thanks. I'll produce a revision of patch 2/2 soon. And I removed the StaticAssertStmt you added,

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Merlin Moncure
On Thu, Nov 13, 2014 at 11:39 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston david.g.johns...@gmail.com wrote: Because I might be quite happy with 100 or 200 lines I can just scroll in my terminal's scroll buffer, but want to use the pager for

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread David Johnston
On Thu, Nov 13, 2014 at 10:55 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Nov 13, 2014 at 11:39 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston david.g.johns...@gmail.com wrote: Because I might be quite happy with 100 or 200 lines I

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 7:52 AM, Andrew Dunstan and...@dunslane.net wrote: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 8:14 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-13 11:09:06 -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Merlin Moncure
On Thu, Nov 13, 2014 at 12:03 PM, David Johnston david.g.johns...@gmail.com wrote: On Thu, Nov 13, 2014 at 10:55 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Nov 13, 2014 at 11:39 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 13, 2014 at 11:54 AM, David G Johnston

[HACKERS] array exclusion constraints

2014-11-13 Thread francisk
CREATE TABLE test ( foo text[], EXCLUDE USING gist (foo WITH ) ); ERROR: data type text[] has no default operator class for access method gist HINT: You must specify an operator class for the index or define a default operator class for the data type. It looks like exclusion

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis pg...@j-davis.com wrote: If two backends both have an exclusive lock on the relation for a join operation, that implies that they need to do their own synchronization, because obviously the lock manager is not doing it for them. This doesn't make

Re: [HACKERS] Another logical decoding assertion failure

2014-11-13 Thread Andres Freund
On 2014-08-15 14:53:45 +0200, Antonin Houska wrote: http://www.postgresql.org/message-id/blu436-smtp12682d628f61ab9736099c3dc...@phx.gbl recalls me that I also saw an assertion failure recently. Although I wanted to isolate and report my issue when my vacation is over, this report made me

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-13 Thread Andres Freund
Hi Steve, On 2014-10-28 22:38:29 -0400, Steve Singer wrote: On 10/28/2014 01:31 PM, Andres Freund wrote: On 2014-10-25 18:18:07 -0400, Steve Singer wrote: My logical decoding plugin is occasionally getting this error could not resolve cmin/cmax of catalog tuple I get this when my output

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Gilles Darold gilles.dar...@dalibo.com writes: There's a segfault when trying to dump global object from a running 7.4.27 with a pg_dumpall of version 9.3.5 but also 9.2.9. Hm ... I make a practice of checking pg_dump's backwards compatibility from time to time, but I confess I've not tested

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis pg...@j-davis.com wrote: If two backends both have an exclusive lock on the relation for a join operation, that implies that they need to do their own synchronization, because obviously the lock manager is

Re: [HACKERS] alter user/role CURRENT_USER

2014-11-13 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: - Storage for new information The new struct NameId stores an identifier which telling what it logically is using the new enum NameIdTypes. I think NameId is a bad name for this. My point is that NameId, as it stands, might be a name for anything, not just a role;

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Andres Freund
On November 13, 2014 8:50:18 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis pg...@j-davis.com wrote: If two backends both have an exclusive lock on the relation for a join operation, that implies that they need to

[HACKERS] alternative model for handling locking in parallel groups

2014-11-13 Thread Robert Haas
Discussion of my incomplete group locking patch seems to have converged around two points: (1) Everybody agrees that undetected deadlocks are unacceptable. (2) Nobody agrees with my proposal to treat locks held by group members as mutually non-conflicting. As was probably evident from the emails

Re: [HACKERS] array exclusion constraints

2014-11-13 Thread Antonin Houska
franc...@hush.com wrote: CREATE TABLE test ( foo text[], EXCLUDE USING gist (foo WITH ) ); ERROR: data type text[] has no default operator class for access method gist HINT: You must specify an operator class for the index or define a default operator class for the data type.

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-13 Thread Peter Eisentraut
On 11/12/14 7:31 PM, Andres Freund wrote: Yes, it sucks. But it beats not being able to reindex a relation with a primary key (referenced by a fkey) without waiting several hours by a couple magnitudes. And that's the current situation. That's fine, but we have, for better or worse, defined

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 8:18 AM, Simon Riggs si...@2ndquadrant.com wrote: Ordering transactions in LSN order is very precisly the remit of the existing logical decoding API. Any user that wishes to see a commits in sequence can do so using that API. BDR already does this, as do other users of

[HACKERS] Size of regression database

2014-11-13 Thread Tom Lane
I was testing backwards compatibility of pg_dumpall just now, and was somewhat astonished to notice the size of the output for the regression database compared to what it was not too long ago: -rw-rw-r--. 1 tgl tgl 4509135 Nov 13 16:19 dumpall.83 -rw-rw-r--. 1 tgl tgl 4514441 Nov 13 16:24

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Peter Eisentraut
On 11/13/14 11:52 AM, Alvaro Herrera wrote: Amit Kapila wrote: This patch is in Ready for committer stage for more than 1.5 months. I believe this is an important functionality such that without this tar format of pg_basebackup is not usable on Windows. I feel this will add a value to

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 4:33 PM, Peter Eisentraut pete...@gmx.net wrote: I'm not fully on board with that premise. (Get a better tar tool. Submit a patch.) Noah was unable to find one that works: http://www.postgresql.org/message-id/20130801161519.ga334...@tornado.leadboat.com If most tar

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-13 Thread Andres Freund
On November 13, 2014 10:23:41 PM CET, Peter Eisentraut pete...@gmx.net wrote: On 11/12/14 7:31 PM, Andres Freund wrote: Yes, it sucks. But it beats not being able to reindex a relation with a primary key (referenced by a fkey) without waiting several hours by a couple magnitudes. And that's the

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-13 Thread Jeff Davis
On Thu, Nov 13, 2014 at 11:26 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 13, 2014 at 3:38 AM, Jeff Davis pg...@j-davis.com wrote: If two backends both have an exclusive lock on the relation for a join operation, that implies that they need to do their own synchronization,

Re: [HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-11-13 Thread Heikki Linnakangas
On 11/11/2014 06:55 PM, Andres Freund wrote: On 2014-11-03 21:58:51 +0200, Heikki Linnakangas wrote: PS. I find the name ReplicationSlotOnDiskDynamicSize confusing, as it is in fact a fixed size struct. I gather it's expected that the size of that part might change across versions, but by that

Re: [HACKERS] Inconsistencies in documentation of row-level locking

2014-11-13 Thread Michael Paquier
On Fri, Nov 14, 2014 at 2:47 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thanks, I applied it to master and 9.4 after some slight editorialization. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Gilles Darold gilles.dar...@dalibo.com writes: In the same query there is another bug introduced by commit 491c029 that add Row-Level Security Policies. Current master code has a broken pg_dumpall when trying to dump from a backend lower than 8.1. Actually, I think that code is not just

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Gilles Darold gilles.dar...@dalibo.com writes: In the same query there is another bug introduced by commit 491c029 that add Row-Level Security Policies. Current master code has a broken pg_dumpall when trying to dump from a backend lower than 8.1.

Re: [HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-11-13 Thread Andres Freund
On 2014-11-14 00:04:52 +0200, Heikki Linnakangas wrote: On 11/11/2014 06:55 PM, Andres Freund wrote: On 2014-11-03 21:58:51 +0200, Heikki Linnakangas wrote: PS. I find the name ReplicationSlotOnDiskDynamicSize confusing, as it is in fact a fixed size struct. I gather it's expected that the

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: I'm happy to fix it either way (and fix it for 8.1, and back to.. what? Postgres95?) Err. That might have come off poorly- I didn't mean that sarcastically but was really wondering how far back you test (or how far back we feel pg_dumpall needs to

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Andres Freund
On 2014-11-13 18:00:45 -0500, Stephen Frost wrote: * Stephen Frost (sfr...@snowman.net) wrote: I'm happy to fix it either way (and fix it for 8.1, and back to.. what? Postgres95?) Err. That might have come off poorly- I didn't mean that sarcastically but was really wondering how far back

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-13 18:00:45 -0500, Stephen Frost wrote: Err. That might have come off poorly- I didn't mean that sarcastically but was really wondering how far back you test (or how far back we feel pg_dumpall needs to work against..). pg_dump

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Actually, I think that code is not just under-tested but poorly thought out. It will dump ALL roles from a pre-9.5 database with NOBYPASSRLS; even superusers. Superusers are always considered to have it,

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Andres Freund
On 2014-11-13 18:24:53 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-11-13 18:00:45 -0500, Stephen Frost wrote: Err. That might have come off poorly- I didn't mean that sarcastically but was really wondering how far back you test (or how far back we feel

Re: [HACKERS] tracking commit timestamps

2014-11-13 Thread Simon Riggs
On 13 November 2014 21:24, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 13, 2014 at 8:18 AM, Simon Riggs si...@2ndquadrant.com wrote: Ordering transactions in LSN order is very precisly the remit of the existing logical decoding API. Any user that wishes to see a commits in sequence

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Simon Riggs
On 12 November 2014 00:54, Robert Haas robertmh...@gmail.com wrote: I'm going to aim for the simpler: Hash Aggregate - Parallel Seq Scan Workers: 4 Yeah, I know that won't perform as well as what you're proposing, but I'm fairly sure it's simpler. Simple is best, so +1. -- Simon

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Simon Riggs
On 12 November 2014 07:54, David Rowley dgrowle...@gmail.com wrote: On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs si...@2ndquadrant.com wrote: This plan type is widely used in reporting queries, so will hit the mainline of BI applications and many Mat View creations. This will allow SELECT

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Simon Riggs
On 12 November 2014 00:54, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 11, 2014 at 3:29 AM, Simon Riggs si...@2ndquadrant.com wrote: * only functions marked as CONTAINS NO SQL We don't really know what proisparallel is, but we do know what CONTAINS NO SQL means and can easily check

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-13 Thread Peter Eisentraut
On 11/13/14 5:07 AM, Simon Riggs wrote: On 13 November 2014 00:20, Jim Nasby jim.na...@bluetreble.com wrote: Isn't the real use-case here that if constraints were valid when you dumped then we shouldn't have to *any* re-validate when we load? (Though, we'd have to be careful of that with

Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
Sam Saffron sam.saff...@gmail.com writes: I have hit a rather odd issue with prepared queries on both pg 9.3 and 9.4 beta. I have this table (copy at http://samsaffron.com/testing.db.gz) with a very odd performance profile: Interesting case. The issue seems to be that your statistics look

[HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread David G Johnston
Tom Lane-2 wrote In the meantime, I assume that your real data contains a small percentage of values other than these two? If so, maybe cranking up the statistics target would help. If the planner knows that there are more than two values in the column, I think it would be less optimistic

[HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Simon Riggs
Limit (cost= rows=20 width=175) (actual time= rows=20 loops=1) - Sort (cost= rows=568733 width=175) (actual time= rows=20 loops=1) Sort Method: top-N heapsort The Sort estimate shows 568733 rows, whereas the actual rows are 20. Both are correct, in a way. The node

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
David G Johnston david.g.johns...@gmail.com writes: Tom Lane-2 wrote In the meantime, I assume that your real data contains a small percentage of values other than these two? If so, maybe cranking up the statistics target would help. If the planner knows that there are more than two values

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Limit (cost= rows=20 width=175) (actual time= rows=20 loops=1) - Sort (cost= rows=568733 width=175) (actual time= rows=20 loops=1) Sort Method: top-N heapsort The Sort estimate shows 568733 rows, whereas the actual rows

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread David Rowley
On Fri, Nov 14, 2014 at 1:19 PM, Simon Riggs si...@2ndquadrant.com wrote: On 12 November 2014 07:54, David Rowley dgrowle...@gmail.com wrote: On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs si...@2ndquadrant.com wrote: This plan type is widely used in reporting queries, so will hit the

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-13 Thread Andreas Karlsson
On 11/13/2014 03:38 AM, Alvaro Herrera wrote: configure is a generated file. If your patch touches it but not configure.in, there is a problem. Thanks for pointing it out, I have now fixed it. -- Andreas Karlsson diff --git a/configure b/configure new file mode 100755 index c4f70e8..bb801b4

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-13 Thread Peter Eisentraut
On 11/12/14 10:59 AM, Andres Freund wrote: No, I really don't want to do that. When trying to see whether logical replication started that's imo quite an importantdetail. Especially when first seing ereport(LOG, (errmsg(logical decoding found initial

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread David Johnston
On Thu, Nov 13, 2014 at 5:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: David G Johnston david.g.johns...@gmail.com writes: Tom Lane-2 wrote In the meantime, I assume that your real data contains a small percentage of values other than these two? If so, maybe cranking up the statistics

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-13 Thread Kouhei Kaigai
On 12 November 2014 07:54, David Rowley dgrowle...@gmail.com wrote: On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs si...@2ndquadrant.com wrote: This plan type is widely used in reporting queries, so will hit the mainline of BI applications and many

Re: [HACKERS] Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: What's bothering me is that I see this in pg_dumpall output from a 9.4 or earlier database: ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION NOBYPASSRLS; Ah, yeah, good point. That means that if you do a pg_upgrade

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-13 Thread Peter Eisentraut
On 11/13/14 7:57 PM, Andreas Karlsson wrote: On 11/13/2014 03:38 AM, Alvaro Herrera wrote: configure is a generated file. If your patch touches it but not configure.in, there is a problem. Thanks for pointing it out, I have now fixed it. There is something odd about your patch. I claims

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
David Johnston david.g.johns...@gmail.com writes: ​While planner hints comes to mind...on the SQL side can we extend the PREPARE command with two additional keywords?​ ​PREPARE name [ ( data_type [, ...] ) ] [ [NO] GENERIC ​] ​ ​AS statement Don't really see the point. The

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread David G Johnston
Tom Lane-2 wrote Simon Riggs lt; simon@ gt; writes: Limit (cost= rows=20 width=175) (actual time= rows=20 loops=1) - Sort (cost= rows=568733 width=175) (actual time= rows=20 loops=1) Sort Method: top-N heapsort The Sort estimate shows 568733 rows, whereas

Re: [HACKERS] EXPLAIN ANALYZE output weird for Top-N Sort

2014-11-13 Thread Tom Lane
David G Johnston david.g.johns...@gmail.com writes: Tom Lane-2 wrote [ shrug... ] The estimated value is the planner's estimate of what would happen *if you ran the node to completion*, which in practice doesn't happen because of the LIMIT. I don't see how a sort node cannot run to

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-13 Thread Peter Eisentraut
On 11/12/14 1:04 PM, Robert Haas wrote: I suggest: 1. Logical decoding will begin using saved snapshot. 2. Transaction ID %u finished; no more running transactions. 3. There are no running transactions. I went with that. Thank you. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-13 Thread Andres Freund
On 2014-11-13 21:03:57 -0500, Peter Eisentraut wrote: On 11/12/14 1:04 PM, Robert Haas wrote: I suggest: 1. Logical decoding will begin using saved snapshot. 2. Transaction ID %u finished; no more running transactions. 3. There are no running transactions. I went with that. Thank

Re: [HACKERS] On partitioning

2014-11-13 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Nov 13, 2014 at 1:39 AM, Stephen Frost sfr...@snowman.net wrote: but with declarative partitioning, I expect us to eventually be able to eliminate complete partitions from consideration on both sides of a partition-table join

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-13 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: Well, if you *only* move RowSecurityDesc and not RowSecurityPolicy, okay, but that seems a bit useless/inconsistent if I'm reading it right that RowSecurityDesc contains a List of RowSecurityPolicy structs. Yes, good point. What seems possibly

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-13 Thread Steve Singer
On 11/13/2014 02:44 PM, Andres Freund wrote: H I've pushed a fix for a bug that could possibly also cause this. Although it'd be odd that it always hits the user catalog table. Except if your tests mostly modify the slony tables, but do not do much DDL otherwise? The test I was running

[HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Sam Saffron
Thank you so much! So to recap the general way to reproduce this issue is: create table products(id int primary key, type varchar); insert into products select generate_series(1,1), 'aaa'; insert into products select generate_series(10001,2), 'bbb'; create index idx on products(type);

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Amit Kapila
On Fri, Nov 14, 2014 at 3:03 AM, Peter Eisentraut pete...@gmx.net wrote: On 11/13/14 11:52 AM, Alvaro Herrera wrote: Amit Kapila wrote: This patch is in Ready for committer stage for more than 1.5 months. I believe this is an important functionality such that without this tar format of

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-11-13 Thread Robert Haas
On Thu, Nov 13, 2014 at 10:37 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Nov 14, 2014 at 3:03 AM, Peter Eisentraut pete...@gmx.net wrote: On 11/13/14 11:52 AM, Alvaro Herrera wrote: Amit Kapila wrote: This patch is in Ready for committer stage for more than 1.5 months. I

[HACKERS] Re: Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Noah Misch
On Thu, Nov 13, 2014 at 08:24:36PM -0500, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: What's bothering me is that I see this in pg_dumpall output from a 9.4 or earlier database: ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION

Re: [HACKERS] Unintended restart after recovery error

2014-11-13 Thread Fujii Masao
On Thu, Nov 13, 2014 at 8:30 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Nov 12, 2014 at 4:52 PM, Antonin Houska a...@cybertec.at wrote: Fujii Masao masao.fu...@gmail.com wrote: On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska a...@cybertec.at wrote: While looking at

Re: [HACKERS] Re: Segmentation fault in pg_dumpall from master down to 9.1 and other bug introduced by RLS

2014-11-13 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Thu, Nov 13, 2014 at 08:24:36PM -0500, Stephen Frost wrote: Agreed. I'll take care of both and we'll make sure the new role attributes being added will do the same for upgrades also. That would make pg_dumpall less faithful for every role other than

  1   2   >