Re: [HACKERS] replicating DROP commands across servers

2015-01-02 Thread Jeff Janes
On Mon, Dec 29, 2014 at 2:15 PM, Alvaro Herrera wrote: > Here's a patch that tweaks the grammar to use TypeName in COMMENT, > SECURITY LABEL, and DROP for the type and domain cases. The required > changes in the code are pretty minimal, thankfully. Note the slight > changes to the new object_ad

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-02 Thread Amit Kapila
On Fri, Jan 2, 2015 at 8:34 PM, Kevin Grittner wrote: > > Amit Kapila wrote: > > > Notes for Committer - > > There is one behavioural difference in the handling of --analyze-in-stages > > switch, when individual tables (by using -t option) are analyzed by > > using this switch, patch will process

Re: [HACKERS] History of WAL_LEVEL (archive vs hot_standby)

2015-01-02 Thread Noah Misch
On Thu, Mar 27, 2014 at 08:16:13PM -0700, David Johnston wrote: > Slightly tangential but are the locking operations associated with the > recent bugfix generated in both (all?) modes or only hot_standby? All modes. > I thought > it strange that transient locking operations were output with WAL t

Re: [HACKERS] Detecting backend failures via libpq / DBD::Pg

2015-01-02 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/30/2014 08:43 AM, Greg Sabino Mullane wrote: > I am working on enhancing the ping() method of DBD::Pg. The goal of > that is for a user to be able to determine if the connection to the > database is still valid. This entire concept is flawed. IM

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-02 Thread Peter Geoghegan
On Thu, Jan 1, 2015 at 11:17 PM, Peter Geoghegan wrote: > I've been working on fixing the bugs that Jeff Janes found [1] with > approach #2 to value locking [2]. Approach #1 was unaffected. Unfortunately, exclusion constraints (which only value locking approach #2 has support for, for the IGNORE

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Claudio Freire
On Fri, Jan 2, 2015 at 7:57 PM, Jim Nasby wrote: > On 1/2/15, 4:18 PM, Tom Lane wrote: >> >> Heikki Linnakangas writes: >>> >>> >On 01/02/2015 11:41 PM, Tom Lane wrote: >>What might be worth trying is establishing a hard-and-fast boundary >>between C land and SQL land, with bitwise

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Jim Nasby
On 1/2/15, 4:18 PM, Tom Lane wrote: Heikki Linnakangas writes: >On 01/02/2015 11:41 PM, Tom Lane wrote: >>What might be worth trying is establishing a hard-and-fast boundary >>between C land and SQL land, with bitwise names in C and bytewise names >>in SQL. This would mean, for example, that

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-02 Thread Josh Berkus
On 01/02/2015 01:57 AM, Heikki Linnakangas wrote: > wal_keep_segments does not affect the calculation of CheckPointSegments. > If you set wal_keep_segments high enough, checkpoint_wal_size will be > exceeded. The other alternative would be to force a checkpoint earlier, > i.e. lower CheckPointSegme

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Tom Lane
Heikki Linnakangas writes: > On 01/02/2015 11:41 PM, Tom Lane wrote: >> What might be worth trying is establishing a hard-and-fast boundary >> between C land and SQL land, with bitwise names in C and bytewise names >> in SQL. This would mean, for example, that int4pl() would be renamed to >> int3

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Heikki Linnakangas
On 01/02/2015 11:41 PM, Tom Lane wrote: What might be worth trying is establishing a hard-and-fast boundary between C land and SQL land, with bitwise names in C and bytewise names in SQL. This would mean, for example, that int4pl() would be renamed to int32pl() so far as the C function goes, but

Re: [HACKERS] Final Patch for GROUPING SETS

2015-01-02 Thread Jim Nasby
On 12/31/14, 3:05 PM, Noah Misch wrote: On Wed, Dec 31, 2014 at 05:33:43PM +, Andrew Gierth wrote: > >"Noah" == Noah Misch writes: > > Noah> Suppose one node orchestrated all sorting and aggregation. > >Well, that has the downside of making it into an opaque blob, without >actually ga

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Tom Lane
Jim Nasby writes: > On 12/31/14, 8:13 AM, Andres Freund wrote: >> Note that the C datatype has been int32/int64 for a while now, it's just >> the SQL datatype and the names of its support functions. Given that, >> afaiu, we're talking about the C datatype it seems pretty clear that it >> should be

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Jim Nasby
On 12/31/14, 8:13 AM, Andres Freund wrote: On 2015-01-01 03:00:50 +1300, David Rowley wrote: 2. References to int16 meaning 16 bytes. I'm really in two minds about this, it's quite nice to keep the natural flow, int4, int8, int16, but I can't help think that this will confuse someone one day.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-01-02 Thread Tom Lane
Ashutosh Bapat writes: > While looking at the patch for supporting inheritance on foreign tables, I > noticed that if a transaction makes changes to more than two foreign > servers the current implementation in postgres_fdw doesn't make sure that > either all of them rollback or all of them commit

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > To be specific, desirable in streaming replication scenarios that don't > use SSL compression. (What percentage is that?) It is something we > should mention in the docs for this feature? Considering how painful the SSL rengeotiation problems were and

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-02 Thread Andres Freund
On 2014-12-29 16:59:05 -0500, Bruce Momjian wrote: > diff --git a/src/backend/storage/buffer/buf_init.c > b/src/backend/storage/buffer/buf_init.c > new file mode 100644 > index ff6c713..c4dce5b > *** a/src/backend/storage/buffer/buf_init.c > --- b/src/backend/storage/buffer/buf_init.c > **

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 02:18:12PM -0300, Claudio Freire wrote: > On Fri, Jan 2, 2015 at 2:11 PM, Andres Freund wrote: > >> , I now see the compression patch as something that has negatives, so > >> has to be set by the user, and only wins in certain cases. I am > >> disappointed, and am trying t

Re: [HACKERS] Publish autovacuum informations

2015-01-02 Thread Jim Nasby
On 1/1/15, 4:17 PM, Noah Misch wrote: I'd be all right with putting the data structure declarations in a file >named something like autovacuum_private.h, especially if it carried an >annotation that "if you depend on this, don't be surprised if we break >your code in future". Such an annotation

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Claudio Freire
On Fri, Jan 2, 2015 at 2:11 PM, Andres Freund wrote: >> , I now see the compression patch as something that has negatives, so >> has to be set by the user, and only wins in certain cases. I am >> disappointed, and am trying to figure out how this became such a >> marginal win for 9.5. :-( > > I

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 06:11:29PM +0100, Andres Freund wrote: > > My negativity is not that I don't want it, but I want to understand why > > it isn't better than I remembered. You are basically telling me it was > > always a marginal win. :-( Boohoo! > > No, I didn't. I told you that *IN ONE

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Andres Freund
On 2015-01-02 12:06:33 -0500, Bruce Momjian wrote: > On Fri, Jan 2, 2015 at 05:55:52PM +0100, Andres Freund wrote: > > On 2015-01-02 11:52:42 -0500, Bruce Momjian wrote: > > > Why are we not seeing the 33% compression and 15% performance > > > improvement he saw? What am I missing here? > > > >

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 05:55:52PM +0100, Andres Freund wrote: > On 2015-01-02 11:52:42 -0500, Bruce Momjian wrote: > > Why are we not seeing the 33% compression and 15% performance > > improvement he saw? What am I missing here? > > To see performance improvements something needs to be the bottl

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Andres Freund
On 2015-01-02 11:52:42 -0500, Bruce Momjian wrote: > Why are we not seeing the 33% compression and 15% performance > improvement he saw? What am I missing here? To see performance improvements something needs to be the bottleneck. If WAL writes/flushes aren't that in the tested scenario, you won'

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 10:15:57AM -0600, k...@rice.edu wrote: > On Fri, Jan 02, 2015 at 01:01:06PM +0100, Andres Freund wrote: > > On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote: > > > I still don't understand the value of adding WAL compression, given the > > > high CPU usage and minimal perfo

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread k...@rice.edu
On Fri, Jan 02, 2015 at 01:01:06PM +0100, Andres Freund wrote: > On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote: > > I still don't understand the value of adding WAL compression, given the > > high CPU usage and minimal performance improvement. The only big > > advantage is WAL storage, but aga

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-02 Thread Kevin Grittner
Amit Kapila wrote: > Notes for Committer - > There is one behavioural difference in the handling of --analyze-in-stages > switch, when individual tables (by using -t option) are analyzed by > using this switch, patch will process (in case of concurrent jobs) all the > given tables for stage-1 and

Re: [HACKERS] What exactly is our CRC algorithm?

2015-01-02 Thread Heikki Linnakangas
On 01/01/2015 09:17 AM, Abhijit Menon-Sen wrote: Hi. OK, here are the patches with the various suggestions applied. I found that the alignment didn't seem to make much difference for the CRC32* instructions, so I changed to process (len/8)*8bytes followed by (len%8)*1bytes, the way the Linux ke

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-02 Thread Amit Kapila
On Fri, Jan 2, 2015 at 11:47 AM, Dilip kumar wrote: > > On 31 December 2014 18:36, Amit Kapila Wrote, > > >The patch looks good to me. I have done couple of > > >cosmetic changes (spelling mistakes, improve some comments, > > >etc.), check the same once and if you are okay, we can move > > >ahead

Re: [HACKERS] parallel mode and parallel contexts

2015-01-02 Thread Amit Kapila
On Thu, Dec 18, 2014 at 1:23 AM, Robert Haas wrote: > > > In the meantime, I had a good chat with Heikki on IM yesterday which > gave me some new ideas on how to fix up the transaction handling in > here, which I am working on implementing. So hopefully I will have > that by then. > > I am also h

Re: [HACKERS] compress method for spgist - 2

2015-01-02 Thread Heikki Linnakangas
On 12/23/2014 03:02 PM, Teodor Sigaev wrote: >I think we'll need a separate SpGistTypeDesc for the input type. Or perhaps a >separate SpGistTypeDesc for the reconstructed value and an optional decompress >method to turn the reconstructedValue back into an actual reconstructed input >datum. Or som

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Andres Freund
On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote: > I still don't understand the value of adding WAL compression, given the > high CPU usage and minimal performance improvement. The only big > advantage is WAL storage, but again, why not just compress the WAL file > when archiving. before: pg_xl

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Thom Brown
On 2 January 2015 at 11:13, Amit Kapila wrote: > > On Fri, Jan 2, 2015 at 4:09 PM, Thom Brown wrote: > > > > On 1 January 2015 at 10:34, Amit Kapila wrote: > >> > >> > Running it again, I get the same issue. This is with > parallel_seqscan_degree set to 8, and the crash occurs with 4 and 2 too

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2015-01-02 Thread Dennis Kögel
Am 31.12.2014 um 11:40 schrieb Michael Paquier : >> As long as master is fixed, I don't actually care. But I agree with Dennis >> that it's hard to see what's been commited with all the different issues >> found, and if any commits were done, in which branch. I'd like to be able to >> tell my custo

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Amit Kapila
On Fri, Jan 2, 2015 at 4:09 PM, Thom Brown wrote: > > On 1 January 2015 at 10:34, Amit Kapila wrote: >> >> > Running it again, I get the same issue. This is with parallel_seqscan_degree set to 8, and the crash occurs with 4 and 2 too. >> > >> > This doesn't happen if I set the pgbench scale to 5

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Thom Brown
On 1 January 2015 at 10:34, Amit Kapila wrote: > > Running it again, I get the same issue. This is with > parallel_seqscan_degree set to 8, and the crash occurs with 4 and 2 too. > > > > This doesn't happen if I set the pgbench scale to 50. I suspect this is > a OOM issue. My laptop has 16GB R

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Amit Kapila
On Thu, Jan 1, 2015 at 11:29 PM, Robert Haas wrote: > > On Thu, Jan 1, 2015 at 12:00 PM, Fabrízio de Royes Mello > wrote: > > Can we check the number of free bgworkers slots to set the max workers? > > The real solution here is that this patch can't throw an error if it's > unable to obtain the d

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Thom Brown
On 1 January 2015 at 17:59, Robert Haas wrote: > On Thu, Jan 1, 2015 at 12:00 PM, Fabrízio de Royes Mello > wrote: > > Can we check the number of free bgworkers slots to set the max workers? > > The real solution here is that this patch can't throw an error if it's > unable to obtain the desired

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-02 Thread Heikki Linnakangas
On 01/01/2015 03:24 AM, Josh Berkus wrote: Please remind me because I'm having trouble finding this in the archives: how does wal_keep_segments interact with the new settings? It's not very straightforward. First of all, min_recycle_wal_size has a different effect than wal_keep_segments. Raisi