Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-09 Thread Michael Paquier
On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier michael.paqu...@gmail.com wrote: The additional process at promotion sounds like a good idea, I'll try to get a patch done tomorrow. This would result as well in removing the XLogArchiveForceDone stuff. Either way, not that I have been able to

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 10:49 PM, Simon Riggs si...@2ndquadrant.com wrote: Do you really expect me to do major work on some aspect of the syntax like that, given, as you say, that nobody explicitly agreed with me (and only you disagreed with me)? The only remark I heard on that was from you

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-10-09 Thread Heikki Linnakangas
On 10/09/2014 07:47 AM, furu...@pm.nttdata.co.jp wrote: If we remove --fsync-interval, resoponse time to user will not be delay. Although, fsync will be executed multiple times in a short period. And there is no way to solve the problem without --fsync-interval, what should we do about it? I'm

Re: [HACKERS] What exactly is our CRC algorithm?

2014-10-09 Thread Heikki Linnakangas
On 10/09/2014 01:23 AM, Gavin Flower wrote: On 09/10/14 10:13, Andres Freund wrote: If we're switching to a saner computation, we should imo also switch to a better polynom - CRC-32C has better error detection capabilities than CRC32 and is available in hardware. As we're paying the price pf

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Simon Riggs
On 9 October 2014 07:27, Peter Geoghegan p...@heroku.com wrote: Please be more specific. Do not use CONFLICTING() which looks like it is a function. Instead, use a row qualifier, such as NEW, OLD etc to reference values from the incoming data e.g. CONFLICTING.value rather than

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-09 Thread Michael Paquier
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Where do the FF files come from? In 9.2, FF-segments are not supposed to created, ever. I think we should add a check in walreceiver, to throw an error if the master sends an invalid WAL pointer, pointing to

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 12:38 AM, Simon Riggs si...@2ndquadrant.com wrote: Do not use CONFLICTING() which looks like it is a function. So is ROW(). Or COALESCE(). Instead, use a row qualifier, such as NEW, OLD etc to reference values from the incoming data e.g. CONFLICTING.value rather than

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 11:11 AM, Peter Geoghegan p...@heroku.com wrote: On Thu, Oct 9, 2014 at 12:38 AM, Simon Riggs si...@2ndquadrant.com wrote: Do not use CONFLICTING() which looks like it is a function. So is ROW(). Or COALESCE(). ROW and COALESCE behave almost like functions: they operate

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Simon Riggs
On 9 October 2014 09:11, Peter Geoghegan p...@heroku.com wrote: You may also wish to support the AS keyword, as MERGE does to make the above even more clear. e.g. SET col = EXISTING.col + NEW.col That's less clear, IMV. EXISTING.col is col - the very same Var. So why qualify that it's the

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:33 AM, Marti Raudsepp ma...@juffo.org wrote: ROW and COALESCE behave almost like functions: they operate on any expression or value you pass to them. Okay, then like CONFLICTING() is like many of the XML expressions. -- Peter Geoghegan -- Sent via pgsql-hackers

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:41 AM, Simon Riggs si...@2ndquadrant.com wrote: YES, which is why I specifically requested the ability to reference the incoming data. My point is that people are not really inclined to use an alias in UPDATEs in general when referring to the target. The thing that

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:56 AM, Peter Geoghegan p...@heroku.com wrote: My point is that people are not really inclined to use an alias in UPDATEs in general when referring to the target. The thing that seems special (and worthy of special qualification) is the reference to what you call the

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-08 13:52:14 -0400, Robert Haas wrote: On Sun, Jun 29, 2014 at 9:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Meh. Even SELECT 1 is going to be doing *far* more pallocs than that to get through raw parsing, parse analysis, planning, and execution startup. If you can find a few

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-10-09 Thread furuyao
If we remove --fsync-interval, resoponse time to user will not be delay. Although, fsync will be executed multiple times in a short period. And there is no way to solve the problem without --fsync-interval, what should we do about it? I'm sorry, I didn't understand that. Here is

[HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alexey Bashtanov
Hello! Autovacuum daemon performs vacuum when the number of rows updated/deleted (n_dead_tuples) reaches some threshold. Similarly it performs analyze when the number of rows changed in any way (incl. inserted). When a table is mostly insert-only, its visibility map is not updated as vacuum

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-09 Thread Andres Freund
On 2014-10-09 14:06:35 +0900, Kyotaro HORIGUCHI wrote: Hello, simplly inhibit set retry flag when ProcDiePending in my_sock_write seems enough. But it returns with SSL_ERROR_SYSCALL not SSL_ERROR_WANT_WRITE so I modified the patch 4 as the attached patch. Why is that necessary? It seems

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Robert Haas
On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund and...@2ndquadrant.com wrote: Interesting - in my local profile AtStart_Inval() is more pronounced than AfterTriggerBeginQuery(). I've quickly and in a ugly fashion hacked AtStart_Inval() out of readonly queries ontop of your patch. Together that

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-09 08:18:18 -0400, Robert Haas wrote: On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund and...@2ndquadrant.com wrote: Interesting - in my local profile AtStart_Inval() is more pronounced than AfterTriggerBeginQuery(). I've quickly and in a ugly fashion hacked AtStart_Inval() out of

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Robert Haas
On Wed, Oct 8, 2014 at 1:52 PM, Michael Banck michael.ba...@credativ.de wrote: Looking at it from a DBA perspective, this would indeed be better, yes. However, I see a few issues with that: 1. If you are using an init script (or another wrapper around pg_ctl), you don't get any of its output

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Andres Freund
On 2014-10-02 15:21:48 +0200, Michael Banck wrote: Hi, we have seen repeatedly that users can be confused about why PostgreSQL is not shutting down even though they requested it. Usually, this is because `log_checkpoints' is not enabled and the final checkpoint is being written, delaying

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-09 Thread MauMau
From: Simon Riggs si...@2ndquadrant.com I hope we can get pgAudit in as a module for 9.5. I also hope that it will stimulate the requirements/funding of further work in this area, rather than squash it. My feeling is we have more examples of feature sets that grow over time (replication, view

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-10-02 15:21:48 +0200, Michael Banck wrote: I've attached a trivial patch for this, should it be added to the next commitfest? How about flipping the default for log_checkpoints instead? There really isn't a good reason for having it

[HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread MauMau
Hello, One user reported a hang problem with 9.4 beta2 on Windows. The PostgreSQL is 64-bit version. I couldn't find the cause, but want to solve the problem. Could you help with this? I heard that the user had run 16 concurrent psql sessions which executes INSERT and UPDATE statements,

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Andres Freund (and...@2ndquadrant.com) wrote: How about flipping the default for log_checkpoints instead? There really isn't a good reason for having it disabled by default. Yeah, I agree with this- it's extremely useful information and it's really

Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Andres Freund
On 2014-10-09 18:17:09 +0530, Amit Kapila wrote: On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas robertmh...@gmail.com wrote: On another point, I think it would be a good idea to rebase the bgreclaimer patch over what I committed, so that we have a clean patch against master to test with.

Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Amit Kapila
On Thu, Oct 9, 2014 at 7:31 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-09 18:17:09 +0530, Amit Kapila wrote: On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas robertmh...@gmail.com wrote: On another point, I think it would be a good idea to rebase the bgreclaimer patch over

Re: [HACKERS] Proposal for better support of time-varying timezone abbreviations

2014-10-09 Thread Chris Bandy
On Tue, Oct 7, 2014 at 5:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: typedef struct { ! char token[TOKMAXLEN + 1]; /* now always null-terminated */ char type; ! int32 value; } datetkn; Being entirely new to this code, now makes me think of the current timestamp. I think this word can

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-09 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/02/2014 03:20 AM, Kevin Grittner wrote: My only concern from the benchmarks is that it seemed like there was a statistically significant increase in planning time: unpatched plan time average: 0.450 ms patched plan time average: 0.536

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread Craig Ringer
On 10/09/2014 09:47 PM, MauMau wrote: I heard that the user had run 16 concurrent psql sessions which executes INSERT and UPDATE statements, which is a write-intensive stress test. He encountered the hang phenomenon twice, one of which occured several hours after the start of the test, and

Re: [HACKERS] Proposal for better support of time-varying timezone abbreviations

2014-10-09 Thread Tom Lane
Chris Bandy bandy.ch...@gmail.com writes: On Tue, Oct 7, 2014 at 5:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: + /* use strncmp so that we match truncated tokens */ result = strncmp(key, position-token, TOKMAXLEN); In your proposal you wanted to remove crufty code that deals with

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread Andres Freund
On 2014-10-09 22:47:48 +0900, MauMau wrote: Hello, One user reported a hang problem with 9.4 beta2 on Windows. The PostgreSQL is 64-bit version. I couldn't find the cause, but want to solve the problem. Could you help with this? I heard that the user had run 16 concurrent psql sessions

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-09 Thread Rahila Syed
Hello, Thank you for review. 1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. Full page write compression information can be stored in varlena struct of compressed blocks as done for toast data in pluggable compression support patch. If I

Re: [HACKERS] Corporate and Individual Contributor License Agreements (CLAs)

2014-10-09 Thread Joshua D. Drake
Arcadiy, You may want to refer them to the license itself which will make it very easy for them to understand why the Contributor License Agreement is not required: PostgreSQL is released under the PostgreSQL License, a liberal Open Source license, similar to the BSD or MIT licenses.

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Jeff Janes
On Wed, Oct 8, 2014 at 10:52 AM, Michael Banck michael.ba...@credativ.de wrote: Hi, Am Samstag, den 04.10.2014, 15:05 -0500 schrieb Jim Nasby: On 10/4/14, 1:21 PM, Jeff Janes wrote: On Thu, Oct 2, 2014 at 6:21 AM, Michael Banck wrote: we have seen repeatedly that users can be

[HACKERS] Expose options to explain? (track_io_timing)

2014-10-09 Thread Joshua D. Drake
Salut! Fellow volunteers, I request assistance in understanding the following: When I explain a query I can get the following information: | I/O Read Time: 0.000, | I/O Write Time: 0.000 I know why it is 0. My question is this, can we expose it to explain

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Marti Raudsepp
On Thu, Oct 2, 2014 at 4:21 PM, Michael Banck michael.ba...@credativ.de wrote: we have seen repeatedly that users can be confused about why PostgreSQL is not shutting down even though they requested it. Usually, this is because `log_checkpoints' is not enabled and the final checkpoint is

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Andres Freund
On 2014-10-09 09:44:09 -0400, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: * Andres Freund (and...@2ndquadrant.com) wrote: How about flipping the default for log_checkpoints instead? There really isn't a good reason for having it disabled by default. Yeah, I agree with this-

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-10-09 09:44:09 -0400, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: Yeah, I agree with this- it's extremely useful information and it's really not that verbose in general.. -1. Every time we've turned on default

Re: [HACKERS] alter user set local_preload_libraries.

2014-10-09 Thread Fujii Masao
On Mon, Sep 15, 2014 at 1:33 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On 9/1/14 7:51 AM, Kyotaro HORIGUCHI wrote: The attached patch simply changes the context for local_... to PGC_USERSET and edits the doc. I had this ready to commit, but then

Re: [HACKERS] replicating DROP commands across servers

2014-10-09 Thread Jim Nasby
On 10/6/14, 11:24 PM, Robert Haas wrote: Offlist. FWIW, I've run into situations more than once in userspace where I need a way to properly separate schema and object name. Generally I can make do using reg* casts and then hitting catalog tables, but it'd be nice if there was an easier way.

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-10-09 09:44:09 -0400, Tom Lane wrote: -1. Every time we've turned on default logging of routine events, there's been pushback and it was eventually turned off again as log spam. We're talking

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: For embedded devices and similar small-scale systems, I can see Tom's point. At the same time, I would expect those to require sufficient configuration that also setting log_checkpoints to 'off' wouldn't be a

Re: [HACKERS] replicating DROP commands across servers

2014-10-09 Thread Alvaro Herrera
Jim Nasby wrote: On 10/6/14, 11:24 PM, Robert Haas wrote: Offlist. FWIW, I've run into situations more than once in userspace where I need a way to properly separate schema and object name. Generally I can make do using reg* casts and then hitting catalog tables, but it'd be nice if there

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-09 Thread Jeff Janes
On Thu, Oct 9, 2014 at 10:17 AM, Joshua D. Drake j...@commandprompt.com wrote: Salut! Fellow volunteers, I request assistance in understanding the following: When I explain a query I can get the following information: | I/O Read Time: 0.000, | I/O Write

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Mon, Oct 6, 2014 at 11:53 AM, Peter Geoghegan p...@heroku.com wrote: On Mon, Oct 6, 2014 at 11:27 AM, Robert Haas robertmh...@gmail.com wrote: Well, really, I was just suggesting that I can spend more time on the patch, but not immediately. We haven't really talked about the idea of the

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Robert Haas
On Thu, Oct 9, 2014 at 8:20 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-09 08:18:18 -0400, Robert Haas wrote: On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund and...@2ndquadrant.com wrote: Interesting - in my local profile AtStart_Inval() is more pronounced than

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Heikki Linnakangas
On 10/09/2014 09:59 PM, Peter Geoghegan wrote: My concern is that if we only get it committed in the last commitfest, we may run out of time to make sortsupport work for B-Tree index builds. That's where the sortsupport for text stuff will be really useful. B-tree index build uses tuplesort.c.

Re: [HACKERS] pgbench throttling latency limit

2014-10-09 Thread Heikki Linnakangas
On 10/05/2014 10:43 AM, Fabien COELHO wrote: Hello Heikki, Here are new patches, again the first one is just refactoring, and the second one contains this feature. I'm planning to commit the first one shortly, and the second one later after people have had a chance to look at it. I looked

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 12:14 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: B-tree index build uses tuplesort.c. What's missing? I don't think that all that much is missing. Tuplesort expects to work with an index scankey when sorting B-Tree tuples. There needs to be something like a

Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Andres Freund
On 2014-10-09 16:01:55 +0200, Andres Freund wrote: On 2014-10-09 18:17:09 +0530, Amit Kapila wrote: On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas robertmh...@gmail.com wrote: On another point, I think it would be a good idea to rebase the bgreclaimer patch over what I committed, so that

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-09 15:01:19 -0400, Robert Haas wrote: On Thu, Oct 9, 2014 at 8:20 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-09 08:18:18 -0400, Robert Haas wrote: On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund and...@2ndquadrant.com wrote: Interesting - in my local profile

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread MauMau
From: Craig Ringer cr...@2ndquadrant.com It'd be interesting and useful to run this test on a debug build of PostgreSQL, i.e. one compiled against the debug version of the C library and with full debuginfo not just minimal .pdb. Although I'm not sure the user can do this now, I'll ask him

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 12:51 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Oh, I didn't realize we don't do that already! I'm surprised, I would've expected index build to have been the first thing we'd use the SortSupport stuff in. The thing is that the most compelling numbers for

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread MauMau
From: Andres Freund and...@2ndquadrant.com What precisely do you mean with Intel64? 64bit x86 or Itanium? 64-bit x86, i.e. x86-64. Also, what's the precise workload? Can you reproduce the problem? IIUC, each client inserts 1000 records into one table, then repeats updating all those

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-09 Thread Kevin Grittner
Tomas Vondra t...@fuzzy.cz wrote: On 9.10.2014 16:55, Kevin Grittner wrote: I've tried various other tests using \timing rather than EXPLAIN, and the patched version looks even better in those cases. I have seen up to 4x the performance for a query using the patched version, higher

Re: [HACKERS] pg_upgrade, locale and encoding

2014-10-09 Thread Bruce Momjian
On Tue, Oct 7, 2014 at 03:52:24PM +0300, Heikki Linnakangas wrote: While looking at bug #11431, I noticed that pg_upgrade still seems to think that encoding and locale are cluster-wide properties. We got per-database locale support in 8.4, and encoding has been per-database much longer than

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Bruce Momjian
On Thu, Oct 9, 2014 at 02:34:17PM +0400, Alexey Bashtanov wrote: Hello! Autovacuum daemon performs vacuum when the number of rows updated/deleted (n_dead_tuples) reaches some threshold. Similarly it performs analyze when the number of rows changed in any way (incl. inserted). When a table

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Robert Haas
On Thu, Oct 9, 2014 at 3:53 PM, Andres Freund and...@2ndquadrant.com wrote: OK, here's an attempt at a real patch for that. I haven't perf-tested this. Neato. With a really trivial SELECT: before: tps = 28150.794776 (excluding connections establishing) after: tps = 29978.767703 (excluding

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alvaro Herrera
Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has: https://wiki.postgresql.org/wiki/Todo Improve setting of visibility map bits for read-only and insert-only workloads

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-09 17:02:02 -0400, Robert Haas wrote: On Thu, Oct 9, 2014 at 3:53 PM, Andres Freund and...@2ndquadrant.com wrote: OK, here's an attempt at a real patch for that. I haven't perf-tested this. Neato. With a really trivial SELECT: before: tps = 28150.794776 (excluding

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Andres Freund
On 2014-10-09 18:03:00 -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has: https://wiki.postgresql.org/wiki/Todo Improve setting of visibility map

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Kevin Grittner
Alvaro Herrera alvhe...@2ndquadrant.com wrote: Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has: https://wiki.postgresql.org/wiki/Todo Improve setting of visibility map bits for

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alvaro Herrera
Andres Freund wrote: On 2014-10-09 18:03:00 -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has: https://wiki.postgresql.org/wiki/Todo Improve

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alvaro Herrera
Kevin Grittner wrote: Wouldn't we get substantially the same thing just by counting tuple inserts toward the autovacuum vacuum threshold? I mean, it unless the table is due for wraparound prevention autovacuum, it will only visit pages that don't have the all-visible bit set, right? And

Re: [HACKERS] schema-only -n option in pg_restore fails

2014-10-09 Thread Josh Berkus
All, Crossing this over to -hackers because it's stopped being a bug and is now a TODO item. See below. For those not on pgsql-bugs, I've quoted the full bug report below my proposal. On 10/09/2014 12:36 PM, Josh Berkus wrote: Summary: pg_restore -n attempts to restore objects to pg_catalog

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Andres Freund
On 2014-10-09 18:16:46 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2014-10-09 18:03:00 -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has:

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Jim Nasby
On 10/8/14, 8:35 AM, Andres Freund wrote: +#define EXCLUSIVE_LOCK (((uint32) 1) (31 - 1)) + +/* Must be greater than MAX_BACKENDS - which is 2^23-1, so we're fine. */ +#define SHARED_LOCK_MASK (~EXCLUSIVE_LOCK) There should at least be a comment where we define MAX_BACKENDS about the

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Andres Freund
On 2014-10-09 16:52:46 -0500, Jim Nasby wrote: On 10/8/14, 8:35 AM, Andres Freund wrote: +#define EXCLUSIVE_LOCK (((uint32) 1) (31 - 1)) + +/* Must be greater than MAX_BACKENDS - which is 2^23-1, so we're fine. */ +#define SHARED_LOCK_MASK (~EXCLUSIVE_LOCK) There should at least be a

[HACKERS] Build (definition?) errors - in bootstrap

2014-10-09 Thread Lou Picciano
Having just git pulled from orgin/master: $ ./configure $ make Mileage: ... make -C bootstrap all make[3]: Entering directory `/path/to/postgresql/src/backend/bootstrap' gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute

[HACKERS] Obsolete reference to _bt_tuplecompare() within tuplesort.c

2014-10-09 Thread Peter Geoghegan
I found a reference made obsolete by commit 9e85183b, which is from way back in 2000. comparetup_index_btree() says: /* * This is similar to _bt_tuplecompare(), but we have already done the * index_getattr calls for the first column, and we need to keep track of * whether any null fields are

Re: [HACKERS] Build (definition?) errors - in bootstrap

2014-10-09 Thread Tom Lane
Lou Picciano loupicci...@comcast.net writes: Having just git pulled from orgin/master: $ ./configure $ make [ fails ] The buildfarm doesn't seem unhappy, so I doubt there's anything wrong with the code as such. Try make clean or even make distclean and rebuild. Also, if your computer's

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Jim Nasby
On 10/9/14, 4:57 PM, Andres Freund wrote: If you modify either, you better grep for them... I don't think that's going to happen anyway. Requiring it during startup would mean exposing SHARED_LOCK_MASK outside of lwlock.c which'd be ugly. We could possibly stick a StaticAssert() someplace in

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Jim Nasby
On 10/9/14, 4:03 PM, Alvaro Herrera wrote: Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has: https://wiki.postgresql.org/wiki/Todo Improve setting of visibility map bits for

Re: [HACKERS] Build (definition?) errors - in bootstrap

2014-10-09 Thread Michael Paquier
On Fri, Oct 10, 2014 at 8:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lou Picciano loupicci...@comcast.net writes: Having just git pulled from orgin/master: $ ./configure $ make [ fails ] The buildfarm doesn't seem unhappy, so I doubt there's anything wrong with the code as such. Try make

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-09 Thread Jim Nasby
On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittnerkgri...@ymail.com wrote: Although the last go-around does suggest that there is at least one point of difference on the semantics. You seem to want to fire the BEFORE INSERT triggers before determining

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-09 Thread Gavin Flower
On 10/10/14 12:38, Jim Nasby wrote: On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittnerkgri...@ymail.com wrote: Although the last go-around does suggest that there is at least one point of difference on the semantics. You seem to want to fire the BEFORE

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:13 PM, Peter Geoghegan p...@heroku.com wrote: Can you write a separate patch to use SortSupport for B-tree index builds, please? Eliminating the FunctionCallInfoData overhead should shave off some some cycles from every index build. I'll look into it. Hopefully an

Re: [HACKERS] schema-only -n option in pg_restore fails

2014-10-09 Thread Erik Rijkers
On Thu, October 9, 2014 23:19, Josh Berkus wrote: All, [dump/restore -n bug] Perhaps this (from five years ago) can be fixed too (esp. if only a doc-fix): http://www.postgresql.org/message-id/4833.156.83.1.81.1240955642.squir...@webmail.xs4all.nl It's not the same problem but also a

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Amit Kapila
On Wed, Oct 8, 2014 at 7:05 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, Attached you can find the next version of my LW_SHARED patchset. Now that atomics are committed, it seems like a good idea to also add their raison d'ĂȘtre. Since the last public version I have: * Addressed

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread Craig Ringer
On 10/10/2014 04:16 AM, MauMau wrote: From: Craig Ringer cr...@2ndquadrant.com It'd be interesting and useful to run this test on a debug build of PostgreSQL, i.e. one compiled against the debug version of the C library and with full debuginfo not just minimal .pdb. Although I'm not sure