Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Guillaume Lelarge
2014-12-30 18:45 GMT+01:00 Jeff Janes : > On Tue, Dec 30, 2014 at 12:35 AM, Guillaume Lelarge < > guilla...@lelarge.info> wrote: > >> Sorry for my very late answer. It's been a tough month. >> >> 2014-11-27 0:00 GMT+01:00 Bruce Momjian : >> >>> On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-30 Thread Guillaume Lelarge
2014-12-12 14:58 GMT+01:00 Heikki Linnakangas : > On 12/10/2014 04:32 PM, Dennis Kögel wrote: > >> Hi, >> >> Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais < >> j...@dalibo.com>: >> >>> Since few months, we occasionally see .ready files appearing on some >>> slave >>> instances from va

Re: [HACKERS] orangutan seizes up during isolation-check

2014-12-30 Thread Robert Haas
On Sun, Dec 28, 2014 at 4:58 PM, Noah Misch wrote: > I wondered whether to downgrade FATAL to LOG in back branches. Introducing a > new reason to block startup is disruptive for a minor release, but having the > postmaster deadlock at an unpredictable later time is even more disruptive. I > am i

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-12-30 Thread Robert Haas
On Fri, Dec 26, 2014 at 7:57 AM, David Rowley wrote: > 1. Do we need to keep the 128 byte aggregate state size for machines without > 128 bit ints? This has been reduced to 48 bytes in the patch, which is in > favour code being compiled with a compiler which has 128 bit ints. I kind > of think th

Re: [HACKERS] Additional role attributes && superuser review

2014-12-30 Thread Amit Kapila
On Tue, Dec 30, 2014 at 6:35 PM, Stephen Frost wrote: > * Amit Kapila (amit.kapil...@gmail.com) wrote: > > On Tue, Dec 30, 2014 at 6:52 AM, Stephen Frost wrote: > > > another could be have a separate privilege (DBA) or a role which is > > not a superuser, however can be used to perform such tasks

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-30 Thread Peter Geoghegan
On Mon, Dec 29, 2014 at 11:52 PM, Jeff Janes wrote: > Correct, I haven't seen any problems with approach #1 That helps with debugging #2, then. That's very helpful. > Generally the problem will occur early on in the process, and if not then it > will not occur at all. I think that is because th

Re: [HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-30 Thread Alvaro Herrera
Michael Paquier wrote: > HI all. > > markhor has run for the first time in 8 days, and there is something > in range e703261..72dd233 making the regression test of brin crashing. > See here: > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=markhor&dt=2014-12-30%2020%3A58%3A49 This shows t

Re: [HACKERS] POLA violation with \c service=

2014-12-30 Thread David Fetter
On Wed, Dec 17, 2014 at 08:14:04AM -0500, Andrew Dunstan wrote: > > On 12/17/2014 04:11 AM, Heikki Linnakangas wrote: > >On 12/17/2014 10:03 AM, Albe Laurenz wrote: > >>David Fetter wrote: > >>>I've noticed that psql's \c function handles service= requests in a > >>>way that I can only characteri

[HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-30 Thread Michael Paquier
HI all. markhor has run for the first time in 8 days, and there is something in range e703261..72dd233 making the regression test of brin crashing. See here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=markhor&dt=2014-12-30%2020%3A58%3A49 Regards, -- Michael -- Sent via pgsql-hacker

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_event_trigger_dropped_objects: Add name/args output columns

2014-12-30 Thread Tom Lane
Alvaro Herrera writes: > Alvaro Herrera wrote: >> pg_event_trigger_dropped_objects: Add name/args output columns > This is causing buildfarm member dunlin to fail: > ... > No other member so far has reported a problem here. Not sure if that's > the strangest bit, or the fact that dunlin is repor

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread Andrew Dunstan
On 12/30/2014 09:20 AM, Tom Lane wrote: Bernd Helmle writes: --On 29. Dezember 2014 12:55:11 -0500 Tom Lane wrote: Given the lack of previous complaints, this probably isn't backpatching material, but it sure seems like a bit of attention to consistency would be warranted here. Now that i r

[HACKERS] Re: [COMMITTERS] pgsql: pg_event_trigger_dropped_objects: Add name/args output columns

2014-12-30 Thread Alvaro Herrera
Alvaro Herrera wrote: > pg_event_trigger_dropped_objects: Add name/args output columns > > These columns can be passed to pg_get_object_address() and used to > reconstruct the dropped objects identities in a remote server containing > similar objects, so that the drop can be replicated. This is c

Re: [HACKERS] pg_ctl {start,restart,reload} bad handling of stdout file descriptor

2014-12-30 Thread Tom Lane
Luis Menina writes: > I've been trying to run some pg_ctl command inside a python script, > and saw that some of them where deadlocking. It seems that the > commands that start postgres handle stdout in a way that that block > the caller. Redirecting stdout to /dev/null or to a file using the -l >

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Peter Geoghegan
On Tue, Dec 30, 2014 at 12:21 PM, Heikki Linnakangas wrote: > In practice, we never store any actual key value for the "minus infinity" > key. I guess the code would ignore it if it was there, but it would make > more sense to explain that the first data key on an internal page does not > have a k

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Heikki Linnakangas
On 12/30/2014 10:07 PM, Peter Geoghegan wrote: On Tue, Dec 30, 2014 at 8:59 AM, Heikki Linnakangas wrote: How much detail on the b-tree internals do we want to put in the pageinspect documentation? I can see that being useful, but should we also explain e.g. that the first item on each (non-rig

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Peter Geoghegan
On Tue, Dec 30, 2014 at 8:59 AM, Heikki Linnakangas wrote: > How much detail on the b-tree internals do we want to put in the pageinspect > documentation? I can see that being useful, but should we also explain e.g. > that the first item on each (non-rightmost) page is the high key? Maybe we shou

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Jeff Janes
On Tue, Dec 30, 2014 at 12:35 AM, Guillaume Lelarge wrote: > Sorry for my very late answer. It's been a tough month. > > 2014-11-27 0:00 GMT+01:00 Bruce Momjian : > >> On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes wrote: >> > It looked to me that the formula, when descending from a previou

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-30 Thread Andres Freund
On 2014-12-30 21:36:19 +0530, Abhijit Menon-Sen wrote: > Thanks for spotting that. I had meant to change the test to "& 7". But > again, now that you mention it, I'm not sure it's necessary. The CRC32* > instructions don't have the usual SSE alignment requirements, and I see > that the Linux kernel

Re: [HACKERS] BUG #12330: ACID is broken for unique constraints

2014-12-30 Thread Greg Stark
On Mon, Dec 29, 2014 at 4:17 PM, Merlin Moncure wrote: > Serialization errors only exist as a concession > to concurrency and performance. Again, they should be returned as > sparsely as possible I think this is fuzzy thinking. Serialization *errors* themselves are a concession to concurrency a

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Heikki Linnakangas
On 12/30/2014 04:08 AM, Peter Geoghegan wrote: Attached documentation patch describes the purpose of bt_page_items()'s ctid field. This has come up enough times in disaster recovery or testing scenarios that I feel it's worth drawing particular attention to. How much detail on the b-tree intern

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-12-30 Thread Jeff Janes
On Sun, Dec 28, 2014 at 12:45 PM, Peter Geoghegan wrote: > On Sun, Dec 28, 2014 at 12:37 PM, Jeff Davis wrote: > > Do others have similar numbers? I'm quite surprised at how little > > work_mem seems to matter for these plans (HashJoin might be a different > > story though). I feel like I made a

Re: [HACKERS] pg_ctl {start,restart,reload} bad handling of stdout file descriptor

2014-12-30 Thread Luis Menina
I knew I'd forget something... I'm running Postgres 9.3.5 on Debian/Linux. Regards, -- Luis MENINA / Software Engineer Web: www.anevia.com Anevia: 1 rue René Anjolvy, 94250 Gentilly, France -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscriptio

[HACKERS] pg_ctl {start,restart,reload} bad handling of stdout file descriptor

2014-12-30 Thread Luis Menina
Hi, I've been trying to run some pg_ctl command inside a python script, and saw that some of them where deadlocking. It seems that the commands that start postgres handle stdout in a way that that block the caller. Redirecting stdout to /dev/null or to a file using the -l option allow me to workar

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread David Johnston
On Tue, Dec 30, 2014 at 8:54 AM, Adrian Klaver wrote: > On 12/30/2014 07:43 AM, David G Johnston wrote: > >> Tom Lane-2 wrote >> >>> Bernd Helmle < >>> >> >> mailings@ >>> >> >> > writes: >>> --On 29. Dezember 2014 12:55:11 -0500 Tom Lane < >>> >> tgl@.pa >>> >> >> > wrote: >>>

Re: [HACKERS] BUG #12330: ACID is broken for unique constraints

2014-12-30 Thread Merlin Moncure
On Mon, Dec 29, 2014 at 3:53 PM, Kevin Grittner wrote: >> I tend to build out applications on top of functions and the >> inability to set isolation mode inside a function confounds me >> from using anything but 'read committed'. > > Hey, no problem -- just set default_transaction_isolation = > 's

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-30 Thread Abhijit Menon-Sen
At 2014-12-30 16:05:50 +0200, hlinnakan...@vmware.com wrote: > > A couple of quick comments: > > bswap32 is unused on on little-endian systems. That will give a > compiler warning. Huh. I don't get a warning, even when I add -Wunused to the build flags. But since you mention it, it would be bette

Re: [HACKERS] Detecting backend failures via libpq / DBD::Pg

2014-12-30 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Andrew Gierth asked: >> this is to send a simple SELECT via PQexec > > Why not PQexec(conn, "") ? Because I want to leave a good clue for debugging; so DBAs are better able to figure out where a mystery slew of queries is coming from. The que

[HACKERS] Re[2]: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Alexey Vasiliev
Tue, 30 Dec 2014 21:39:33 +0900 от Michael Paquier : > On Tue, Dec 30, 2014 at 9:33 PM, Michael Paquier > wrote: > > On Tue, Dec 30, 2014 at 9:10 PM, Alexey Vasiliev > > wrote: > >> To not modify current pg_usleep calculation, I changed > >> restore_command_retry_interval value to seconds (not m

Re: [HACKERS] Additional role attributes && superuser review

2014-12-30 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Dec 29, 2014 at 11:01 PM, Stephen Frost wrote: > > That said, a 'DUMP' privilege which allows the user to dump the contents > > of the entire database is entirely reasonable. We need to be clear in > > the documentation though- such a 'DUMP

Re: [HACKERS] Serialization exception : Who else was involved?

2014-12-30 Thread Olivier MATROT
Indeed, NOTICE is wrong because it would doom the transaction that sets the flag if it should be later PREPARED. I think that reporting the PIDs and the current activity of each process would be nice. DeadLoackReport() is using pgstat_get_backend_current_activity() to get the process activity.

Re: [HACKERS] Additional role attributes && superuser review

2014-12-30 Thread Magnus Hagander
On Mon, Dec 29, 2014 at 11:01 PM, Stephen Frost wrote: > > * Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > > > I'd suggest it's called DUMP if that's what it allows, to keep it > separate > > > from the backup parts. > > > > Makes sense to me. > > I'm fine calling it 'DUMP',

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread Tom Lane
Bernd Helmle writes: > --On 29. Dezember 2014 12:55:11 -0500 Tom Lane wrote: >> Given the lack of previous complaints, this probably isn't backpatching >> material, but it sure seems like a bit of attention to consistency >> would be warranted here. > Now that i read it i remember a client compl

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-30 Thread Heikki Linnakangas
On 12/30/2014 09:40 AM, Abhijit Menon-Sen wrote: Hi. I'm re-attaching the two patches as produced by format-patch. I haven't listed any reviewers. It's either just Andres, or maybe a lot of people. Is anyone in a position to try out the patches on MSVC and see if they build and work sensibly, p

Re: [HACKERS] Additional role attributes && superuser review

2014-12-30 Thread Stephen Frost
* Amit Kapila (amit.kapil...@gmail.com) wrote: > On Tue, Dec 30, 2014 at 6:52 AM, Stephen Frost wrote: > > > >There is one issue that occurs to me, however. We're talking about > > > >pg_dump, but what about pg_dumpall? In particular, I don't think the > > > >DUMP privilege should provide access

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 9:33 PM, Michael Paquier wrote: > On Tue, Dec 30, 2014 at 9:10 PM, Alexey Vasiliev wrote: >> To not modify current pg_usleep calculation, I changed >> restore_command_retry_interval value to seconds (not milliseconds). In this >> case, min value - 1 second. > Er, what the

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread Bernd Helmle
--On 29. Dezember 2014 12:55:11 -0500 Tom Lane wrote: Given the lack of previous complaints, this probably isn't backpatching material, but it sure seems like a bit of attention to consistency would be warranted here. Now that i read it i remember a client complaining about this some time

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 9:10 PM, Alexey Vasiliev wrote: > To not modify current pg_usleep calculation, I changed > restore_command_retry_interval value to seconds (not milliseconds). In this > case, min value - 1 second. Er, what the problem with not changing 100L to 1000L? The unit of your pa

Re: [HACKERS] Compression of full-page-writes

2014-12-30 Thread Andres Freund
On 2014-12-30 21:23:38 +0900, Michael Paquier wrote: > On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis wrote: > > On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: > >> Speeding up the CRC calculation obviously won't help with the WAL volume > >> per se, ie. you still generate the same amou

Re: [HACKERS] Compression of full-page-writes

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis wrote: > On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: >> Speeding up the CRC calculation obviously won't help with the WAL volume >> per se, ie. you still generate the same amount of WAL that needs to be >> shipped in replication. But the

[HACKERS] Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Alexey Vasiliev
Hello. Thanks, I understand, what look in another part of code. Hope right now I did right changes. To not modify current pg_usleep calculation, I changed  restore_command_retry_interval value to seconds (not milliseconds). In this case, min value - 1 second. Mon, 29 Dec 2014 00:15:03 +0900

Re: [HACKERS] Coverity and pgbench

2014-12-30 Thread Tatsuo Ishii
> Just wondering, where are those reports located? It would have been > good to point out where the leaks actually were.. The report is a closed one. You need to be a member of "PostgreSQL project" of Coverity to look into the report. If want to join the project, you need to contact to one of admi

Re: [HACKERS] Coverity and pgbench

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 8:39 PM, Tatsuo Ishii wrote: >> Anybody looks into problems in pgbench pointed out by Coverity? If no, >> I would like to work on fixing them because I need to write patches >> for "-f option" related issues anyway. > > Done. Just wondering, where are those reports located?

Re: [HACKERS] Coverity and pgbench

2014-12-30 Thread Tatsuo Ishii
> Anybody looks into problems in pgbench pointed out by Coverity? If no, > I would like to work on fixing them because I need to write patches > for "-f option" related issues anyway. Done. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:ht

Re: [HACKERS] Compression of full-page-writes

2014-12-30 Thread Jeff Davis
On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: > Speeding up the CRC calculation obviously won't help with the WAL volume > per se, ie. you still generate the same amount of WAL that needs to be > shipped in replication. But then again, if all you want to do is to > reduce the volu

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Guillaume Lelarge
Sorry for my very late answer. It's been a tough month. 2014-11-27 0:00 GMT+01:00 Bruce Momjian : > On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes wrote: > > It looked to me that the formula, when descending from a previously > stressed > > state, would be: > > > > greatest(1 + checkpoint_c

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-12-30 Thread Michael Paquier
On Tue, Dec 23, 2014 at 5:54 PM, Oskari Saarenmaa wrote: > > If the short-lived lock is the only blocker for this feature at the > moment could we just require an additional qualifier for CONCURRENTLY > (FORCE?) until the lock can be removed, something like: > =# [blah] FWIW, I'd just keep only C